when should fuel pump come on?

ok, well my problem currently is when i fuel up the car will turn over but not start. You must wait a few minutes before it starts then it will run rough for about 30 seconds then the car runs great. There is no in line fuel filter to replace and no fuel pressure check valves either. Not sure if a faulty gas cap will cause this or something else. I tried another little test yesterday, with the car running i removed the gas cap and the engine stalled. Thanks