If my car happens to stall while taking off from a stop, it's hard to get my car restarted or if I park my car and then try to start it, it doesn't start right away. I'm wondering if it could be the "relay switch"?

Start by checking your battery - is it fully charged and able to handle a load? Following this, check all cabling to the battery, starter and ground to ensure all is clean and secure. When you say it won't start right away, do you mean the engine cranks but will not start, or the engine does not crank at all?