is there a fuse for the panic button

I would guess that you have an aftyermarket alarm added on to the factory keyless entry remote. Look under the dash on the drivers side to see if an add-on alarm module and wiring is installed. Many car dealerships installed these even without the customer knowing. Your problem most likely lies there.