What is that clicking noise near fuse box with engine on or off

Hello, That is a relay that is closing and opening. When it closes, it draws power, when it opens it stops drawing power. That's why the lights etc are dimming. You will need to identify which one it is by placing your hand on the relays and waiting for it to click. You'll feel it. (in the fuse panel) Once you identify the relay, check the terminals it plugs into and look for signs of physical and electrical damage. It might just be that the relay should be on all the time with the key turned on, so you might just have a relay that is failing.