Replaced Battery, Warning Light Still On

What is your question?

I was driving home and noticed my battery warning light was on. My accessories started to dim so I shut everything off and made it home safely. Once I tried to start my car again it would not start. First I tried to give my car a jump but no luck. Next went to purchase a new battery and once installed my car started no problem. I did notice the battery warning light was still on, flickering. After driving a short distance I noticed the battery warning light was steadily on. While my car was idle, I accelerated and noticed the battery warning light would go off once I reached 4RPM or higher. I have examined my alternator belt and don't find it to be lose or cracked so I am not sure what is causing my battery light to still be on as I know my alternator is responsible for charging. I am hoping to find a solution to my problem.