My car has a heavy brake pedal is the brake booster a problem?

What is your question?

Our 2001 Lexus has had a heavy pedal since we purchased the car. I just completed a front brake job with new rotors, pads and calipers. It still requires a lot of pedal pressure to stop the car. The pedal does go down more and easier when the engine is running vs. the engine off. I noticed the idle does go up if I pump the brakes repeatability. Is this a problem? Any suggestions for testing the booster or related components?