As you drive along a flat straight level road (where safe) pull up the hand brake a few notches just to lightly engage the hand brake, if you feel the vibration (and are not applying the brake pedal) you have isolated the problem to the rear.
Jack up, support and remove the brake drums, reinspect that everything is assembled correctly. Look at the brake shoe material and see that you have even contact, make sure there is no oil or grease contamination on any of the brake shoes.Measure the brake drums ensure they are "in round" and above minimum diameter specification. Perhaps index the brake drums before taking them off, move the brake drums 90 degrees, if the vibration on braking doesn't go away move the brake drums another 90 degrees. I work on some old vintage race cars with rear drum brakes, sometimes the brake drum and carrier hub needed to be indexed and "re-assembled as they came off" to avoid vibration after rear brake work.