This may be a difficult problem to locate. It sounds like there is a poor electrical connection in the circuit for the speedometer. There is a speed sensor on the transmission, the speedometer and the wiring in between them and the problem can be with any of these.
The Vehicle Speed Sensor frequently failed on these vehicles. It is located on the top of the transmission (toward the rear), is black in color, and secured with two bolts (10mm heads). With the hood open, the VSS is visible looking downward into the engine compartment from the right front strut tower.The speed sensor is a "pulse generator" and sends a signal that operates both the speedometer and cruise control. Check to see if your cruise control (if fitted) works. If cruise control works the Vehicle Speed Sensor is working. I have seen intermittent failure of the speed sensor but have rarely seen the speedometer instrument to fail.
You said you experience speedometer failure while driving over rough roads, ensure the wires and connections at the Vehicle Speed Sensor are in tact and make a good connection.