You say the speeds don't change in manual mode. Is that full manual, or shutter/aperture priority.
Try each of the modes and see when the speeds change. If they change in full auto and aperture priority but not in full manual and shutter priority then chances are it's a bad contact on either the mode or shutter speed switch.