The Subaru Coronagraphic Extreme Adaptive Optics (SCExAO) instrument, under development for the Subaru Telescope, has currently the fastest on-sky wavefront control loop, with a pyramid wavefront sensor running at 3.5 kHz. But even at that speed, we are still limited by low-frequency vibrations. The current main limitation was found to be vibrations attributed mainly to the rotation of the telescope. Using the fast wavefront sensors, cameras and accelerometers, we managed to identify the origin of most of the vibrations degrading our performance. Low-frequency vibrations are coming from the telescope drive in azimuth and elevation, as well as the elevation encoders when the target is at transit. Other vibrations were found at higher frequency coming from the image rotator inside Subaru's adaptive optics facility AO188. Different approaches are being implemented to take care of these issues. The PID control of the image rotator has been tuned to reduce their high-frequency contribution. We are working with the telescope team to tune the motor drives and reduce the impact of the elevation encoder. A Linear Quadratic Gaussian controller (LQG, or Kalman filter) is also being implemented inside SCExAO to control these vibrations. These solutions will not only improve significantly SCExAOs performance, but will also help all the other instruments on the Subaru Telescope, especially the ones behind AO188. Ultimately, this study will also help the development of the TMT, as these two telescopes share very similar drives.