OpenMV AE3 Achieves Hardware Synchronization Between IMU and Camera to Improve VIO Accuracy

The accuracy of visual-inertial odometry (VIO) depends heavily on the quality of time synchronization between the camera and IMU. A project recently shared in the Reddit community uses the OpenMV AE3—a compact camera integrating a global shutter sensor and IMU—to achieve hardware-level synchronization between camera exposure and specific IMU measurements by modifying firmware and writing Python scripts, offering a new approach for low-cost VIO solutions.

Hardware Synchronization: The Key to VIO Accuracy

VIO estimates motion trajectories by fusing camera images and IMU data. If there is a timestamp offset between the two, the fusion algorithm introduces errors, especially in fast-motion scenarios. The project cites the paper "A Synchronized Visual-Inertial Sensor System with FPGA Pre-Processing for Accurate Real-Time SLAM" by Nikolic et al., published at ICRA 2014, which points out that hardware synchronization can significantly improve SLAM accuracy.

The OpenMV AE3 itself integrates a global shutter sensor and an IMU and supports Python programming, which facilitates custom synchronization logic. Instead of relying on software timestamp alignment, the author took a firmware-level approach to make the two sensors share a clock.

Visual-inertial odometry (VIO) is a technique that estimates a carrier's motion trajectory by fusing camera images and inertial measurement unit (IMU) data. The camera provides rich environmental texture and structural information, but its frame rate is low and it is susceptible to lighting conditions; the IMU provides high-frequency angular velocity and acceleration measurements, but suffers from drift. By complementarily fusing the two, real-time, high-precision pose estimation can be achieved in GPS-denied environments, and it is widely used in drones, robots, and AR/VR devices. VIO accuracy is highly dependent on time synchronization between sensors: if there is a timestamp offset between images and IMU data, the fusion algorithm incorrectly associates motion states from different moments, leading to accumulated estimation errors, especially during fast motion or violent rotation. Therefore, hardware-level synchronization (sharing a clock or trigger signal) becomes a key means of improving VIO performance.

Implementation: Shared Clock and Trigger Logic

The author modified the firmware of the OpenMV AE3 to add a shared clock between the two CPU cores. The IMU runs at 208 Hz, and camera exposure is set to trigger on every 10th IMU measurement, resulting in an image frame rate of about 20 FPS.

This triggering method ensures that each image frame corresponds to a determined IMU measurement moment, eliminating the jitter that software timestamps may introduce. The code is open-sourced on GitHub (repository: chobitsfan/openmv_img_imu), including scripts and firmware modification instructions.

The difference between a global shutter sensor and a rolling shutter sensor lies in the exposure method: a rolling shutter exposes row by row, producing a jelly effect during high-speed motion that causes geometric distortion in the image; a global shutter exposes all pixels simultaneously, accurately capturing fast-motion scenes, making it more suitable for VIO and SLAM applications. The global shutter sensor used in the OpenMV AE3 can avoid motion distortion, but global shutter sensors are usually more expensive. In addition, the IMU's 208 Hz operating frequency means a measurement is produced approximately every 4.8 milliseconds, while camera exposure is triggered every 10 IMU measurements, i.e., approximately every 48 milliseconds, corresponding to a frame rate of about 20.8 FPS. This integer-multiple frequency-division triggering method ensures strict correspondence between each image frame and a specific IMU measurement moment, eliminating the jitter and uncertainty of software timestamps.

Practical Verification: Indoor and Outdoor Flight Tests

The author used Kimera-VIO as the back-end fusion algorithm and conducted flight tests in outdoor and indoor scenarios. The outdoor flight speed reached 3 m/s, while the indoor flight demonstrated performance in confined spaces. The test video was published on YouTube, indirectly verifying the feasibility of the synchronization scheme in actual dynamic scenarios.

Although the material does not provide quantitative accuracy comparison data, the author's choice of Kimera-VIO for closed-loop testing indicates that the synchronization mechanism is sufficient to support the operation of a complete VIO pipeline.

Kimera-VIO is a visual-inertial odometry pipeline open-sourced by the MIT SPARK Lab, including modules such as IMU preintegration, feature tracking, sliding window optimization, and loop closure detection. It supports fusion of monocular or stereo cameras with an IMU and outputs high-frequency pose estimation and sparse 3D maps. Kimera-VIO is relatively sensitive to time synchronization and is often used as a benchmark algorithm for evaluating hardware synchronization effects. In this project, the author chose Kimera-VIO as the back-end fusion algorithm for indoor and outdoor flight tests, indicating that the synchronization scheme can meet the input requirements of a complete VIO pipeline. The outdoor flight speed reached 3 m/s, which belongs to medium dynamic scenarios and imposes certain requirements on time synchronization accuracy; indoor confined spaces test the algorithm's robustness under fast rotation and close-range textures.

Implications for Low-Cost VIO Development

The OpenMV AE3 is a relatively affordable embedded vision module, and combined with the open-source code, developers can reproduce this synchronization scheme. For applications such as drones and robots that require real-time pose estimation, a hardware-synchronized VIO solution can reduce reliance on high-cost sensors.

However, this scheme is still experimental, and firmware modification requires some embedded development experience. The author did not mention long-term stability test results in the Reddit post, so performance under different lighting and motion conditions still needs to be evaluated before actual deployment.

Conclusion

This project demonstrates how firmware-level modifications can achieve hardware synchronization between IMU and images on resource-constrained embedded devices, providing a reproducible low-cost solution for VIO researchers and enthusiasts. As open-source flight controllers and VIO algorithms mature, such hardware synchronization practices are expected to promote the deployment of more accurate visual-inertial systems on small robots.