DIY iToF+RGB Rig: First Metric Room Map and Gaussian Splatting

A maker from reddit (also the video creator) shared his homemade RGB-D capture rig: pairing a 320×240 epc660-class iToF (indirect time-of-flight) engineering sample with a Raspberry Pi camera, he produced the first colored room point cloud map with true metric scale, and trained a Gaussian splat on it. Interestingly, the entire pipeline is still offline, but it already produces usable 3D reconstruction results.

Hardware Approach: Engineering Sample + Raspberry Pi Camera

The core of this rig is a 320×240 resolution iToF sensor engineering sample. The iToF (indirect time-of-flight) approach obtains depth by measuring the phase difference of modulated light; its advantages are compactness and controllable cost, but resolution is typically low. The author paired it with a Raspberry Pi camera to provide higher-resolution RGB texture, and after time synchronization the two output "color depth" — that is, each pixel has both a color and a distance value.

The author emphasizes that this is not a review of a finished depth camera, but a progress log for a self-built experimental platform. The hardware choices lean toward low cost and reproducibility, suitable for a home lab environment.

The difference between iToF and dToF: Indirect time-of-flight (iToF) and direct time-of-flight (dToF) are two mainstream depth sensing approaches. dToF obtains distance by directly measuring the flight time of light pulses (e.g., single-photon avalanche diode SPAD timing), offering high precision and long range, but with complex hardware and higher cost; iToF instead emits continuously modulated sinusoidal light and infers distance by measuring the phase difference between reflected and emitted light. Because phase is periodic, iToF has a maximum unambiguous range (determined by the modulation frequency) and is more sensitive to multipath interference and low-reflectivity surfaces, but its sensor structure is simple and can be manufactured on standard CMOS processes, making it low-cost and compact — suitable for consumer and embedded applications. The epc660 in the article belongs to the iToF class of sensors, and its 320×240 resolution is on the low end among depth sensors, which is also why the article later emphasizes edge error and noise modeling.

Offline Mapping Pipeline: From RGB Poses to Gaussian Splatting

The pipeline is divided into several key steps: first recover camera poses from recorded RGB frames, then use the iToF depth readings to provide metric scale, then fuse color and depth, and finally train a Gaussian splatting model in the same coordinate system.

Gaussian splatting is a popular representation method in recent 3D reconstruction, modeling the scene with a set of anisotropic Gaussian ellipsoids, offering high rendering quality and relatively friendly training speed. The author's innovation lies in injecting the absolute scale information of the low-resolution iToF into RGB-based pose estimation, thereby avoiding the scale ambiguity problem of monocular vision.

Multi-view Voxel Voting Denoising

In this capture, the author introduced multi-view voxel voting to clean outliers from the depth map, removing 27,675 floating points in one pass. Such floating points typically come from depth sensor noise or erroneous estimates at edges, and are especially pronounced on low-resolution iToF. Voxel voting works by counting, in 3D space, the consistency with which each voxel is observed from multiple viewpoints; points below a threshold are discarded as noise.

Calibration Matters More Than Rendering

The author repeatedly emphasizes one point: the value of calibration work exceeds that of rendering quality itself. In testing, the median edge alignment error dropped from 2.2 pixels to 1.0 pixel, which directly determines the accuracy of color-depth alignment. Overflow and saturation of bright objects still require conditional handling — iToF sensors are prone to overexposure or crosstalk on highly reflective surfaces, causing depth readings to drift.

The author explicitly states that these results do not constitute a claim of general accuracy, but only the performance of this particular rig under specific conditions.

Scale ambiguity in monocular vision: When recovering camera poses from monocular RGB images, it is essentially solving a projective geometry problem — projecting 3D points onto a 2D pixel plane loses absolute distance information, so monocular images alone can only recover a similarity transform of the scene, i.e., the scale is arbitrary. This means the reconstructed map can be scaled arbitrarily without affecting the consistency of image observations — this is the so-called scale ambiguity. There are usually three ways to resolve it: introducing a calibration object of known size, using a stereo/multi-view baseline, or fusing a depth sensor that provides absolute distance. This article adopts the third approach, using iToF depth readings as an anchor for metric scale to constrain the RGB pose estimation results to true physical scale, thereby avoiding the scale drift common in monocular SLAM.

Edge alignment error and color-depth registration: Edge alignment error measures the pixel deviation between object edges in the RGB image and the edges of the depth map projected to the same viewpoint. Since the RGB camera and the iToF sensor are physically separate, there are differences in extrinsics (relative pose) and intrinsics; if calibration is inaccurate, color texture and depth geometry cannot align precisely, manifesting as "ghosting" or color overflow at object contours in the colored point cloud, directly affecting the supervision quality during Gaussian splatting training. The median dropping from 2.2 pixels to 1.0 pixel means more than half of the edge pixels have deviation within one pixel, which is the key threshold for color-depth fusion to be practical. In addition, on highly reflective surfaces (such as metal, glass, glossy paint), iToF can cause sensor saturation due to overly strong returns, producing crosstalk and phase unwrapping errors that make depth readings drift, so conditional handling or mask-based rejection is needed for bright regions.

Remaining Issues

The author candidly lists the current limitations: incomplete room coverage, dark walls "disappearing" in the depth map, and the Raspberry Pi camera intrinsics still needing a formal checkerboard calibration. The so-called "dark walls disappearing" is a typical pain point of iToF — low-reflectivity surfaces return too weak a signal of the modulated light for the sensor to resolve phase.

The ultimate goal is real-time Gaussian SLAM, but the current step is only offline reconstruction. From offline to real-time, there are still several hurdles: pose tracking, incremental Gaussian updates, and compute constraints.

An Engineering Question Worth Discussing

The author poses a question at the end: for such a rig, should one first independently verify intrinsics and depth scale, or wait until initial alignment is stable and go straight into joint optimization?

There is no standard answer to this question. The benefit of independent verification is that error sources are traceable — if joint optimization diverges, you know which stage the problem is in; the benefit of joint optimization is that it can leverage multi-view consistency to automatically compensate for calibration residuals, making it more robust when calibration is not precise enough. A common compromise in practice is: first fix the intrinsics to a credible range using a checkerboard, then perform an independent calibration of depth scale, and only then enable joint optimization. The author clearly tends toward making calibration solid first, which also explains why he put so much effort into edge alignment and conditional handling.

For enthusiasts who want to reproduce a similar rig, this lesson is worth noting: the bottleneck of low-cost depth sensors is often not the algorithm, but calibration and noise modeling.

One-line Summary

A maker built an RGB-D rig from an iToF engineering sample plus a Raspberry Pi camera, offline generating a colored point cloud with metric scale and training Gaussian splatting.

Paragraph Summary

A Reddit maker shared his homemade RGB-D capture rig: pairing a 320×240 epc660-class iToF engineering sample with a Raspberry Pi camera, outputting color depth via time synchronization, producing the first room point cloud with true metric scale, and offline training a Gaussian splatting model. The pipeline includes recovering camera poses from RGB frames, using iToF depth for absolute scale, color-depth fusion, and removing 27,675 floating points in one pass via multi-view voxel voting. The author stresses that calibration matters more than rendering, with the median edge alignment error dropping from 2.2 pixels to 1.0 pixel. Current limitations include dark wall loss, incomplete room coverage, and camera intrinsics pending checkerboard calibration; the ultimate goal is real-time Gaussian SLAM.