Thermal-RGB Fusion: Self-Supervised Wheel Track Detection Tackles Snow Autonomous Driving

The Hidden Challenge of Autonomous Driving in Snow

Accurate drivable area detection is one of the core capabilities of autonomous driving and Advanced Driver Assistance Systems (ADAS). Yet when roads are covered in snow, existing general-purpose traversability estimation methods often fall short — featureless white snowscapes lack texture and visual cues, making it nearly impossible for traditional computer vision algorithms to find reliable driving references in these "whiteout" conditions.

A recent study proposes a targeted solution to this pain point: rather than trying to estimate the entire traversable region, the approach focuses on tracking existing wheel tracks. Following wheel tracks left by preceding vehicles offers three key advantages — better traction, more predictable vehicle dynamics, and verified path stability. For winter autonomous driving, this is a highly practical angle of attack.

Self-Supervised Labels: Letting GNSS Trajectories Auto-Generate Training Data

The most elegant design in this research is its self-supervised label generation mechanism. The biggest bottleneck in supervised learning is typically annotation cost, and in scenarios like snowy wheel tracks — where features are ambiguous and hard for humans to identify — manual labeling becomes even more daunting.

The research team's approach: use the vehicle's recorded GNSS (Global Navigation Satellite System) data to back-project the vehicle's future trajectory onto the current image frame. In other words, the path the vehicle actually drives over naturally constitutes ground-truth labels for "this is a drivable wheel track." This requires no human intervention, enabling automatic large-scale generation of training data and dramatically lowering the barrier and cost of data preparation.

This is an increasingly popular "self-annotation through physical behavior" paradigm in robotics and autonomous driving — converting real vehicle motion into supervision signals that are both efficient and grounded in actual driving logic.

The technical implementation of projecting GNSS trajectories onto image frames relies on camera-GNSS extrinsic calibration. The system must precisely know the spatial relationship between the camera coordinate frame and the vehicle coordinate frame to map 3D world-coordinate GNSS trajectory points onto the 2D image plane via a projection matrix. This process typically requires addressing time synchronization — GNSS sampling rates and camera frame rates are often mismatched, requiring interpolation to align timestamps. Furthermore, the validity of "future trajectory" as a label rests on an assumption: that the vehicle's current path is safely traversable. This generally holds in car-following scenarios with existing wheel tracks, but may introduce noisy labels when pioneering new paths. The core advantage of self-supervised learning lies precisely here — trading a small number of assumptions for massive automated annotation, which is the shared logic of weakly supervised and self-supervised methods in perception tasks.

RGB-I Four-Channel Fusion: Seeing Thermal Differences Invisible to the Naked Eye

Relying solely on a visible-light camera, it is nearly impossible to distinguish wheel tracks from surrounding snow in whiteout conditions. Another core element of the research is the introduction of Long-Wave Infrared (LWIR) thermal imaging, fused synchronously with RGB images into a four-channel RGB-I input for the neural network.

The value of thermal imaging lies in this: snow compacted by vehicle tires differs subtly from surrounding loose snow in its thermal radiation characteristics — differences that are nearly invisible in visible light. By fusing the spatial detail of visible light with the thermal information from infrared, the neural network can reliably identify safety-critical wheel track paths in environments that are visually monotonous and featureless.

Another highlight of this multi-modal sensing approach is cost control. The research repeatedly emphasizes that the solution is a "low-cost" alternative — compared to expensive sensors like LiDAR, the combination of RGB plus infrared cameras is far more price-accessible, offering a pragmatic engineering path for winter autonomous driving.

LWIR thermal imaging sensors operate in the 8–14 micrometer band, sensing thermal radiation emitted by objects themselves rather than reflected light, making them independent of lighting conditions and equally effective at night or on overcast days. The thermal difference between wheel tracks and surrounding snow originates from two physical mechanisms: first, tire compression compacts the snow layer, altering its thermal conductivity and heat capacity, leading to different surface temperature response characteristics; second, compacted snow in contact with residual heat conducted from the vehicle undercarriage dissipates heat at a different rate than surrounding loose snow. These temperature differences are typically only a fraction of a degree Celsius — completely imperceptible to regular RGB cameras — while uncooled LWIR cameras can detect temperature differences on the order of 0.05°C. With four-channel fused inputs, the neural network effectively learns spatial texture features (from RGB) and thermal radiation distribution patterns (from infrared) simultaneously, with the complementarity of the two being especially critical in visually degraded environments like snowy terrain.

ThermalTrack Dataset and Experimental Results

Experimental results demonstrate that the method exhibits strong robustness in visually monotonous, featureless harsh environments, validating the effectiveness of thermal-RGB fusion in winter autonomous driving scenarios.

Alongside the research, the team has released the new ThermalTrack dataset, publicly hosted at Michigan Technological University's digital repository (digitalcommons.mtu.edu/thermaltrack). Open datasets are critically important for advancing research in specialized domains — niche yet crucial scenarios like snowy terrain, thermal imaging, and wheel track detection have long lacked public benchmarks, and ThermalTrack's release provides subsequent researchers with a reproducible, comparable experimental foundation.

The value of open datasets in autonomous driving subfields can be understood by analogy with how benchmark datasets like KITTI and nuScenes have advanced general perception research. The core contribution of such benchmarks is not just the data itself, but a unified evaluation protocol — only when all researchers use the same dataset and metrics does cross-method comparison become possible, and can technical progress be quantitatively measured. ThermalTrack fills precisely the benchmark gap at the three-way intersection of snowy terrain + thermal imaging + wheel track detection. The dataset's sensor configuration (RGB camera + LWIR camera + GNSS) is relatively lightweight, meaning other research teams can reproduce and extend experiments at a lower hardware threshold — helping foster a genuine research community rather than a domain exclusive to teams with expensive equipment.

Implications for Winter Autonomous Driving

The value of this work lies not only in the technology itself, but in how it reframes the problem. In extreme weather, pursuing "detection of all drivable areas" may be overly idealistic, whereas "following verified wheel tracks" is a more robust strategy that better aligns with human driving intuition.

Combining self-supervised annotation to reduce data costs, multi-modal fusion to enhance perception, and an open dataset to advance community research, this solution offers a technical pathway for deploying autonomous driving in icy and snowy regions — balancing robustness with economic viability. For autonomous driving research long dominated by fair-weather data, this kind of exploration targeting harsh environments is especially valuable.