Machine Learning Restores Missing Band on Mars Orbiter: HiPredict Reconstructs HiRISE Color Imagery

The High Resolution Imaging Science Experiment (HiRISE) camera aboard the Mars Reconnaissance Orbiter (MRO) has continuously imaged the Martian surface at 25–30 cm/pixel resolution, making it a core data source for studying Mars' geology, climate history, and landing site selection. But data loss caused by equipment aging is becoming an increasingly serious problem. A new paper proposes the HiPredict framework, which uses machine learning to reconstruct the RED4 spectral channel corresponding to the failed detector, offering a reusable approach for "extending the lifespan" of planetary exploration data.

Detector Failure and the Limitations of SYN4

The HiRISE focal plane assembly contains 14 detectors, several of which overlap, enabling the instrument to acquire three-channel color-infrared imagery—crucial for identifying compositional differences and studying active surface processes. In 2023, a detector at the center of the focal plane (RED4) began to fail intermittently, causing gaps in the imagery and halving the instrument's color imaging capability.

The existing processing pipeline uses a linear approach: it linearly scales the co-located near-infrared channel (IR10) to generate a synthetic red channel (SYN4). The problem is that the relationships between color channels are actually nonlinear, and linear scaling cannot capture these relationships, limiting both accuracy and scope of application. According to the paper, SYN4 has a normalized mean absolute error of 3.1%, which is insufficient for scenarios requiring fine color interpretation.

HiRISE's focal plane is assembled from 14 CCD detectors, with overlapping regions between adjacent detectors in space, so the same surface area is simultaneously imaged by multiple detectors through different filters. This redundant design is the prerequisite for HiRISE's ability to simultaneously acquire three visible-to-near-infrared bands (blue-green, red, near-infrared): each band is not covered across the full swath by a single detector, but rather by multiple detectors working in segments and stitched together. RED4 is located at the center of the focal plane, and its failure not only creates gaps in the imagery but also directly disrupts the pairing relationship between the red channel and the near-infrared channel, making color compositing impossible under the original scheme. Understanding this hardware layout helps explain why reconstruction must rely on neighboring channels rather than simple interpolation.

How HiPredict Reconstructs RED4

The core idea of HiPredict is to transform the reconstruction of the missing channel into a multi-input learning problem. The model's inputs include four types of information: co-located IR10 imagery, blue-green channel (BG12) imagery, neighboring red channel observations, and acquisition metadata. The network uses a U-Net architecture to learn the nonlinear spatiotemporal-spectral relationships between these inputs and the target channel.

The training scale is considerable: 81,717 training samples corresponding to 13,625 observations. Evaluation is performed on 23,348 test samples drawn from 3,892 pre-failure observations—using real pre-failure data as the benchmark gives the evaluation results practical reference value. Ultimately, HiPredict achieves a normalized mean absolute error of 1.1%, significantly better than SYN4's 3.1%.

U-Net is an encoder-decoder convolutional neural network architecture originally developed for biomedical image segmentation. Its encoder progressively downsamples to extract multi-scale features, while the decoder upsamples to restore spatial resolution, and skip connections directly concatenate the encoder's high-resolution features to the corresponding decoder layers, thereby preserving fine spatial details in the output. This architecture achieves good results with relatively few training samples, so it is also widely used for image-to-image translation tasks such as denoising, super-resolution, and missing band reconstruction. In this case, U-Net's skip connections help transfer the spatial structure information from IR10 and BG12 to the reconstructed RED4 channel, preventing the output from being over-smoothed and losing geological details.

Reconstruction Quality and Scientific Value

Beyond the error numbers, what is more noteworthy is the performance of the reconstructed products in actual scientific interpretation. The paper points out that the reconstructed imagery improves the recovery quality of color images, presenting several geological features of interest more faithfully, including ice-bearing craters, strata exposed along ridge tops, and chloride deposits.

These features are precisely the most sensitive targets for color interpretation—the tonal differences between ice and deposits, and the color changes at stratigraphic boundaries, all depend on subtle nonlinear relationships between channels. Linear-scaling SYN4 loses information in these scenarios, while the learned nonlinear mapping can preserve more detail. This also explains why the authors emphasize the method's "applicability" rather than just accuracy improvement.

What It Means for Aging Planetary Missions

The significance of this work extends beyond the repair of a single detector. Planetary missions are often designed with lifespans far shorter than their actual service time, and as equipment naturally degrades, data loss gradually accumulates. HiPredict demonstrates a path: using information from other channels within the same instrument plus machine learning to compensate for information loss caused by failed components.

The paper releases HiPredict as an open-source software tool, meaning other mission teams can adapt this framework to handle similar channel-missing problems. Of course, the boundaries of this method are also worth noting—it relies on the existence of co-located other channels and neighboring observations, and is not applicable to scenarios where redundant information is completely lost. But from the perspective of "reducing data loss," this is already a solution with practical deployment value.

Frequently Asked Questions

How does HiPredict differ from traditional interpolation methods?

Traditional methods like SYN4 rely on linear scaling and cannot model nonlinear relationships between channels; HiPredict uses U-Net to learn nonlinear spatiotemporal-spectral mappings, reducing the error from 3.1% to 1.1%.

Can this method be applied to other planetary exploration missions?

The framework itself is general-purpose, but it requires the target instrument to have co-located other spectral channels and neighboring observations as inputs. The authors have open-sourced the tool for other teams to adapt.

Will the reconstructed imagery introduce artifacts?

The paper evaluates using real pre-failure observations as the test benchmark. The reconstructed products are closer to real imagery on features such as ice-bearing craters, strata, and chloride deposits, but they are still synthetic data, and scientific use should be combined with confidence assessment.

One-line Summary

HiPredict uses U-Net to reconstruct HiRISE's failed RED4 channel, reducing error from 3.1% to 1.1%.

Paragraph Summary

The RED4 detector of the Mars Reconnaissance Orbiter's HiRISE camera began to fail intermittently in 2023, halving its color imaging capability. The existing SYN4 approach uses linear scaling of the near-infrared channel to generate a synthetic red channel, with a normalized mean absolute error of 3.1%, making it difficult to capture nonlinear relationships between channels. A new paper proposes the HiPredict framework, which takes IR10, BG12, neighboring red channel, and acquisition metadata as inputs, and uses U-Net to learn nonlinear spatiotemporal-spectral mappings for reconstruction. The model is trained on 81,717 training samples and evaluated on 23,348 test samples from 3,892 pre-failure observations, reducing the error to 1.1%. The reconstructed imagery is closer to reality on features such as ice-bearing craters, ridge strata, and chloride deposits. The tool has been open-sourced, providing a reusable approach for data loss compensation in aging planetary missions.