[KongchangAI]
· 2 min read· 1,293 words

FM-Recon: How Frozen Foundation Model Features Accelerate Cardiac Cine MRI Reconstruction

FM-Recon: How Frozen Foundation Model Features Accelerate Cardiac Cine MRI Reconstruction

FM-Recon injects refined frozen foundation model features into an unrolled network to boost cardiac cine MRI reconstruction.

FM-Recon integrates pretrained foundation model features into accelerated cardiac cine MRI reconstruction. A frozen encoder extracts multi-scale features from undersampled images, which a lightweight refinement module adjusts before injecting into each layer of an unrolled network. Experiments on CMRxRecon2023 show that even with severe artifacts, the frozen encoder yields informative representations, and reconstruction quality significantly improves at moderate acceleration factors, with layer-wise fusion being more effective than input/output-only use.

In medical imaging, foundation models are demonstrating powerful feature extraction capabilities, yet their potential in MRI reconstruction remains underexplored—especially within unrolled reconstruction networks and when dealing with artifact-contaminated inputs. A recent study titled FM-Recon: Exploring Foundation Model Features for Accelerated Cardiac Cine MRI Reconstruction systematically investigates how to integrate features from pretrained foundation models into accelerated cardiac cine MRI reconstruction and offers encouraging answers.

Why Focus on Cardiac Cine MRI Reconstruction?

Cardiac cine MRI is a key technology for assessing cardiac function and morphology, but slow acquisition has long been a bottleneck in clinical application. Accelerated acquisition (e.g., undersampling) can shorten scan time but introduces artifacts, necessitating robust reconstruction algorithms. In recent years, deep learning reconstruction methods—particularly unrolled networks—have achieved remarkable results by unfolding iterative optimization into network layers. However, these networks are typically trained from scratch, rely on large amounts of labeled data, and have limited robustness to artifacts.

Foundation models are pretrained on large-scale data and can extract generalizable and robust features. A natural question arises: can these features be leveraged to improve reconstruction quality? Especially when the input itself contains artifacts, can foundation models still extract useful information? This is precisely the question FM-Recon seeks to answer.

FM-Recon study overview

Unrolled Reconstruction Network is a class of deep learning methods that unfold traditional iterative optimization algorithms (such as alternating minimization based on compressed sensing) into a finite number of neural network layers. In MRI reconstruction, iterative algorithms typically alternate between data consistency (ensuring the reconstruction matches the acquired k-space data) and regularization (removing artifacts and incorporating priors). Unrolled networks map each iteration to a layer in the network, where the regularization step is replaced by learnable convolutional layers or modules, while the data consistency step remains analytical. This preserves physical model constraints while learning optimal priors through end-to-end training. Compared to purely data-driven black-box networks, unrolled networks have fewer parameters, are more interpretable, and are more robust to variations in undersampling patterns, making them a mainstream paradigm for accelerated MRI reconstruction.

The FM-Recon Framework: Frozen Encoder + Feature Refinement

The core design of FM-Recon is simple yet effective: it extracts features from a frozen image encoder (from a pretrained foundation model), processes them through a refinement module, and injects them into each layer of the unrolled reconstruction network. The key point is that the foundation model's parameters remain frozen, and only lightweight feature refinement and fusion modules are trained, which avoids catastrophic forgetting and reduces computational overhead.

Specifically, the framework consists of three parts:

  • Feature Extraction: A frozen foundation model encoder is applied to the undersampled image (with artifacts) to obtain multi-scale features.
  • Feature Refinement: A trainable refinement network adjusts the foundation model features into a representation suitable for the reconstruction task.
  • Feature Fusion: At each iteration step of the unrolled network, the refined features are combined with the network's intermediate features to guide the reconstruction process.

This design enables the general knowledge of the foundation model to be effectively transferred to the reconstruction task while preserving the data consistency constraints of the unrolled network.

Foundation Model refers to deep neural networks pretrained on large-scale, diverse data (typically using self-supervised or masked reconstruction objectives) that can learn generalizable and transferable feature representations. In computer vision, such models (e.g., CLIP, DINOv2) are pretrained on massive natural images, and their encoder outputs exhibit strong generalization across various downstream tasks. In medical imaging, foundation models can be pretrained on large-scale medical images (e.g., multimodal MRI, CT) to capture general patterns of anatomical structures and textures. Using a foundation model as a frozen feature extractor for downstream tasks is a parameter-efficient transfer learning strategy: without fine-tuning all parameters, only lightweight adaptation modules are trained to leverage pretrained knowledge while avoiding overfitting and catastrophic forgetting on small datasets.

Experimental Setup and Key Findings

The study conducted extensive experiments on the CMRxRecon2023 dataset, systematically evaluating the impact of foundation model features under different reconstruction settings. Key findings include:

  • Foundation models can extract informative representations from artifact-contaminated inputs: Even when input images contain severe artifacts, the frozen encoder still outputs features that aid reconstruction, challenging the stereotype that "foundation models are only suitable for clean images."
  • Reconstruction performance improves, especially at moderate acceleration factors: Experiments show that at moderate acceleration rates, FM-Recon achieves better reconstruction quality than baseline methods (without foundation model features), while improvements are limited at very high acceleration factors, possibly due to severe information loss.
  • Effectiveness of the feature fusion strategy: Injecting features into each layer of the unrolled network is more effective than using features only at the input or output, indicating that continuous guidance during iteration is crucial.

These results are validated across multiple acceleration factors and sampling patterns, demonstrating the method's robustness.

CMRxRecon2023 Dataset is a public dataset used in the CMRxRecon challenge at the 2023 Medical Image Computing and Computer Assisted Intervention (MICCAI) conference, focusing on accelerated cardiac cine MRI reconstruction. The dataset contains cardiac cine MRI data acquired from multiple centers and vendors, providing k-space data with various acceleration factors (e.g., 4x, 8x, 10x) and different undersampling patterns (e.g., 1D Cartesian, 2D random), along with fully sampled reference images for supervised training and evaluation. Its multi-center nature requires models to handle domain shifts caused by different scanners and imaging protocols, making it an important benchmark for evaluating reconstruction algorithm robustness and generalization. FM-Recon's experiments on this dataset cover multiple acceleration settings, enabling systematic analysis of the contribution of foundation model features under varying degrees of information loss.

Implications for Medical Image Reconstruction

The significance of FM-Recon lies not only in a specific method but in opening a new avenue: using frozen foundation models as feature priors can cost-effectively enhance the performance of existing reconstruction networks. This is particularly valuable for the data-scarce field of medical imaging, as foundation models can be pretrained on large-scale natural or medical images and then directly transferred.

Furthermore, the study reveals the potential of foundation models with artifact-laden inputs, encouraging bolder use of pretrained features in real clinical scenarios without pursuing perfect input quality. Of course, how to select appropriate foundation models and design more efficient refinement modules remain directions worth exploring.

Limitations and Future Directions

The authors also point out limitations: experiments were conducted only on cardiac cine MRI, without validation on other anatomical regions or modalities; the choice of foundation model was relatively limited; and the feature fusion approach can be further optimized. Future work could extend to 3D reconstruction, multi-contrast MRI, and explore end-to-end fine-tuning strategies.

Overall, FM-Recon provides a practical and effective solution for accelerated MRI reconstruction and adds new evidence for the deployment of foundation models in medical imaging. As more studies follow, we have reason to expect foundation models to become standard components in medical image reconstruction pipelines.

One-line Summary

FM-Recon refines frozen foundation model features and injects them into an unrolled network to improve accelerated cardiac cine MRI reconstruction quality.

Paragraph Summary

FM-Recon explores how to integrate pretrained foundation model features into accelerated cardiac cine MRI reconstruction. To address the issues of unrolled reconstruction networks trained from scratch relying on large data and having limited artifact robustness, the method uses a frozen foundation model encoder to extract multi-scale features from undersampled images, which are adjusted by a lightweight refinement module and injected into each layer of the unrolled network to guide iterative reconstruction. Experiments on the CMRxRecon2023 dataset show that even with severe artifacts in the input, the frozen encoder can still extract informative representations; at moderate acceleration factors, reconstruction quality significantly outperforms baselines, and layer-wise feature fusion is more effective than using features only at input/output. This work offers a new approach for cost-effectively leveraging foundation models to enhance medical image reconstruction performance.

#FM-Recon#cardiac cine MRI#foundation model#MRI reconstruction#unrolled network
Share:

Related articles