SCOPE-4D: How an Endoscopic 4D Geometry Foundation Model Solves Surgical Navigation Challenges

The Dual Challenge of Endoscopic Geometric Understanding

In minimally invasive surgery and robot-assisted procedures, precise geometric understanding is a prerequisite for reliable navigation. Yet geometric learning in endoscopic settings has long faced two thorny problems: first, geometric annotation data is extremely scarce, making high-quality 3D labels costly to obtain; second, there is an inherent ambiguity between camera motion and tissue deformation — when the scene changes, it's difficult for algorithms to determine whether the lens is moving or the soft tissue itself is deforming.

These two challenges directly limit the reliability of endoscopic navigation and robot-assisted systems. SCOPE-4D is a systematic solution to this pain point. It is a 4D geometry foundation model purpose-built for endoscopic scenes, capable of simultaneously predicting camera parameters, dense geometric structure, and 3D tissue motion trajectories from monocular RGB video in a single forward pass.

SCOPE-5K: A Large-Scale Endoscopic Geometry Dataset

The capabilities of a foundation model depend heavily on the scale and quality of its training data. The research team developed a data curation and annotation pipeline, producing a dataset called SCOPE-5K. This dataset contains approximately 5,000 video clips covering both real and synthetic sources, spanning multiple surgical scenarios including gastrointestinal endoscopy and laparoscopy.

The value of this dataset lies not only in its scale, but also in the rich geometric supervision signals it provides. In addition, the team collected new phantom data and real colonoscopy evaluation sets, establishing reliable validation benchmarks for model performance in real clinical environments. In the endoscopic domain, where geometric annotations have long been scarce, a dataset covering both real and synthetic scenarios is itself a significant contribution to the research community.

Phantom data refers to data collected using artificial constructs that simulate human organ structures — common examples include 3D-printed intestinal models or silicone abdominal cavity models. Compared to real clinical video, the advantage of phantom data is that precise ground truth geometric information can be obtained — for example, by synchronously recording camera poses and tissue deformation via optical tracking systems. This makes phantom datasets ideal benchmarks for evaluating geometric estimation algorithms, though their appearance and deformation characteristics still differ from real human tissue. The research team therefore also incorporated real colonoscopy videos as complementary validation, with both together forming a more convincing evaluation framework.

CRM Motion Constraints: Separating Tissue Deformation from Camera Motion

One of the core technical contributions of SCOPE-4D is its approach to handling the ambiguity between camera motion and tissue deformation. The model first undergoes geometric supervised fine-tuning on SCOPE-5K, learning geometric priors specific to endoscopic scenes, which improves the accuracy of camera pose and depth estimation.

Building on this, the research introduces the Common-Residual Motion (CRM) mechanism. The core idea of CRM is to constrain local tissue deformation within the frame of overall tissue motion — decomposing motion into a "common motion" component and a "residual motion" component. This design elegantly mitigates the difficulty of distinguishing camera motion from tissue motion.

Experiments show that CRM, combined with trajectory supervision, further improves camera and depth estimation beyond geometric fine-tuning alone, while also enabling dense 3D tissue tracking. This means the model can not only reconstruct static geometry but also track the dynamic changes of soft tissue across time — which is precisely what the "4D" in the name signifies.

Geometric supervised fine-tuning typically builds on pre-trained general-purpose visual foundation models, such as depth estimation or camera pose estimation networks trained on natural scenes. Because natural and endoscopic scenes differ significantly in lighting (intense point light sources, specular highlights), texture (uniformly low-texture moist mucosa), and scene dynamics (soft tissue peristalsis), direct transfer often performs poorly. The fine-tuning stage leverages the geometric labels provided by SCOPE-5K to help the model learn endoscopic-specific depth distributions and camera motion patterns, adapting the representational capacity of general visual models to this highly specialized application domain.

Experimental Validation: From Public Benchmarks to Clinical Video

Across multiple public and newly collected benchmarks, SCOPE-4D demonstrates strong geometric estimation capabilities, performing robustly in both in-domain and out-of-domain scenarios. On 3D tracking tasks, the model shows clear advantages over existing methods and exhibits greater stability in long-sequence colon reconstruction — precisely the aspect that is most challenging for real surgical navigation.

The research team also conducted a blinded user study, the results of which support the model's perceptual reconstruction quality on real clinical video. This approach of incorporating human subjective evaluation compensates for the limitations of purely objective metrics in reflecting clinical utility.

Blinded user studies carry particular significance in medical image evaluation. Because endoscopic 3D reconstruction lacks unified clinical quality standards, purely numerical metrics (such as depth error or trajectory error) may not reflect how surgeons actually experience the results in practice. Such studies typically invite evaluators with clinical or surgical experience to rank or score multiple reconstruction results from different methods without knowing their source, yielding quality judgments that are closer to real-world usage. This human-in-the-loop evaluation approach is increasingly becoming an important validation method in surgical AI research, helping to bridge the gap between algorithmic performance and clinical utility.

Significance and Outlook

The work on SCOPE-4D highlights two key factors critical to endoscopic geometric understanding: large-scale geometric supervision and a dedicated constraint mechanism targeting motion ambiguity. The former addresses the data scarcity problem, while the latter resolves the fundamental confusion between camera and tissue motion.

Integrating joint prediction of camera parameters, dense geometry, and 3D trajectories into a single forward inference pass not only improves accuracy but also lays the groundwork for real-time applications. For clinical scenarios such as surgical navigation, robot assistance, and intraoperative 3D reconstruction, the maturation of such foundation models has the potential to significantly lower the barrier to geometric perception. As endoscopy-specific foundation models continue to evolve, geometric understanding capabilities are poised to become a standard component of next-generation intelligent surgical systems.