The MARVIS Project: A Complete Analysis of Embedded AI Agents Empowering Autonomous Space Exploration

At the Flight Software Workshop 2026 (FSW 2026), George Williams from Armijo Innovations shared the latest progress on the MARVIS project — exploring how to deploy generative AI and large language model agents onto spacecraft, granting probes the capability for autonomous scientific discovery. This work spans three major domains: AI agent architecture design, edge hardware inference performance evaluation, and space general intelligence benchmarking, charting a clear technical roadmap for future deep-space autonomous exploration.

From Carl Sagan's Vision to AI-Driven Autonomous Exploration

The speaker opened with a personal story: his interest in autonomous space exploration traces back to the scene Carl Sagan painted in Cosmos — future probes should be "able to think for themselves, loaded with sophisticated biological and chemical instruments, roaming, sniffing, tasting, probing, choosing their own horizons each day." In Sagan's vision, humans would be largely passive observers throughout the entire mission.

From remote-controlled Mars rovers to progressively enhanced autonomous navigation capabilities, space exploration has made significant strides over the past few decades. But truly autonomous scientific exploration requires spacecraft to possess the following key capabilities:

• Solid foundational scientific knowledge and an innate curiosity for exploration
• The ability to autonomously design and execute experiments
• Comprehensive risk-reward assessment capabilities
• The ability to identify hazards and autonomously call Earth for emergency support
• Execution capabilities to control instruments and interact with navigation systems

This is precisely the starting point for the MARVIS project — leveraging generative AI and large language models to endow space probes with these attributes.

MARVIS Project AI Agent Architecture and POC Validation

Dataset Construction and LLM Selection

The team carefully curated a dataset of images simulating typical or challenging scenarios a planetary probe might encounter, covering geological formations acquired from ground and aerial perspectives, atmospheric weather systems, and potential biosignatures.

For model selection, the team compared two approaches:

• GPT-4o Mini: Over 10 billion parameters, requiring multiple server-grade GPUs, suitable for cloud deployment
• Gemma (Google): 4 billion parameters, weights heavily quantized to 4-bit, capable of running on various edge computing devices

Both support multimodal reasoning and can perform scientific analysis on images, providing the foundation for subsequent edge deployment comparisons.

A Clean and Efficient Agent Architecture Design

MARVIS adopted an extremely streamlined AI agent architecture. From top to bottom, sensor data (images) is first submitted to the Science Analysis Agent, guided by specific system prompts for scientific assessment; the analysis results are then passed to the Triage Agent, which decides the mission's next action based on the scientific analysis conclusions — continue exploring or issue a hazard alert.

A notable detail: all agents are "programmed" through natural language prompts and communicate with each other via natural language. The only traditional code is approximately half a page of Python scaffolding used to create the agents' directed acyclic graph (DAG). The team used OpenAI's open-source Swarm framework for agent orchestration, while noting that AutoGen, CrewAI, LangChain, and similar frameworks are equally applicable.

At the leaf nodes, agents are required to output structured data (JSON format), leveraging the language model's tool-calling capabilities — because in real aerospace systems, models need to interact with instruments and devices that only have structured interfaces (such as software APIs or library calls).

Key point: The team performed no task-specific training or fine-tuning on the models, relying solely on system prompts to define each agent's role, which significantly lowers the deployment barrier.

POC Test Results: Validating Scientific Analysis and Autonomous Decision-Making

In testing with layered sedimentary rock images, the Science Analysis Agent accurately identified "layered sedimentary rock formations, potentially representing ancient riverbeds or lake deposits, with obvious color banding indicating variations in mineral composition." The Triage Agent correctly recommended continued exploration, deeming this a priority area worthy of deeper investigation.

In dust storm image testing, the system not only accurately detected the dust storm but also automatically issued a safety alert, noting that "significant atmospheric disturbance poses a potential hazard to vehicle operations" — despite the team never explicitly instructing the agent to issue alerts when seeing clouds, only to detect dangers to the vehicle. This emergent safety judgment capability is particularly impressive.

Expert Evaluation and Edge Hardware Inference Performance Comparison

Human Expert Assessment of AI Agent Output Quality

To systematically validate the quality of AI analysis, the team surveyed 88 respondents using Likert scales to separately evaluate the agent's scientific analysis and mission decision-making performance:

• Non-aerospace domain experts: Gave ratings of "good to excellent"
• Aerospace domain experts (with mission engineering or planning experience): Ratings dropped by approximately one grade level but still remained near the "good" range

A key finding: there was no statistically significant difference between large cloud-based models and small edge models. This means that in space scenarios, quantized small models are fully capable of handling scientific analysis tasks, providing strong support for embedded AI edge deployment.

Edge Hardware Token Generation Rate Benchmarks

The team argued that hardware vendors' commonly used MAC rates or TOPS metrics do not accurately predict actual LLM inference performance, so they directly measured token generation rates across platforms:

Hardware Platform	Power Consumption	Inference Runtime	Performance Summary
MacBook Pro	70W	MLX on Metal	Serves as reference baseline, highest token rate
Snapdragon X Elite	—	Qualcomm Neural Network SDK	Reasonable performance; Qualcomm believes there's room for optimization
Jetson Orin NX	—	llama.cpp / TensorRT-LLM	Good performance when fully utilizing the Tegra GPU
Vertium AI Dev Kit (DeepX chip)	5W	DeepX proprietary library	Ultra-low power yet features advanced AI inference silicon

The team also highlighted an easily overlooked issue: the same model may produce different inference results on different hardware, and even repeated runs on the same hardware can yield subtle differences. Combined with LLMs' inherent hallucination problem, establishing a systematic accuracy evaluation framework is critical for aerospace flight control scenarios.

Future Plans: Space General Intelligence Benchmarks and Autonomous Experiment Design

MMMUSGI: An AI Benchmark Suite for Space Scenarios

The team is developing a language model agent general intelligence evaluation dataset specifically for the space domain — MMMUSGI — inspired by the cross-modal foundation model evaluation dataset MMU. The dataset includes two core testing dimensions:

• Real Mars helicopter navigation camera images: For testing geological identification and anomaly detection capabilities
• Synthetically generated structural damage images: For testing visual structural damage detection capabilities

The establishment of this benchmark suite will provide a unified standard for evaluating the capabilities of space-embedded AI agents.

Distributed Collaborative Exploration Prototype System

The team is designing a distributed collaborative exploration prototype system, planning to collaborate with universities in California and Arizona to extend the existing architecture into a multi-agent system for rover and drone collaborative exploration. The system will be equipped with:

• Portable microscopes and portable gas chromatographs
• Portable mass spectrometers
• Low-cost robotic arms (approximately $150 Hugging Face robot arms), controlled by Vision-Language-Action (VLA) models

The project's most central goal is: developing an AI agent capable of autonomously designing experiments using available instruments. Ideally, the agent would only need functional descriptions of instruments to independently infer what scientific research can be conducted — this is precisely the engineering implementation path for Sagan's decades-old vision of "autonomous thinking, autonomous exploration" in the AI era.

The Engineering Path from POC to Autonomous Space Exploration

The MARVIS project reveals an important technology trend: pushing cutting-edge AI capabilities — generative AI, large language models, multi-agent architectures — from data centers to extreme edge environments, namely spacecraft. The POC-stage results demonstrate that even small, non-fine-tuned language models, through carefully designed agent architectures and prompt engineering, can exhibit satisfactory capabilities in planetary science analysis and mission decision-making.

As edge AI hardware continues to advance and model quantization techniques mature, embedding AI agents into aerospace flight control systems is no longer science fiction but an engineering path being progressively validated. Based on MARVIS's practical experience, the technical foundation for autonomous space exploration is already in place. The challenges ahead lie in reliability verification, benchmark standardization, and the engineering implementation of distributed multi-agent systems.