OSGR Framework: Two-Stage Graph Reasoning Boosts Earth Observation QA Accuracy

Introduction

Earth observation (EO) archives store massive amounts of satellite imagery and geospatial data, but efficiently answering complex spatiotemporal queries has long been a challenge. Traditional LLM agents in the retrieval and inference process either fail to accurately ground observations or face the dilemma of search space explosion. The recently proposed OSGR (Observation-Support Graph Reasoning) framework offers a new approach to this problem through a two-stage decomposition of offline symbolic concept derivation and online relation-conditioned grounding.

Problem Background: Two Major Bottlenecks of LLM Agents

Earth observation QA requires systems to simultaneously complete retrieval and inference under strict spatiotemporal constraints. For example, queries like "find observation records in a certain region over the past five years where the vegetation index significantly declined and is correlated with decreased precipitation" require both retrieving relevant imagery from archives and combining meteorological data for causal inference.

Existing LLM agents perform poorly on such tasks, with two core reasons: first, models struggle to establish reliable mappings between abstract queries and specific observation items (i.e., "grounding" failure); second, the search space grows exponentially with combinations of constraints, causing inference efficiency to plummet. These problems are particularly evident in benchmarks like Earth-Bench—according to the paper, GPT-4o's baseline accuracy is only 49.3%.

Earth-Bench is a benchmark test suite for earth observation QA, specifically designed to evaluate agents' retrieval and reasoning capabilities under complex spatiotemporal constraints. It contains multiple types of queries covering vegetation monitoring, disaster assessment, climate change, and other scenarios, requiring systems to not only retrieve relevant imagery but also combine auxiliary data (such as meteorological records) to complete causal or trend inference. The difficulty of this benchmark lies in the diverse combinations of constraints and the enormous search space, making it an important yardstick for testing LLM agents' spatiotemporal reasoning capabilities. The GPT-4o baseline accuracy of 49.3% reported in the paper was measured on this benchmark, reflecting the significant shortcomings of current general-purpose models in specialized earth observation tasks.

OSGR's Core Design: Decomposing the Search Space

OSGR's core idea is to decompose the joint search problem into two stages, thereby avoiding combinatorial explosion during online inference.

Offline Stage: Observation-Support Graph and Symbolic Concept Derivation

In the offline stage, OSGR constructs an Observation-Support Graph that derives valid support paths from targets to results at the concept level. This graph does not store specific observation items but rather encodes support relationships between concepts (e.g., "vegetation index decline" can be supported by "NDVI time series analysis"). Through symbolic derivation, the system can pre-eliminate large numbers of unreasonable retrieval paths, significantly narrowing the scope of online search.

Symbolic concept derivation refers to using symbolic knowledge representations (such as logical rules and ontological relations) to formally reason about support relationships between concepts, rather than relying on numerical computation or black-box models. In OSGR, the system establishes symbolic links between abstract concepts in queries (such as "vegetation index decline") and possible supporting concepts (such as "NDVI time series analysis"), and derives through rules which concept paths are logically feasible. This approach enables pre-pruning at the offline stage, avoiding traversal of all possible observation item combinations during online retrieval, thereby effectively controlling the search space. The advantage of symbolic derivation lies in its interpretability and determinism, but its effectiveness highly depends on the completeness and accuracy of concept knowledge.

Online Stage: Relation-Conditioned Grounding and Spatiotemporal Compatibility Kernels

The online stage is responsible for instantiating concept paths into specific sets of observation items. OSGR employs a Relation-Conditioned Grounding mechanism combined with spatiotemporal compatibility kernels to evaluate the degree of matching between candidate observation items and query constraints. This process transforms abstract concept paths into executable retrieval operations while ensuring consistency of spatiotemporal constraints.

Spatiotemporal compatibility kernels are functions used to measure the degree of matching between candidate observation items and query spatiotemporal constraints. They typically consider factors such as temporal proximity, spatial overlap, and resolution consistency to compute a compatibility score for each candidate observation item. In OSGR's online grounding process, spatiotemporal compatibility kernels are used to filter and rank the set of observation items instantiated from concept paths, ensuring that final retrieval results strictly align with query requirements in both time and space. For example, for a query about "a certain region over the past five years," the kernel function penalizes observation items that fall outside the time range or have incomplete spatial coverage, thereby improving grounding accuracy.

Experimental Results: Dual Improvement in Accuracy and Efficiency

OSGR achieved significant results on two benchmarks: Earth-Bench and DisasterM3-Reason. According to the paper, OSGR with GPT-4o improved accuracy from 49.3% to 63.7% while reducing the execution step ratio from 2.9 to 2.5. This means the system not only answers more accurately but also requires fewer reasoning steps, achieving higher efficiency.

This magnitude of improvement is particularly valuable in complex spatiotemporal reasoning tasks. The reduction in execution step ratio indicates that OSGR's offline derivation effectively reduces ineffective exploration in the online stage, alleviating the search space explosion problem.

DisasterM3-Reason is a multimodal reasoning benchmark focused on disaster scenarios, requiring agents to combine satellite imagery, geographic information, and socioeconomic data to conduct impact assessment and causal analysis of disaster events. Queries in this benchmark often involve multi-source data fusion and complex spatiotemporal reasoning, such as "identify flood inundation extent and correlate with rainfall data to infer causative factors." OSGR's performance on this benchmark further validates the effectiveness of its decomposition strategy in complex reasoning tasks within specialized domains. Compared to Earth-Bench, DisasterM3-Reason places greater emphasis on multimodal information synergy and disaster chain derivation, thus imposing higher requirements on grounding precision and reasoning efficiency.

Multi-Source Perspectives and Limitations

Currently, OSGR's validation is mainly concentrated on academic benchmarks, and its generalization capability in real-world business scenarios (such as disaster emergency response and agricultural monitoring) remains to be tested. Additionally, the construction of offline graphs relies on concept-level knowledge engineering, and how to automatically expand support graphs to cover broader geographic query types is a direction worth attention in the future.

Conclusion

OSGR decomposes the joint search in earth observation QA into offline symbolic derivation and online relation-conditioned grounding, significantly improving LLM agents' answer accuracy and execution efficiency while maintaining strict spatiotemporal constraints. This approach provides valuable reference for geospatial QA system design and demonstrates the potential of combining graph reasoning with LLMs in specialized domains.

One-line Summary

OSGR significantly improves earth observation QA accuracy and efficiency through two-stage decomposition of offline symbolic derivation and online relation-conditioned grounding.

Paragraph Summary

The OSGR framework addresses two major bottlenecks of LLM agents in earth observation QA—grounding difficulty and search space explosion—by proposing a two-stage decomposition strategy: the offline stage constructs an observation-support graph and performs symbolic derivation at the concept level to pre-eliminate unreasonable retrieval paths; the online stage employs relation-conditioned grounding and spatiotemporal compatibility kernels to instantiate concept paths into specific observation item sets, ensuring spatiotemporal constraint consistency. On the Earth-Bench and DisasterM3-Reason benchmarks, OSGR with GPT-4o improved accuracy from 49.3% to 63.7% and reduced the execution step ratio from 2.9 to 2.5, achieving dual improvements in accuracy and efficiency. The method is currently limited to academic benchmark validation, and future work should focus on real-world generalization and automated expansion of support graphs.