DR Pixel: Bypassing OCR to Let Vision-Language Models Directly 'Read' Page Images for Deep Research

From OCR to Pixels: A Paradigm Shift for Deep Research Agents

Complex documents such as scientific papers and technical reports contain a wealth of information in tables, charts, and formulas, yet existing Deep Research (DR) agents typically compress papers into OCR-extracted plain text. While lightweight, this approach is detached from the document's original structure: charts are discarded, formulas become garbled, and layout relationships are flattened.

According to the recently published research work DR Pixel, the team asked the opposite question: can vision-language models (VLMs) directly retrieve and reason over rendered document page images? They released DR Pixel, the first end-to-end multimodal deep research agent that uses page images as the sole retrieval and reasoning unit, operating over a corpus of 1.06 million pages.

The core value of this setup is: the page is the data unit. The model sees the paper's true visual presentation rather than "second-hand information" after text extraction.

Vision-Language Models (VLMs) are a class of multimodal large models that process both image and text inputs, with typical representatives including GPT-4o, the Gemini series, and Qwen-VL. Unlike pure-text LLMs, a VLM's vision encoder splits images into patches and maps them into a representation space shared with text, enabling it to "read" layouts, recognize chart structures, and parse formula symbols. This capability is the premise for the page-native approach to work: only when the model can directly extract structured semantics from pixels will skipping OCR avoid information loss. In recent years, VLM advances in high-resolution document understanding (such as dynamic resolution and slice encoding) have significantly reduced the accuracy loss of page-image reasoning, making full-page retrieval units engineering-feasible.

Deep Research Agents refer to LLM agents that can autonomously plan, perform multi-round retrieval, and integrate evidence to answer complex questions, typically consisting of retrieval tools, a reasoning loop, and answer synthesis. Unlike single-round retrieval in traditional RAG, DR agents emphasize multi-step tool calls and evidence chain construction, so their performance depends not only on model capability but also heavily on the design of retrieval units—exactly the link that page-level approaches attempt to change.

DRPixelBench: An Evaluation Benchmark Designed for Visual Reasoning

To validate the effectiveness of page-level deep research, the team built DRPixelBench, a benchmark specifically targeting scientific papers that requires models to simultaneously understand text, charts, tables, and formulas. This stands in sharp contrast to traditional text-based QA benchmarks—the latter typically test only snippet retrieval and text reasoning, whereas DRPixelBench forces agents to locate information within visual layouts and perform multi-step reasoning.

The benchmark's design reflects the research team's redefinition of the "deep research" task: true depth lies not in how much text is retrieved, but in whether evidence across modalities in a document can be integrated.

Zero Training Still Matches Text-Based Approaches

A surprising but important finding is that page-level deep research works quite well without additional training. On frontier models including GPT-5.4, Gemini-3.5-Flash, and Qwen3.6-27B, page-image-based approaches comprehensively outperform OCR-based baselines while requiring fewer tool calls.

This means efficiency gains on two levels: better results and less interaction. The traditional "OCR first, then retrieve" pipeline is not only redundant but may even be suboptimal in complex document scenarios. The research team put it directly: directly retrieving and reasoning over visual document pages makes complex deep research both more effective and more efficient.

For product and engineering teams, the takeaway is: when building document agents, rendering page images and handing them directly to a VLM may be simpler than building an OCR pipeline, while preserving the complete information of charts and formulas.

The Training Challenge: "Noise" in Teacher Trajectories

The research did not stop at the zero-training baseline. The team further explored DR training but soon encountered a key challenge: teacher trajectories used for agent supervised fine-tuning are noisy, mixing critical reasoning and action steps with redundant or even failed attempts. If these trajectories are learned uniformly, the model will absorb erroneous behaviors as well.

To address this, the research proposes counterfactual credit reweighting, a fine-grained credit assignment method that weights each reasoning and action step according to its counterfactual contribution to the final answer. In other words, useful steps are amplified, while ineffective or misleading steps are suppressed. This approach turns the supervision signal from "accepting everything" to "absorbing selectively."

From a methodological perspective, this is a broadly applicable idea in agent training—the quality of teacher model trajectories is almost always uneven, and fine-grained credit assignment preserves effective information better than simple filtering.

Teacher trajectories refer to complete interaction records sampled from a strong model (teacher) on agent tasks, including observations, reasoning, tool calls, and final answers, commonly used for supervised fine-tuning of smaller models (i.e., trajectory distillation). The core problem is uneven quality: within the same trajectory there are both critical steps that determine success and exploratory, backtracking, or even erroneous actions. Counterfactual credit assignment draws on the idea of credit assignment in reinforcement learning, estimating a single step's contribution by asking "how would the final outcome change if this step were removed or replaced," thereby concentrating supervision weight on the steps that truly affect the answer. Compared with weighting entire trajectories solely by whether the final answer is correct, this method has finer granularity and can suppress noise while preserving effective information—an increasingly noticed training technique in agent distillation.

Training Gains and Cross-Benchmark Performance

After training, DR Pixel improves retrieval recall on DRPixelBench by about 15 percentage points and final accuracy by more than 10 percentage points, and also outperforms the untrained baseline on four existing benchmarks.

More convincing is the out-of-distribution test: on ViDoRe-V3, the gap between DR Pixel and the oracle that is "directly given the golden page" is only 1.8 percentage points. This number shows that the page-level agent's retrieval capability is approaching the boundary of ideal retrieval—reasoning on pages it found itself is almost equivalent to being told which page holds the answer.

ViDoRe is a series of visual document retrieval evaluations proposed by Illuin and other teams, focusing on retrieval tasks at the page-image unit level and covering various layouts such as scientific papers, financial reports, and administrative documents. ViDoRe-V3 continues this series' setup, evaluating a model's ability to locate relevant pages within large-scale page corpora. The oracle setting refers to directly feeding the annotated golden page to the downstream reasoning model, representing the performance upper bound when retrieval is perfect; the gap between an agent and the oracle can therefore be interpreted as "retrieval loss." DR Pixel's 1.8-percentage-point gap from the oracle on this benchmark indicates that its self-retrieved pages are already very close to human-annotated pages in terms of reasoning effectiveness—a key metric for measuring the practicality of page-level retrieval.

Implications for Document Agent Architecture

DR Pixel's conclusions challenge a long-standing default engineering path in deep research: extract text via OCR, build a text index, then perform retrieval-augmented generation. The evidence presented by the research shows that this pipeline is neither necessary nor optimal for complex documents.

The potential advantages of the page-native approach are multifaceted: layout information is preserved, charts and formulas can be understood without conversion, and fewer tool calls mean lower latency and cost. For applications that need to process visually rich documents such as research papers, financial reports, and technical manuals, this offers a new route worth evaluating.

Of course, this direction also brings new engineering considerations: the storage and transmission costs of page images, the compute overhead of VLM inference, and retrieval efficiency over large-scale page corpora are all variables to weigh in deployment. As open-source work, DR Pixel provides the community with a reproducible starting point.

One-line Summary

DR Pixel replaces OCR text with page images, proving that page-native deep research is both more effective and more efficient.

Paragraph Summary

The article introduces the open-source work DR Pixel: the first end-to-end multimodal deep research agent that uses rendered page images as the sole retrieval and reasoning unit, operating over a 1.06 million-page corpus. The team built the DRPixelBench benchmark, requiring models to reason across text, charts, tables, and formulas. Experiments found that in the zero-training state, the page-based approach comprehensively outperforms OCR baselines on GPT-5.4, Gemini-3.5-Flash, and Qwen3.6-27B with fewer tool calls. In further training, to address teacher-trajectory noise, they propose counterfactual credit reweighting, weighting steps by their counterfactual contribution to the answer, improving retrieval recall by about 15 percentage points and accuracy by more than 10 percentage points; on ViDoRe-V3, the gap from the golden-page oracle is only 1.8 percentage points. The conclusions challenge the OCR-then-retrieve pipeline, arguing that the page-native approach is superior for visually rich documents, while also introducing new engineering trade-offs in storage, compute, and retrieval efficiency.