AVID: The First 5T fMRI Benchmark Dataset for Auditory-Induced Visual Mental Imagery Decoding

Visual Mental Imagery (VMI) decoding attempts to reconstruct internally generated visual scenes from brain activity, offering a potential path toward externalizing inner experience and developing neural prosthetic communication interfaces. However, this field has long been constrained by the lack of dedicated, training-scale datasets. Existing paradigms mostly rely on visual cues or short-term memory images, introducing confounds between vision and memory that make it difficult for researchers to distinguish neural activity related to endogenous image construction from signals driven by prior or concurrent visual input. In addition, existing datasets offer limited control over the description of individual visual attributes, making it hard to systematically evaluate fine-grained visual details.

Why a Dedicated VMI Dataset Is Needed

The core challenge of VMI decoding lies in separating "internally generated" visual experience from external sensory input. If subjects still have visual cues in front of them while imagining, or need to rely on short-term memory to maintain an image, the recorded brain activity inevitably carries components of visual encoding and memory maintenance. This allows decoding models to learn shortcuts rather than truly capturing the neural mechanisms of imaginative construction.

AVID's starting point is precisely to eliminate this sensory contamination. It uses auditory descriptions to induce imagery rather than visual cues, reducing the interference of visual input on brain signals at the source. At the same time, the dataset reaches training scale—about 67 hours of dense recordings from nine participants, which represents a significant resource investment in VMI research.

Visual Mental Imagery (VMI) refers to the cognitive process of actively constructing visual scenes in the "mind's eye" in the absence of external visual input. It is distinct from passively recalling an image just seen (which would introduce short-term memory components). VMI relies on top-down activation of cortical areas such as the parietal and occipital cortex, and shares part of its neural representation with real vision, so the induced brain activity is easily confused with visual encoding signals. This is exactly why AVID emphasizes "auditory induction, no visual cues": only by excluding external visual input can the signals of endogenous imaginative construction be more purely isolated.

Dataset Structure and Design

AVID is organized into two complementary subsets, corresponding to different research needs.

Scene-Level Complex Subset

This subset uses naturalistic Mandarin descriptions to induce imagery. After hearing a spoken description of a scene, participants construct the corresponding visual picture in their minds. This design is closer to the scenarios in which imagination occurs in the real world—we usually trigger imagination through language or internal monologue rather than staring at a picture.

Controlled Simple Subset

The other subset explicitly specifies visual attributes, providing finer descriptive control over the imagined content. This allows researchers to systematically examine how specific visual dimensions such as color, shape, and spatial layout perform in decoding, without worrying about multiple attributes being entangled in natural scenes.

This dual-track structure of "natural + controlled" balances ecological validity and experimental controllability—a pragmatic trade-off in AVID's data design.

Standardized Evaluation Protocol

Data alone is not enough. The VMI decoding field previously lacked unified evaluation standards, making results across studies difficult to compare directly. AVID proposes a standardized evaluation protocol that decouples source-image consistency from semantic alignment.

The significance of this distinction is that a reconstruction may not match the original image at the pixel level, yet correctly capture the scene content described at the semantic level. Evaluating the two separately enables a more nuanced judgment of what the model has actually learned—pixel-by-pixel visual details, or scene-level semantic structure. For a task like auditory-induced imagery, where there is inherently no single "correct image," this evaluation approach is especially reasonable.

Source-image consistency measures how well the reconstructed image matches the original target image in pixel or low-level features, such as structural similarity or correlation distance; semantic alignment measures the degree to which the reconstruction agrees with the described content in terms of category, object, or scene semantics, often quantified using embedding similarity from pretrained visual models or classification accuracy. In VMI tasks, the image each person imagines is not unique, so pixel-level comparison is overly strict, whereas semantic alignment can reflect whether the model has captured "what was imagined." Decoupled evaluation is therefore more faithful to the nature of the task.

What the Baseline Results Reveal

The baselines and quantitative results reported in the paper show that current reconstruction pipelines still perform limitedly on AVID. However, the most successful reconstructions are able to preserve identifiable scene-level semantics from the spoken descriptions.

This result conveys two messages. On one hand, auditory-induced VMI decoding is feasible at the semantic level, which provides basic confidence for subsequent model development. On the other hand, reconstruction of fine-grained visual details remains an open problem, and existing pipelines are still clearly far from a practical neural prosthetic communication interface. AVID's value lies precisely in providing a benchmark that exposes these shortcomings, rather than showcasing only success cases.

Impact on the Field

AVID's contributions can be summarized at three levels. At the data level, it provides the first 5.0 Tesla fMRI training-scale resource for auditory-induced VMI decoding, filling the gap in dedicated datasets. At the methodological level, the dual-track subset design offers selectable experimental conditions for different research questions. At the evaluation level, the decoupled consistency/semantic protocol establishes a common language for cross-study comparison.

For researchers working in neural decoding, brain-computer interfaces, and multimodal representation learning, AVID is both a directly usable data foundation and a testbed for examining whether models truly learn "imagination" rather than "visual input." As more models iterate on this benchmark, the timeline for VMI decoding to move from the semantic level to the detail level may become clearer.

One-line Summary

AVID releases the first 5.0T fMRI training-scale dataset for auditory-induced VMI decoding, along with a decoupled evaluation protocol.

Paragraph Summary

To address the lack of dedicated training-scale datasets for Visual Mental Imagery (VMI) decoding, AVID constructed a 5.0 Tesla fMRI dataset of about 67 hours from nine participants. It uses auditory descriptions rather than visual cues to induce imagery, reducing visual and memory confounds, and is divided into a naturalistic scene-level subset and a controlled subset with explicitly specified visual attributes, balancing ecological validity and controllability. The paper also proposes a standardized evaluation protocol that decouples source-image consistency from semantic alignment, enabling nuanced judgment of whether models learn pixel details or scene semantics. Baselines show that reconstruction remains limited but can preserve identifiable scene-level semantics.