SCRIBE

SCRIBE's interpretability is built on the intermediate layer of a "semantic feature space" rather than directly dissecting the internal neurons of a model.

The research team recruited 921 participants for behavioral experiments to rate the semantic features of words.

The context-dependent semantic representations generated by SCRIBE can track shifts in semantic feature ratings of the same word across different contexts, reaching 75%–81% of the human noise ceiling.

SCRIBE significantly outperformed non-contextual semantic representations in predicting neural responses in language-selective brain regions during a natural narrative story listening task.

Variance decomposition analysis showed that SCRIBE still explained unique brain-predictive information beyond the syntactic features selected by the researchers.

SCRIBE did not train a new model; instead, it used an ensemble of open-source large language models to generate semantic representations, in order to lower the reproduction barrier and make the source of semantic representations more transparent.

SCRIBE relies on the semantic space of a large model ensemble, and whether this space is fully aligned with the brain's semantic organization remains an open question.

In SCRIBE's behavioral experiments, the noise ceiling estimate depends on the specific experimental design and participant group.

Although the natural narrative listening task is closer to real language use than traditional experiments, it is still a controlled setting.

SCRIBE is currently a research framework, not a product.

SCRIBE's methodological path—using large models to generate interpretable intermediate representations and then using neural data to verify their alignment with brain computation—can be transferred to other cognitive domains.

SCRIBE uses an ensemble of open-source large models to generate interpretable word-level semantic representations, and validated at both the behavioral experiment and brain imaging levels the consistency of these representations with human semantic judgments and brain activity.

SCRIBE is an interpretable contextual semantic representation covering 65 neurocognitively well-grounded semantic attributes.

Using natural fMRI data collected from 7 subjects while listening to narrative stories, the study uses variance decomposition techniques to split the variance in brain activity predicted by LLaMA2-7B and SCRIBE each into shared variance and their own unique variance.

Shared variance dominates both across the whole brain and in language-related regions, indicating that LLMs predict brain responses largely because they capture the semantic information encoded by SCRIBE.

Both LLaMA2-7B and SCRIBE representations retained significant unique contributions.

This study is the first to use an interpretable semantic framework to quantify how much of brain-LLM alignment comes from comprehensible semantics.

The concreteness semantic attribute showed significantly higher shared variance with LLaMA2-7B across the whole brain and most examined regions than abstract attributes.

Abstract attributes did not exhibit greater SCRIBE-unique variance; SCRIBE's unique contribution was distributed across a broader range of semantic dimensions.

Ablation experiments on semantic attributes revealed that semantic contributions span multiple dimensions, producing cumulative effects on the predicted variance of subjects' brains rather than being dominated by a single dimension.