The Capacity of Thought: Benchmarking Llama 3.2 in Semantic fMRI Neural Language Decoding and Improving the Huth Encoding-Model Baseline
Quick Answer
The study enhances the Huth encoding model for fMRI language decoding, achieving a 11% METEOR improvement, and introduces fMRIFlamingo with Llama-3.2-1B, revealing that high-capacity models may obscure decoding failures without proper controls.
Quick Take
The study enhances the Huth encoding model for fMRI language decoding, achieving a 11% METEOR improvement, and introduces fMRIFlamingo with Llama-3.2-1B, revealing that high-capacity models may obscure decoding failures without proper controls.
Key Points
- Improved Huth model with 15K voxel selection and GPT-2, achieving METEOR = 0.149.
- fMRIFlamingo maps BOLD activity to Llama-3.2-1B, achieving 42.86% Top-1 accuracy.
- Blind control tests show decoding success may rely on language prior, not neural input.
- Study highlights challenges in non-invasive brain-computer interface research.
- Results indicate high-capacity models do not inherently enhance fMRI decoding.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Decoding continuous language from fMRI signals remains a core challenge in non-invasive brain-computer interface research. We present two complementary investigations. First, we improve the Huth et al. ridge regression encoding pipeline through expanded voxel selection (10K->15K), substitution of GPT-2 medium for GPT-1 as the beam-search proposal model, and GPU-accelerated bootstrap training, achieving mean METEOR = 0.149 and BLEU-1 = 0.200 across three held-out narratives for subject UTS03 -- an 11% relative METEOR gain over our replication baseline. Second, we introduce fMRIFlamingo, which maps BOLD activity to a frozen Llama-3.2-1B with trainable gated cross-attention layers via a learned brain tokenizer and a Perceiver Resampler. Despite achieving 42.86% Top-1 accuracy on a 1-in-100 ranking task, well above chance, a blind control ablation with zeroed fMRI inputs yields near-identical scores, revealing that apparent decoding success is driven primarily by the frozen language prior rather than by neural input. These results demonstrate that high-capacity language models do not inherently improve fMRI decoding and can actively obscure failures without rigorous blind-control evaluation.
| Comments: | 7 pages, 2 tables, 2 figures. Preprint. NLP 244: Advanced Machine Learning for Natural Language Processing report, UC Santa Cruz |
| Subjects: | Computation and Language (cs.CL) |
| MSC classes: | 92C55, 68T50 |
| ACM classes: | I.2.7; J.3 |
| Cite as: | arXiv:2607.12079 [cs.CL] |
| (or arXiv:2607.12079v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.12079 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aidan Pinero [view email]
[v1]
Mon, 13 Jul 2026 18:56:17 UTC (164 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.