Encoding EEG Signals to Examine Human-Like Next-Word Prediction Behaviour in Language Models
Quick Answer
This study investigates how well advanced language models (LMs) predict next words compared to human cognitive responses, using EEG data.
Quick Take
It finds that while LMs achieve high prediction accuracy, only the surprisal measure correlates with human-like event-related potentials, challenging the assumption that larger models always improve linguistic processing alignment with humans.
Key Points
- Advanced LMs show high accuracy in next-word prediction tasks.
- Surprisal measure correlates with human-like cognitive processing in EEG.
- Modeling ERP patterns provides insights into cognitive plausibility of LMs.
- Findings suggest scaling LMs doesn't guarantee better human-like processing.
- Study raises questions about the cognitive alignment of LMs and humans.
DeepSignal Analysis
What happened
The study explores the performance of advanced language models (LMs) in predicting next words compared to human cognitive responses, utilizing EEG data. It finds that while LMs demonstrate high prediction accuracy, only the surprisal measure aligns with human-like brain responses, questioning the assumption that larger models inherently improve linguistic processing alignment with humans.
Key evidence
- Advanced language models achieve prediction accuracies closely aligned with human performance in next-word prediction tasks.
- The study generates regressors based on top-1 prediction and surprisal to predict event-related potentials (ERPs) from EEG recordings.
- Only the surprisal measure shows potential correlation with language-processing ERPs, particularly for open-class words with high semantic content.
Why it matters
This research is significant as it challenges the prevailing notion that increasing the size and complexity of language models will lead to better alignment with human cognitive processes. It suggests that understanding the underlying cognitive mechanisms, as reflected in EEG data, is crucial for evaluating the effectiveness of LMs in language tasks. The findings may influence future developments in AI language processing and its applications in understanding human cognition.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Language models (LMs) are trained to excel at predicting the next word in the sequence given prior context, and humans also share this predictability in reading comprehension. Neuroscience research reveals that next-word predictability influences brain response, as recorded at millisecond resolution using electroencephalography (EEG). While our evidence indicates that advanced LMs achieve accuracies closely aligned with human performance at the next-word prediction task, this raises the question: Does higher prediction accuracy necessarily mean that these models adequately capture the cognitive signals associated with human reading comprehension? Here, we generate regressors for both humans and LMs based on two information measures, including top-1 prediction and surprisal, to predict event-related potential (ERP) elicited from EEG recordings which reflect different stages of cognitive processing during reading. We argue that modelling ERP patterns offers fine-grained analysis of the cognitive plausibility of various LMs during reading. Our results indicate that only surprisal potentially correlates with language-processing ERPs, especially for open-class words with high semantic content. Moreover, our findings challenge the assumption that scaling LMs with increased parameters and computational budgets will consistently lead to improved convergence with human-like linguistic processing.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.16549 [cs.CL] |
| (or arXiv:2607.16549v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.16549 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mai-Boi Quach [view email]
[v1]
Fri, 17 Jul 2026 23:11:16 UTC (5,268 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.