Neural Activation Patterns Across Language Model Architectures: A Comprehensive Analysis of Cognitive Task Performance
Quick Take
The study analyzes neural activation patterns in six LLM architectures across twelve cognitive tasks.
Key Points
- Mathematical reasoning shows highest attention entropy across all models.
- Decoder models exhibit greater sparsity than encoder models.
- Findings inform model selection and optimization for big data.
📖 Reader Mode
~2 min readAbstract:This paper presents a comprehensive analysis of neural activation patterns across six distinct large language model (LLM) architectures, examining their performance on twelve cognitive task categories. Through systematic measurement of final activation values, attention entropy, and sparsity patterns, we reveal fundamental differences in how encoder and decoder architectures process diverse cognitive tasks. Our analysis of 144 task-model combinations demonstrates that mathematical reasoning consistently produces the highest attention entropy across all architectures, while decoder models exhibit significantly higher sparsity patterns compared to encoder models. The findings provide critical insights into the computational characteristics of modern language models and their task-specific neural behaviors, with implications for model selection and optimization in big data applications.
| Comments: | 8 pages, accepted at IEEE BigData 2025 |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2605.15436 [cs.CL] |
| (or arXiv:2605.15436v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.15436 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mahdi Naser Moghadasi [view email]
[v1]
Thu, 14 May 2026 21:31:19 UTC (12 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →arXiv cs.CL·Leyao Wang, Yanan He, Peng Chen, Asaf Yehudai, Yixin Liu, Rex Ying, Michal Shmueli-Scheuer, Arman Cohan
2d ago FeaturedOriginal
Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
AI Summary
The reliability of LLM judges for evaluating deep research agents is critically assessed using the REFLECT benchmark.