Beyond Parallel Tracking: Interactive Multi-Feature Fusion Drives Semantic Reconstruction from Non-invasive Brain Recordings
Quick Answer
This study presents a multi-feature fusion framework for semantic reconstruction from non-invasive brain recordings, outperforming traditional methods.
Quick Take
The non-linear Multi-Head Cross-Attention approach achieved state-of-the-art results, demonstrating improved alignment between neural signals and semantic features, crucial for brain-to-text decoding.
Key Points
- Introduces a multi-feature fusion framework for semantic reconstruction.
- Benchmarks linear Naive Concatenation against non-linear Multi-Head Cross-Attention.
- Achieves superior performance hierarchy: Cross-Att > Concat > GPT > W2V.
- Demonstrates the importance of integrating lexical and contextual information.
- Offers a viable method for non-invasive brain-to-text decoding.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Continuous semantic reconstruction from non-invasive neural recordings remains limited by the representational mismatch between semantic feature spaces and neural coding patterns, which severely impedes cross-modal alignment between high-noise neural signals and target semantic features. Prior semantic decoders have predominantly relied on static lexical representations or dynamic contextualized representations in isolation. This single-dimension approach inevitably leads to severe information loss, as it fails to account for the human brain's capacity to integrate stable word attributes and dynamic contexts this http URL bridge this gap, this study introduces a multi-feature fusion framework for non-invasive semantic reconstruction, systematically benchmarking two integration approaches: linear Naive Concatenation and non-linear Multi-Head Cross-Attention. Within this framework, our approach complements static lexical representations (W2V) with dynamic contextual representations (GPT) via an interactive gating mechanism to facilitate cooperative processing during language this http URL through extensive semantic reconstruction and text generation experiments, our framework reveals a robust performance hierarchy: Cross-Att > Concat > GPT > W2V. Crucially, the non-linear cross-attention fusion method achieves state-of-the-art performance, demonstrating that neural language decoding benefits from simulating the collaborative modulation between contextual information and core lexical attributes rather than depending on isolated individual features, while also offering a viable non-invasive brain-to-text decoding method.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.12071 [cs.CL] |
| (or arXiv:2607.12071v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.12071 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Boda Xiao [view email]
[v1]
Mon, 13 Jul 2026 18:44:40 UTC (1,689 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.