On the Computational Complexity of Structural Generalization
Quick Answer
This paper formalizes structural generalization in computational complexity, showing that pure Transformers cannot learn it under the assumption that TC0 ≠ NC1.
Quick Take
Neuro-symbolic systems outperform pure Transformers by incorporating semantic rules, highlighting a significant gap in benchmark evaluations.
Key Points
- Structural generalization is defined mathematically for the first time.
- Pure Transformers are limited to the learnable class TC0, unable to achieve structural generalization.
- Neuro-symbolic systems excel in benchmarks by leveraging semantic rules.
- The paper argues that benchmark scores fail to differentiate learned from hard-coded rules.
- A Montagovian approach reveals the complexity of compositional rules in Transformers.
Paper Resources
📖 Reader Mode
~3 min read
Bibliographic and Citation Tools
Code, Data and Media Associated with this Article
Demos
Recommenders and Search Tools
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.