Adversarial Concept Search: Predicting Compositional Errors From Feature Geometry
Quick Answer
This paper introduces a method to predict compositional errors in LLMs by analyzing their representational geometry.
Quick Take
By identifying interference between salient features, the approach anticipates failure modes in tasks like multihop reasoning and multilingual recall, enabling targeted stress tests and enhancing active learning for real-world applications.
Key Points
- Predicts failure modes using representational geometry without specific input evaluation.
- Identifies interference between features as a cause of compositional failure.
- Demonstrates reliable composition in near-orthogonal concept pairs.
- Applicable to tasks requiring systematic composition like multilingual factual recall.
- Lays groundwork for scalable active learning and stress testing in deployment.
Paper Resources
Article Excerpt
From source RSS / original summaryarXiv:2606. 13934v1 Announce Type: new Abstract: Humans cannot always intuit what scenarios are most challenging to . Hoping to capture challenging edge cases, developers either design problems to be difficult for humans or curate extensive benchmarks. What if we could instead anticipate which scenarios a model will fail on? In this paper, we use an LLM's representational geometry to predict which concept combinations it will fail on.
We attribute this compositional failure to interference between salient features. In tasks that require systematic composition - toy programmatic settings, multihop reasoning, multilingual factual recall - we find that when a pair of concepts is encoded near-orthogonally, the model reliably composes them. When their linear encodings are close, producing interference, the model fails to compose them.
Our method reliably anticipates failure modes across different compositional tasks, without evaluating specific inputs. These results lay the groundwork to use representational geometry to identify high-risk examples, construct targeted stress tests, and provide a scalable foundation for active learning in real-world deployment.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.