On the Persistent Effects of Lexicality in Large Language Mod
Quick Take
This study reveals that lexical overlap significantly influences representations in large language models (LLMs), affecting their performance across various architectures and tasks. The findings indicate a mid-depth region where both lexical and semantic signals degrade, leading to poor representations for meaning and surface form, impacting applications like summarization and model editing.
Key Points
- Lexical influence persists across model depth and architectures, affecting performance.
- Mid-depth region shows simultaneous degradation of lexical and semantic signals.
- Study connects findings to information theory, enhancing understanding of LLMs.
- Implications observed in downstream tasks like summarization and model editing.
- Lexical overlap often overshadows semantic content in LLM representations.
Article Content
From source RSS / original summaryarXiv:2606. 02750v1 Announce Type: new Abstract: Representations extracted from large language models (LLMs) play an important role in many downstream applications. However, the structure of these representations is often influenced by lexical overlap rather than semantic content. Our understanding of the relationship between this lexical influence and semantic content, and its implications for downstream tasks, remains limited.
In this work, we investigate representations to quantify the effect of lexical overlap relative to semantic content. We consider several adversarial semantic stress tests and further connect our findings to the information theory perspective. We find that lexical influence extends across the depth of models, consistently across architectures, training regimes, and objective functions, including the models trained for semantic similarity.
Moreover, we observe a mid-depth region in which both lexical and semantic signals degrade simultaneously, indicating a transitional regime where representations are poor for both surface form and meaning. We further demonstrate the effect of lexical influence on downstream uses of LLMs using summarization and model editing as a case study.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
The REFLECT benchmark reveals that current LLM judges are unreliable, achieving below 55% accuracy in evaluating reasoning and evidence use, highlighting the need for improved evaluation methods for deep research agents.