Are Arithmetic Heuristic Neurons Form-Invariant? A Mechanistic Analysis of Symbols, Text, and Code in LLMs
Quick Answer
The study investigates arithmetic heuristic neurons in Llama-3 models, revealing that they are form-invariant across symbolic arithmetic, natural language problems, and Python code.
Quick Take
A shared set of neurons enables high accuracy in arithmetic tasks, with over 97% recovery of incorrect predictions when transferring activations between formats, indicating that failures stem from activation states rather than distinct circuits.
Key Points
- Arithmetic heuristic neurons in Llama-3 models are form-invariant across different formats.
- A compact set of neurons is shared among symbolic arithmetic, natural language, and code.
- Transferring activations recovers over 97% of incorrect predictions for addition and subtraction.
- Failures in arise from activation states, not distinct internal circuits.
- Shared neurons belong to the same heuristic families across formats.
DeepSignal Analysis
What happened
The study examines arithmetic heuristic neurons in Llama-3 models, finding that these neurons are form-invariant across different formats such as symbolic arithmetic, natural language problems, and Python code. The research indicates that a shared set of neurons is responsible for high accuracy in arithmetic tasks, with over 97% recovery of incorrect predictions when transferring activations between formats.
Key evidence
- The study identifies arithmetic heuristic neurons in three Llama-3 models using a two-stage pipeline of attribution and activation patching.
- A compact set of neurons is shared across symbolic arithmetic, natural language problems, and Python code, demonstrating their necessity for arithmetic computation.
- Transferring activations from successful executions to failed ones recovers over 97% of incorrect predictions, suggesting that failures are due to activation states rather than separate circuits.
Why it matters
Understanding the form-invariance of arithmetic heuristic neurons in LLMs could lead to improved model designs that leverage shared circuits for better performance across various problem formats. This insight may also inform future research on the interpretability of neural networks and their underlying mechanisms, potentially enhancing their reliability in critical applications.
Paper Resources
Source Excerpt
often succeed on one formulation of a problem while failing on an equivalent formulation. Whether these failures arise from distinct internal circuits or different activation states of a shared circuit remains unknown. Recent mechanistic interpretability studies suggest that arithmetic in LLMs emerges from a "bag of heuristics," encoded by a sparse set of MLP neurons that represent distinct arithmetic strategies. We investigate whether arithmetic heuristic neurons are form-
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.