Quantifying Aleatoric Uncertainty of In-Context Learning for Robust Measure of LLM Prediction Confidence
Quick Answer
This paper introduces self-function vectors to quantify aleatoric uncertainty in In-Context Learning (ICL) for LLMs, enhancing prediction reliability.
Quick Take
The proposed evaluation protocol allows for precise separation of aleatoric from epistemic uncertainty, demonstrating improved measurement of predictions over existing methods. This advancement is crucial for applications like hallucination detection.
Key Points
- Introduces self-function vectors for modeling aleatoric uncertainty in LLMs.
- Proposes a new evaluation protocol for separating aleatoric and epistemic uncertainty.
- Demonstrates improved reliability in measuring LLM predictions under ICL.
- Framework applicable to real-world datasets and synthetic tasks.
- Supports trustworthy applications like hallucination detection.
Paper Resources
📖 Reader Mode
~2 min readAbstract:In-Context Learning (ICL) allows LLMs to adapt to new tasks from a few demonstrations, but its reliability remains a concern: predictions are highly sensitive to both prompt design and the model's ability to understand the context, obscuring whether failures arise from data properties or model limitations. Uncertainty decomposition-separating aleatoric from epistemic sources-is particularly crucial in this setting, yet existing methods, designed for standard generation tasks, fail to capture the unique dynamics of ICL. To address this, we introduce a concept of self-function vectors, built upon Bayesian views and the mechanistic interpretability of ICL. These vectors leverage internal model representations to model the latent concept learned during in-context prompting, thereby enabling a direct estimation of aleatoric uncertainty within a Bayesian framework and circumventing the reliance on brittle input or decoding manipulations. Given the lack of established benchmarks and suitable evaluation protocols, we also propose the first and rigorous evaluation protocol, in which data is manipulated in controlled ways so as to quantify aleatoric uncertainty precisely and separately from epistemic uncertainty. With this new evaluation framework, initially grounded in synthetic tasks for conceptual development and subsequently extended to real-world datasets, we show that our proposed methodology can measure uncertainty of LLM predictions made under ICL more reliably than existing alternative methods. Moreover, we show it can be used as a practical tool for trustworthy-related applications, such as hallucination detection. Our findings pave a new direction for connecting the quantitative view of uncertainty with the mechanistic understanding of model behavior.
| Comments: | Accepted to ACL 2026 |
| Subjects: | Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.19353 [cs.CL] |
| (or arXiv:2606.19353v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.19353 arXiv-issued DOI via DataCite |
Submission history
From: Jinseok Chung [view email]
[v1]
Tue, 28 Apr 2026 09:47:40 UTC (1,668 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.