LLM Doesn't Know What It Doesn't Know: Detecting Epistemic Blind Spots via Cross-Model Attribution Divergence on Clinical Tabular Data
Quick Answer
This study reveals that large language models (LLMs) like Qwen 2.5 7B struggle with epistemic self-awareness on clinical tabular data, showing constant confidence levels regardless of accuracy.
Quick Take
This study reveals that large language models (LLMs) like Qwen 2.5 7B struggle with epistemic self-awareness on clinical tabular data, showing constant confidence levels regardless of accuracy. By employing cross-model attribution divergence, the research demonstrates that integrating few-shot examples and SHAP-derived features can significantly enhance prediction accuracy from 49% to 75.3% and reduce calibration error.
Key Points
- LLM confidence remains constant (0.856-0.937) despite varying accuracy levels.
- Accuracy drops to 64.8% when XGBoost is 99% correct, indicating an inverse difficulty effect.
- Few-shot examples and SHAP features reduce Attribution Disagreement Score from 1.54 to 0.38.
- Cross-model calibrator improves expected calibration error from 0.254 to 0.080.
- Findings highlight a cold start problem for LLMs on structured data.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 19509v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly applied to structured clinical data, yet whether they can recognize the limits of their own knowledge on such tasks remains unexplored. We study this question through the lens of cross-model attribution divergence with the goal of reducing epistemic uncertainty for structured tasks, comparing Qwen 2. 5 7B and XGBoost on a prediction task via attribution divergence analysis. We report four findings.
First, LLM verbalized confidence is epistemically vacuous, it outputs a near-constant (0. 856-0. 937) regardless of whether accuracy is 49% or 75. 3%, tracking prompt format rather than prediction quality. Second, the LLM exhibits an inverse difficulty effect: accuracy drops to 64. 8% when XGBoost is 99% correct, but matches XGBoost (73. 8% vs. 73. 1%) when it is moderately uncertain.
Third, few-shot examples and SHAP-derived feature evidence are orthogonal, super-additive interventions: they reduce the Attribution Disagreement Score (ADS) from 1. 54 to 0. 38 and improve accuracy from 49% to 75. 3% without training. Fourth, a cross-model calibrator that determined LLM reliability using attribution divergence signals reduces expected calibration error from 0. 254 to 0.
080, replacing uninformative verbalized confidence with patient-specific reliability estimates, without accessing model internals or requiring repeated inference. We frame these findings as a cold start problem for LLMs on structured data and outline a path toward genuine epistemic self-awareness.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Arbor: Tree Search as a Cognition Layer for Autonomous Agents
Arbor introduces a multi-agent framework utilizing structured tree search for optimizing LLM inference, achieving up to 193% throughput-latency improvement compared to vendor-optimized systems. It employs an Orchestrator and Critic agent for stability and coordination, demonstrating hardware-agnostic performance with minimal variance.