The Ghost Annotator: a Framework to Explore Human Label Variation in Content Moderation through Conformal Prediction
Quick Take
The Ghost Annotator framework integrates conformal prediction with Collaborative Filtering to analyze human label variation in content moderation. It reveals that larger LLMs exhibit increased confidence in classifications misaligned with human annotations, highlighting demographic biases in pretraining data. Evaluations across four datasets show that model uncertainty rises with annotator disagreement.
Key Points
- Introduces Ghost Prediction metric to quantify model-human annotation divergence.
- Evaluates four LLMs, revealing increased uncertainty with annotator disagreement.
- Larger models show higher confidence in misaligned classifications.
- Demonstrates consistent demographic misalignment, indicating structural bias.
- Utilizes Non-Conformity Scores for detailed analysis of model behavior.
Article Content
From source RSS / original summaryarXiv:2606. 02911v1 Announce Type: new Abstract: Current research primarily focuses on model performance, while comparatively less attention has been devoted to uncertainty estimation, particularly in settings where LLMs are increasingly used to generate annotated data. We introduce a framework combining conformal prediction with Collaborative Filtering-style annotators' representation to model LLM behavior in relation to human annotators and to analyze patterns of agreement and disagreement.
Using Non-Conformity Scores, we introduce the Ghost Prediction metric and the Ghost Annotator representation to quantify cases in which model predictions diverge from all available human annotations. We compute cosine similarity measures to explore differences in model behavior across sociodemographic axes. We evaluated four LLMs of different size and families across four content moderation datasets.
Our finding shows that while we find that all models uncertainty increases with annotator disagreement, larger models tend to be more confident in the classification of texts that are not aligned with any human annotation. Finally, the Ghost Annotator framework reveals a consistent and robust pattern of demographic misalignment, suggesting a structural bias likely rooted in pretraining corpora.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
The REFLECT benchmark reveals that current LLM judges are unreliable, achieving below 55% accuracy in evaluating reasoning and evidence use, highlighting the need for improved evaluation methods for deep research agents.