Using AI Agents to Automate Black-Box Audits of Personalization Algorithms at Scale
Quick Answer
This study introduces a framework using generative AI agents for black-box audits of personalization algorithms, revealing that X's algorithm amplifies toxic content based on user ideology.
Quick Take
This study introduces a framework using generative AI agents for black-box audits of personalization algorithms, revealing that X's algorithm amplifies toxic content based on user ideology. The deployment of 1,120 agents across 14 personas collected over 200,000 content exposures, demonstrating significant variations in content delivery influenced by demographic signals.
Key Points
- Generative AI agents serve as behavioral engines for synthetic accounts in audits.
- 1,120 agents were deployed on X post-2024 U.S. election, spanning 14 personas.
- Over 200,000 content exposures were collected to analyze algorithmic behavior.
- X's algorithm amplifies toxic and right-leaning content compared to chronological feeds.
- Demographic signals affect content delivery in varied ways across different personas.
Paper Resources
Article Content
From source RSS / original summaryarXiv:2606. 30801v1 Announce Type: new Abstract: Personalization algorithms determine what content users encounter on online platforms. Auditing these systems is difficult because independent auditors have only black-box access to the algorithms, while personalization depends on users' attributes, behavior, and evolving interaction histories.
Existing auditing methods face a tradeoff: studies with real users capture realistic behavior but are costly and hard to control, whereas sock-puppet audits scale more easily but often rely on scripted behavior that limits realism. Beyond this, both approaches struggle to decouple user attributes from user behavior, limiting our ability to causally understand personalization.
To address this gap, we introduce a framework for black-box audits of personalization algorithms using generative AI agents as behavioral engines for synthetic accounts. Each agent is instantiated with a fixed persona, grounded in demographic and political survey data, and interacts with a platform's content by reasoning about it and choosing actions.
Because behavior is fixed within each persona while platform-visible signals such as age, gender, or location can be experimentally perturbed, our design enables counterfactual auditing of how platforms respond to user attributes. As a case study, we deploy 1,120 agents on X shortly after the 2024 U. S. election, spanning 14 personas and three counterfactual conditions, collecting over 200,000 content exposures.
We find that X's algorithmic feed amplifies toxic, polarizing, political, and right-leaning content relative to the chronological feed, with amplification varying sharply by user ideology. Counterfactual analyses show that demographic signals affect content delivery in persona-dependent ways: pooled effects are largely null, while subgroup-level effects vary in direction and magnitude. Our work establishes GenAI-based agents as a new tool for algorithmic auditing.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Quantifying Prior Dominance in Systems
The study introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, revealing that Small Language Models (SLMs) outperform larger models in factual extraction. The findings indicate that traditional scaling laws yield diminishing returns, with a commercial API frequently failing against adversarial evidence due to systemic confidence collapse.