When AI Takes Sides on Questions of Faith: Persistent Asymmetries in AI-Mediated Faith Guidance
Quick Take
Large language models (LLMs) show persistent asymmetries in religious conversion advice, favoring Catholicism, Bahá'í, and Sikhism while discouraging Atheism, Agnosticism, and Jehovah's Witnesses. Testing 20 models revealed that Grok 4.20 exhibited the strongest biases, with implications for real-world faith guidance.
Key Points
- Models favored certain religions for conversion while discouraging others.
- Grok 4.20 showed the strongest asymmetries among 20 tested models.
- Catholicism, Bahá'í, and Sikhism received high support for joining.
- Atheism, Agnosticism, and Jehovah's Witnesses were primarily disfavored.
- Asymmetries persisted across various question phrasings and models.
Article Content
From source RSS / original summaryarXiv:2605. 22975v1 Announce Type: new Abstract: We ask whether large language models (LLMs) treat queries about religious conversion symmetrically. The answer is no. When asked for advice on hypothetical faith transitions from one religion to another, then asked the reversed question, models exhibited consistent asymmetries, favoring some religions while subtly discouraging conversion to others.
On average Catholic, Bah\'a'\'i, and Sikh religions were broadly favored (high support for joining, low support for leaving), while Atheists, Agnostics, and Jehovah's Witnesses were primarily disfavored. Patterns varied by model size and model provider, with Grok 4. 20 exhibiting the strongest asymmetries. We tested 20 commercial and open-source language models across 182 religion pairings using a human-verified LLM-as-a-judge framework.
Each model was probed via interactions with a simulated user asking for advice on a potential faith conversion. Models tended to use more encouraging language for some faith transitions over others; these patterns were systematically repeatable across multiple trials. All LLMs tested exhibited reproducible asymmetry, though the pattern of preferences differed for each. Overall preferences persist across multiple question phrasings and variations in the religious pairing dataset.
Taken together, these results suggest that asymmetry is a robust property of model behavior rather than an artifact of how the models' answers were scored. It is important to consider that any imbalances deployed and reproduced en masse can have real-world implications.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?
The reliability of LLM judges for evaluating deep research agents is critically assessed using the REFLECT benchmark.