Calibrating Artificial Guilt: Neurally Grounded Reward Shaping for Prosocial Multi-Agent Reinforcement Learning
Quick Answer
This study introduces a method for calibrating guilt signals in multi-agent reinforcement learning using human neural data, achieving closer alignment with human social choices.
Quick Take
The calibrated agents demonstrated a social safe-choice rate of 0.459, closely matching the human rate of 0.484, outperforming other reward shaping methods significantly.
Key Points
- Guilt weight calibrated from fMRI data shows significant impact on agent behavior.
- Calibrated agents achieved a KL divergence of 0.0012 compared to human choices.
- Four reward shaping regimes were tested: neurally calibrated, uniform, zero, and oracle.
- The study utilized the SoDec responsibility fMRI dataset with 40 participants.
- Neurobehavioral priors can enhance prosocial behavior in artificial agents.
DeepSignal Analysis
What happened
The study presents a method for calibrating guilt signals in multi-agent reinforcement learning by utilizing human neural data. The calibrated agents achieved a social safe-choice rate of 0.459, closely aligning with the human rate of 0.484, and significantly outperformed other reward shaping methods.
Key evidence
- The research used the SoDec responsibility fMRI dataset, which included data from 40 participants, to derive a guilt weight for the agents.
- Calibrated agents were tested in a Social Lottery environment using Proximal Policy Optimization, showing a close match to human social choices with a KL divergence of 0.0012.
- The study compared four reward shaping regimes, revealing that the calibrated method significantly outperformed uniform constant, zero, and unit-coefficient oracle methods in tracking human behavior.
Why it matters
This research highlights the potential of integrating human neurobehavioral data into artificial intelligence systems, particularly in multi-agent environments. By aligning artificial agents' behaviors more closely with human social choices, the findings could enhance the effectiveness of collaborative AI applications in real-world scenarios.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Cooperative multi-agent reinforcement learning often adds social terms to individual rewards, yet the scale of those terms is usually chosen by hand. We ask whether a guilt signal can instead be calibrated from human neural and behavioural data and transferred to artificial agents. Using the public SoDec responsibility fMRI dataset (40 participants), we fit a subject-fixed-effects regression of momentary-happiness changes on outcome-type counts and recover a guilt weight as the Partner-negative minus Social-negative contrast ($\hat{w}=1.118$, Cohen's $d=0.214$). We embed this weight in a two-agent Social Lottery environment and train independent Proximal Policy Optimization actor-critics under four shaping regimes: neurally calibrated, uniform constant, zero (selfish), and a unit-coefficient oracle. Across 1{,}000 evaluation episodes per condition, the calibrated agents track the human Social safe-choice rate most closely ($0.459$ vs.\ human $0.484$; $\mathrm{KL}=0.0012$), while the other three conditions deviate by one to three orders of magnitude in KL. Human neurobehavioural priors can therefore act as quantitative constraints on prosocial reward shaping.
| Comments: | 12 pages, 6 figures, 3 tables |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.04663 [cs.AI] |
| (or arXiv:2608.04663v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2608.04663 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Arya Shah [view email]
[v1]
Wed, 5 Aug 2026 10:21:11 UTC (1,196 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →HOBA: Hierarchical On-Policy Bidding Agents for Adaptive Online Advertising
HOBA (Hierarchical On-policy Bidding Agents) is a novel hierarchical reinforcement learning framework that enhances online advertising bidding systems by improving adaptability and reducing hyperparameter tuning costs. It utilizes a for hyperparameter inference, a SARSA agent for expert model selection, and a dynamic expert pool for bid execution, achieving a +3.6% increase in target cost during large-scale deployment and outperforming state-of-the-art baselines on AuctionNet.