Bounded Morality: Defining the Space of Moral Computation
Quick Answer
The paper introduces 'Bounded Morality,' a framework analyzing moral computation for finite agents, balancing moral breadth and depth under resource constraints.
Quick Take
The paper introduces 'Bounded Morality,' a framework analyzing moral computation for finite agents, balancing moral breadth and depth under resource constraints. It suggests that moral alignment in AI systems relies on the allocation of reasoning capacity rather than mimicking human judgments.
Key Points
- Bounded Morality formalizes moral cognition beyond fixed ethical theories.
- Moral breadth and depth are defined as key dimensions of moral computation.
- The framework highlights trade-offs in moral reasoning due to limited resources.
- Ethical theories are viewed as efficient strategies for different moral demands.
- Moral alignment in AI depends on reasoning capacity allocation.
Paper Resources
Article Excerpt
From source RSS / original summaryarXiv:2607. 00002v1 Announce Type: new Abstract: Moral cognition has traditionally been modeled as adherence to fixed ethical theories--deontology, consequentialism, virtue ethics--implemented as static rules or value functions. We propose Bounded Morality, a formal framework for analyzing the computational demands of moral problems faced by finite agents.
Extending Herbert Simon's notion of bounded rationality, we formalize moral situations along two orthogonal dimensions: moral breadth, the scope of entities treated as morally relevant, and moral depth, the inferential integration required to evaluate their interactions. Limited resources impose an unavoidable tradeoff between these dimensions, defining a feasible space of moral computation.
Within this space, ethical theories correspond to locally efficient strategies adapted to different demand regimes rather than competing accounts of moral truth. The framework yields a formal notion of moral regret and moral progress under constraint, and implies that moral alignment in artificial systems depends on the scaling and allocation of moral reasoning capacity rather than on direct imitation of human judgments.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →The Verification Horizon: No Silver Bullet for Coding Agent Rewards
As coding agents evolve, verifying solutions becomes more challenging than generating them, necessitating a focus on scalable, faithful, and robust verification methods. The study reveals that no fixed reward function can sustain effectiveness as model capabilities advance, emphasizing the need for verification to evolve alongside solution generation.