Identifying and Understanding Human Values in Text: A Tailorable LLM-based Architecture
Quick Take
This paper presents a novel LLM-based architecture for identifying and quantifying human values in text, featuring three modular components that enhance detection performance. Evaluated with the ValueEval dataset, the architecture demonstrates effective alignment with ethical considerations in decision-making systems, overcoming limitations of previous models.
Key Points
- Introduces a modular LLM-based architecture for human value detection in text.
- Architecture includes value specification generation, labeling, and support assessment modules.
- Evaluated using the ValueEval dataset, showing good detection performance.
- Scalable and reproducible process adaptable to various theoretical frameworks.
- Addresses ethical considerations in autonomous decision-making systems.
Article Content
From source RSS / original summaryarXiv:2605. 27373v1 Announce Type: new Abstract: As intelligent systems become more autonomous, the scientific community focuses on creating decision-making mechanisms that include ethical and moral considerations, unlike traditional utility-maximisation models. To achieve this, a key aspect is assessing how well these decisions align with human values.
To this end, a promising line of research is centred on developing approaches based on Large Language Models (LLMs) to identify human values from text, whether explicit or implicit, enabling their recognition throughout. This paper introduces a LLM-based architecture to detect and quantify the intensity of human values in text, avoiding the limitations of previous approaches tied to specific value theory or complex prompt engineering.
The architecture comprises three coordinated modules: one that generates structured value specifications from the foundational texts of any theoretical framework; one that labels texts using these specifications; and one that assigns graded support or resistance based on rhetorical and semantic evidence. This modular approach separates the tasks of conceptualising from detecting human values, creating a scalable and reproducible process driven by value specifications adaptable to various theories.
The architecture was instantiated with multiple LLMs and evaluated using the ValueEval dataset. The experiments demonstrate good detection performance, confirming the generality of the pipeline.
Reader Mode unavailable (could not extract clean content).
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →The Importance of Out-of-Band Metadata for Safe Autonomous Agents: The Redpanda Agentic Data Plane
The Redpanda Agentic Data Plane (ADP) introduces out-of-band metadata channels to enhance the safety of autonomous AI agents, ensuring secure data access and tamper-proof audit trails. This architecture mitigates risks associated with unpredictable AI behavior by enforcing governance throughout the agent lifecycle, demonstrated in a multi-agent trading system with strict data scoping and approval thresholds.