TRACES: Proactive Safety Auditing for Multi-Turn LLM Agents via Trajectory-State Modeling
Quick Answer
TRACES is a proactive safety auditing model for multi-turn LLM agents, learning trajectory risk states from hidden representations.
Quick Take
TRACES is a proactive safety auditing model for multi-turn LLM agents, learning trajectory risk states from hidden representations. It improves safety predictions and proactive risk discrimination across benchmarks, suggesting potential for training safer agents.
Key Points
- TRACES learns prefix-level trajectory risk states from an observer LLM's hidden representations.
- It uses weak trajectory-level supervision to produce dense prefix-level risk estimates.
- The model improves full-trajectory safety prediction across multiple agent safety benchmarks.
- Proactive auditing can help train safer agents for long-horizon tasks.
Paper Resources
Article Excerpt
From source RSS / original summaryarXiv:2605. 27690v1 Announce Type: new Abstract: LLM agents increasingly operate through multi-turn and environment interaction, where safety risks often emerge from intermediate steps long before they surface in the final outcome. Reactive auditing is therefore insufficient: post-hoc diagnosis frequently misses the chance to flag risks while they are unfolding.
We propose TRACES, a representation-based proactive auditor that learns prefix-level trajectory risk states from the hidden representations of an observer LLM. TRACES induces latent mechanism features from step representations and models their temporal evolution to estimate whether a partial trajectory is drifting toward unsafe behavior. To sidestep the cost and ambiguity of step-level risk annotation, TRACES is trained with weak trajectory-level supervision while still producing dense prefix-level risk estimates.
Across multiple agent safety benchmarks, TRACES improves both full-trajectory safety prediction and proactive risk discrimination. Our analyses further suggest that these risk states can help train a safer agent, highlighting the broader potential of proactive auditing for long-horizon agent safety.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →Letting the Data Speak: Extracting Keywords from Crowdsourced Collections with AI
The study evaluates three NLP approaches—Named Entity Recognition, Keyword Extraction, and Topic Modelling—using the Their Finest Hour Online Archive to automate keyword extraction from crowdsourced WWII collections. Findings suggest that while NLP methods show promise, no single approach is sufficient, and ethical considerations in automated keyword extraction are crucial for responsible stewardship.