Not all uncertainty is alike: volatility, stochasticity, and exploration
Quick Take
The study differentiates between volatility and stochasticity in exploration strategies for decision-making.
Key Points
- Volatility promotes exploration; stochasticity suppresses it.
- Introduces Cause-Aware Uncertainty-Sensitive Exploration (CAUSE).
- CAUSE outperforms traditional strategies in noisy environments.
📖 Reader Mode
~2 min readAbstract:Adaptive decision-making in biological and artificial intelligence requires balancing the exploitation of known outcomes with the exploration of uncertain alternatives. Although prior work suggests that uncertainty generally promotes exploration, it has typically treated distinct sources of environmental uncertainty as equivalent. We consider environments with latent reward states that drift over time (volatility) and are observed through noisy outcomes (stochasticity). Both increase posterior uncertainty, yet we show they drive optimal exploration in opposite directions: volatility enhances it, stochasticity suppresses it. We establish this asymmetry formally by extending the Gittins index framework to Gaussian state-space bandits with latent dynamics. We further derive Cause-Aware Uncertainty-Sensitive Exploration (CAUSE), a closed-form exploration bonus obtained via control-as-inference that inherits the same monotonicities. CAUSE outperforms standard exploration strategies in environments with heterogeneous noise structure, and also improves on a Gittins-per-arm policy whose rested-bandit optimality does not transfer to restless settings. Learning and exploration are governed by the same noise-inference asymmetry, and the framework predicts that pathological noise inference produces \emph{reversed} rather than merely impaired exploration, with implications for computational accounts of psychiatric conditions.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.19215 [cs.AI] |
| (or arXiv:2605.19215v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.19215 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Payam Piray [view email]
[v1]
Tue, 19 May 2026 00:38:00 UTC (633 KB)
— Originally published at arxiv.org
More from arXiv cs.AI
See more →From Prompts to Protocols: An AI Agent for Laboratory Automation
An AI agent integrates large language models for automating laboratory protocols, enhancing efficiency and accuracy.