From Black Box to Executable Logic: Explainable Reinforcement Learning through Prolog Expert Systems
Quick Answer
This study presents a method to convert deep reinforcement learning policies into executable Prolog logic programs, enhancing explainability.
Quick Take
The approach achieves optimal returns in a two-room key-and-door task and matches neural performance in continuous-control tasks, demonstrating significant improvements in interpretability and policy editing capabilities.
Key Points
- Transforms deep reinforcement learning policies into Prolog programs for better explainability.
- Achieves exact optimal return in a two-room key-and-door task with 16,944 states.
- Surpasses stochastic teacher performance in a budget-capped regime across ten seeds.
- Matches neural teacher performance in Acrobot and recovers 97% return on CartPole.
- Demonstrates exponential cost in observation dimension for oblique decision boundaries.
DeepSignal Analysis
What happened
This study introduces a method for transforming deep reinforcement learning policies into executable Prolog logic programs, enhancing their explainability. The approach demonstrates optimal performance in a specific task and matches neural network performance in continuous-control scenarios, indicating improvements in interpretability and policy modification.
Key evidence
- The method involves a three-stage transformation that converts a proximal policy optimization teacher into a Prolog program, allowing for human readability and machine execution.
- In a two-room key-and-door task with 16,944 states, the Prolog program achieved optimal returns across all trials, outperforming the stochastic teacher in a budget-capped scenario.
- For continuous-control tasks, the Prolog program matched the neural teacher's performance within noise on the Acrobot task and recovered approximately 97% of the return on the CartPole task.
Why it matters
The ability to convert deep reinforcement learning policies into interpretable logic programs could significantly enhance the transparency of AI systems. This transformation allows stakeholders to understand decision-making processes better and facilitates policy adjustments, which is crucial for applications in sensitive areas like healthcare and finance where explainability is paramount.
Paper Resources
Source Excerpt
A trained deep reinforcement learning policy is a black box, and we ask whether it can be made explainable by rewriting it as an executable logic program that reproduces its behaviour and that a person can read, a logic engine can run, and an optimizer can edit. We present a three-stage post-hoc transformation that extracts a frozen proximal policy optimization teacher, induces an ordered rule list from its decisions in the manner of classical relational learning, and emits the result as a Prolo
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Automatic Ordinary Differential Equations Discovery For Biological Systems Using Powered Agentic System
The MEDA system utilizes large language models and symbolic regression to autonomously discover ordinary differential equations for biological systems, achieving strong structural recovery and biologically plausible models. It outperforms existing methods by integrating domain knowledge and mechanistic constraints, demonstrating effective retrieval and extrapolation capabilities.