RuleChef: Grounding LLM Task Knowledge in Human-Editable Rules
Quick Answer
RuleChef is an open-source framework that leverages large language models to generate and iteratively improve executable rules for NLP tasks like text classification and Named Entity Recognition.
Quick Take
By synthesizing rules from task descriptions and labeled examples, it creates a fast and inspectable rule system, enhancing performance through human feedback and additional examples.
Key Points
- RuleChef generates executable rules for NLP tasks using .
- The framework improves rules based on human feedback and additional examples.
- It can bootstrap rules from observed input-output pairs of existing models.
- Preliminary evaluations show effectiveness in classification and NER tasks.
- RuleChef is available as open-source software under Apache 2.0.
Paper Resources
📖 Reader Mode
~2 min readAbstract:We present RuleChef, a framework that uses large language models (LLMs) to generate executable rules for NLP tasks such as text classification, Named Entity Recognition (NER), or relation extraction. Rules are generated based on a task description and a set of labeled examples, then they are iteratively improved based both on additional examples and on human feedback overexisting rules. RuleChef can also be used to bootstrap rules using the observed input-output pairs from any existing model for a given task. LLMs are used only at learning time, synthesizing rules and iteratively patching them based on failures measured on a held-out split. The result of this process is a fast, deterministic, and inspectable rule system. Preliminary evaluation is performed on both classification and NER tasks. We release RuleChef as open-source software under an Apache 2.0
| Comments: | 8 pages |
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.01293 [cs.CL] |
| (or arXiv:2607.01293v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.01293 arXiv-issued DOI via DataCite |
Submission history
From: Ádám Kovács [view email]
[v1]
Wed, 1 Jul 2026 13:09:49 UTC (46 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.