Verdict: Encode Once, Decide Many
Quick Answer
This paper shows that Verdict-MQ, a 395M parameter model, outperforms the 421M Laya specialist on typed-decisions benchmark with better accuracy (0.801 vs 0.767) and lower KL divergence (0.080 vs 0.117).
Quick Take
It maintains performance across various tasks, though it shows instability on RACE and BoolQ. The model's structured attention mask allows for independent state encoding, enhancing efficiency in decision-making.
Key Points
- Verdict-MQ achieves significant improvement over Laya on typed-decisions benchmark.
- Model accuracy is 0.801 compared to Laya's 0.767, with lower KL divergence.
- Structured attention mask allows independent state encoding for efficiency.
- Performance varies on RACE and BoolQ, showing instability in training.
- Controlled comparisons indicate state sharing costs 1.4-2.5x latency for slight gains.
Paper Resources
📖 Reader Mode
~2 min readAbstract:Automating organizational work means answering many small typed questions about one piece of state (does this email need a reply, how urgent is it, who owns it) with probabilities good enough to act on without a human. The open typed-decision model we compare against, like a standard cross-encoder, answers each question with a separate pass over the input. We present Verdict-MQ, an open encoder that reads a case once and answers all of its questions: given identical retained state tokens after truncation (verified on every reported case), a structured attention mask makes the state encoding independent of the questions, keeps questions from influencing each other, and makes option order irrelevant by construction. On the typed-decisions benchmark, a 395M Verdict-MQ matches the soft gold labels significantly better than the released 421M Laya specialist on all five fresh seeds (KL 0.080 vs 0.117, Brier 0.045 vs 0.061), an advantage that survives per-system temperature scaling; its higher accuracy (0.801 vs 0.767) lies past the benchmark's own saturation point, so we read that margin as closer agreement with the benchmark's labelling teacher. On RACE, BoolQ and MultiRC, against a matched fine-tuned baseline it wins on MultiRC and loses on RACE and on BoolQ, where its training is unstable; we measure zero-shot references ourselves, including a closed general system (Jev) that beats every fine-tuned encoder. A controlled comparison measures what the shared state costs: giving each question its own question-aware copy of the state gains 0.9-3.5 points on RACE and MultiRC at 1.4-2.5x the per-case latency, and nothing measurable on typed decisions, with the invariance guarantees unchanged.
| Comments: | 19 pages, 2 figures, 14 tables. Code: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.20778 [cs.CL] |
| (or arXiv:2610.20778v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20778 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Avdhesh Singh Chouhan [view email]
[v1]
Wed, 14 Oct 2026 18:27:25 UTC (59 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →The "10th Juror": Open-Set Standpoint Screening for Bureaucratic Bias Detection
MARS-Gov introduces a framework for detecting bureaucratic bias in Dutch government documents, achieving a new state-of-the-art F1 score of 0.880. This model outperforms existing zero-shot detectors by 20.2 points and reduces unnecessary interventions to just 2.5%. The framework's dynamic '10th juror' adapts to emerging biases, enhancing legal language processing.