Parameter Golf: What Really Works?
Quick Answer
This paper shows that The Parameter Golf challenge tested language model optimization under a strict 16 MB artifact budget, achieving a 13.6% reduction in bits-per-byte (BPB) from 1.2244 to 1.058 across 2,037 submissions.
Quick Take
Despite individual techniques showing minimal improvements, a taxonomy of 84 methods was developed to isolate effective strategies.
Key Points
- Participants trained models within 10 minutes on 8xH100 SXM GPUs.
- Quality measured in bits-per-byte (BPB) for unseen text encoding.
- A total of 2,037 pull requests and 1,430 clean submissions were analyzed.
- Most optimization techniques showed diminishing returns in competitive settings.
- The leaderboard score demonstrated significant improvement through collaborative efforts.
Paper Resources
📖 Reader Mode
~2 min readAbstract:How far can a language model improve under a strict artifact budget? Parameter Golf posed this question as an open community challenge in which participants trained the best language model, with the complete artifact (training code + compressed weights) required to fit within 16 MB and be trained in under ten minutes on 8xH100 SXM GPUs. Quality was measured in bits-per-byte (BPB), the average number of bits required to encode each byte of unseen text. We analyze 2,037 pull requests and 1,430 clean scored submissions from the contest, build a taxonomy of 84 optimization techniques, and measure each technique's contribution to BPB. The verified leaderboard score dropped from 1.2244 to 1.058 BPB across three phases -- a 13.6% reduction, despite individual techniques rarely improving BPB by more than 1%. We show that most gains in techniques shrink across competitive submissions, isolating the few methods that improve performance across stacks.
| Subjects: | Computation and Language (cs.CL) |
| Cite as: | arXiv:2607.01517 [cs.CL] |
| (or arXiv:2607.01517v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.01517 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Prashanna Mani Paudel [view email]
[v1]
Wed, 1 Jul 2026 22:29:40 UTC (1,044 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.