Small Experiments, Cheaper Decisions: A Case Study in Staged Promotion for Micro-Pretraining
Quick Answer
This study presents a staged promotion protocol for micro-pretraining on Windows A100 and Linux L40S, demonstrating that short pretraining runs can lead to over-promotion of configurations.
Quick Take
The protocol, involving budgets from 2 minutes to 12 hours, shows that the top-ranked condition after 12 hours does not align with earlier 10-minute rankings, highlighting the instability of early screens and the importance of operational promotion evidence.
Key Points
- Staged budgets ranged from 2 minutes to 12 hours across two host types.
- The 12-hour top-ranked condition did not match the 10-minute mean-best condition.
- The protocol executed 144 GPU-hours for the 12-hour branch, totaling 169.2 GPU-hours overall.
- Continuing all 60-minute candidates would require 192 GPU-hours, while 10-minute candidates would need 432 GPU-hours.
- Findings indicate bounded cost allocation, not claims of global optimality.
Paper Resources
Source Excerpt
arXiv:2606. 11387v1 Announce Type: new Abstract: Short pretraining runs can reduce experimental cost, but they can also over-promote configurations that only look strong at tiny budgets. We study an auditable staged-promotion protocol for a fixed micro-pretraining runner on two heterogeneous host blocks: Windows A100 and Linux L40S.
Starting from twelve prior-screened configurations, we use staged budgets of 2 minutes, 5 minutes, 10 minutes, 60 minutes, and 12 hours, with frozen promotion rules before expensive continuations. …
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.CL
See more →TriAgent: Divergence-Aware Committees for Cost-Efficient Financial Sentiment Analysis
TriAgent introduces a cost-efficient multi-agent system for financial sentiment analysis, combining VADER, FinBERT, and Qwen2.5. It achieves an F1 score of ~0.87 with significant savings of $9.3M/year at a 10M-user scale compared to GPT-4o-mini, while also detecting hallucinations with an AUC of 0.90.
