Kimi K3 vs GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing
Quick Answer
Kimi K3 outperforms GPT-5.6 Sol in pass@4 (89.4% vs 85.8%) and cost per rollout ($4.65 vs $8.37), while Sol leads in pass@1 (72.7% vs 68.5%).
Quick Take
Both models excel in different programming languages, making them complementary for task routing.
Key Points
- Kimi K3 has a lower cost per rollout at $4.65 compared to Sol's $8.37.
- In pass@4, Kimi K3 achieves 89.4%, outperforming Sol's 85.8%.
- Sol excels in pass@1 with 72.7%, while Kimi K3 is at 68.5%.
- Kimi K3 solves 14.7 tasks per $100, significantly more than Sol's 5.3.
- Both models cover 95.6% of tasks when routed together.
Source Excerpt
We ran 904 DeepSWE rollouts on Kimi K3 and GPT-5. 6 Sol. Sol leads pass@1; Kimi K3 wins pass@4 at 2. 8x the solves per dollar, and routing between them reaches ~85. 6%.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from Together AI
See more →
Open, convenient and predictable: Introducing Provisioned Throughput
Together AI introduces Provisioned Throughput, offering guaranteed inference capacity for MiniMax M3 and GLM-5.2 at $0.05 per PTU per minute, achieving costs up to 90% lower than Claude Opus 4.8. This new model provides predictable pricing and a 99% uptime SLA, catering to companies transitioning to open weight models for production workloads.
