Open, convenient and predictable: Introducing Provisioned Throughput

Inference spend has become a line item the board asks about.

In a not too distant future every knowledge worker will have several agents working on their behalf, but companies are struggling to achieve this without astronomic inference budgets. To succeed companies need to consume a range of models that span the Pareto frontier, matching task difficulty with model quality (and cost). Open weight models occupy an ever-greater fraction of that frontier.

As companies move to a multi-model, multi-harness setup, open models are powering more and more of their everyday agents for: coding, finance, marketing, and workflow automation.

Companies that have made the switch report 6-20x lower inference costs on open models compared to proprietary alternatives. We see the shift in our own numbers: In nine months, token volume through Together AI's APIs has grown from 30 billion to more than 400 trillion tokens a month. A substantial share of that traffic is workloads that used to run on closed APIs.

What's been missing: Token capacity you can trust

Abundant AI requires more than open models on the Pareto frontier. Abundant AI also requires peace of mind: committed capacity, predictable (and influenceable) pricing, and an SLA customers can depend on. Until now, the open-model market offered two options, and neither matched that shape. Serverless inference is largely best-effort: fine for development, but unpredictable for production workloads. Dedicated inference (i.e. dedicated GPUs) gives you guarantees and full control of the serving environment, but many teams don't want to manage the complexity of a fully configurable inference platform. They want a dependable model provider behind a simple API, not GPU-hour math, capacity planning, and a serving stack to configure.

What teams actually want is simple: The same token-based capacity agreement they already have with their closed-model provider, pointed at frontier open models. That's the product we built.

Introducing Provisioned Throughput

Whereas Together AI’s serverless inference is “pay for what you use” with a best effort SLA, Provisioned Throughput is “pay for the guaranteed capacity to use” with an uptime SLA.

You buy Provisioned Throughput Units (PTUs). Each PTU is a fixed slice of capacity: a guaranteed rate of tokens per minute for the model you're running, held exclusively for you, priced at $0.05 per PTU per minute. Provisioned Throughput accounts for variations in your traffic; input, cached input, and output tokens burn down PTUs differently allowing you to optimize spend for your traffic patterns. There's no GPU-hour math and no infrastructure to manage. The inference API for Provisioned Throughput, Dedicated Inference and Serverless Inference is identical.

Provisioned Throughput is available today for MiniMax M3 and GLM-5.2, with more models coming. We have capacity available in North America, EMEA, and beyond, with a one-month minimum term and discounts at higher levels of commitment.

"Our models are built for demanding production workloads and Together AI's Provisioned Throughput gives teams a way to run them with the reliability those workloads require. As teams shift volume off closed APIs, Together AI's infrastructure lets them adopt M3 at scale with commitments they can build a business on. That's exactly the kind of access we want for our models."

— Linda Sheng, President of Global Business, MiniMax

The economics

A PTU covers three types of traffic: input tokens, cached input tokens, and output tokens, each drawing down your capacity at a different rate. On MiniMax M3, one PTU delivers 138,840 input tokens per minute, 694,200 cached input tokens per minute, 23,140 output tokens per minute, or any mix of the three.

Real workloads have different shapes. Output-heavy traffic consumes capacity faster than cache-efficient traffic, but traffic shape never changes the SLA. It only changes how fast you consume your capacity.

At full utilization, a PTU on MiniMax M3 works out to roughly $0.36 per million input tokens and $2.16 per million output tokens, against $5 and $25 list on Claude Opus 4.8. Across three representative production profiles:

These are estimates. Actual PTU usage will vary with your traffic shape. Run your own numbers in the pricing calculator

Where it fits

Serverless is the fastest way to start building on open models. Provisioned Throughput is for production workloads on standard open models that need guarantees. If you need a fine-tuned or custom model with deep control over the serving environment, Dedicated Inference is the right fit.

If you're running production traffic on a proprietary API that a frontier open model can handle, the migration path is now complete: frontier-quality models, token-based pricing you can plan around, and reliability guarantees your customers can depend on, at up to 90% lower cost.

→ Estimate your PTUs with the pricing calculator

→ Talk to our team

→ Read the docs