Routing rules now available on AI Gateway
Quick Answer
Vercel AI Gateway now offers routing rules, enabling teams to manage model usage at the gateway level.
Quick Take
Vercel AI Gateway now offers routing rules, enabling teams to manage model usage at the gateway level. This feature allows instant rerouting of requests when a model is down or retired, streamlining operations without code changes. Two types of rules include rewriting requests to a different model and denying access to specific models.
Key Points
- Routing rules are currently in beta and managed via the .Vercel CLI.
- Rewrite rules allow seamless traffic flow by redirecting requests to alternative models.
- Deny rules block access to specific models, returning a 403 error for unauthorized requests.
- These rules apply to all requests made with team credentials at the gateway level.
- Routing rules help teams standardize model usage and manage costs effectively.
Article Content
From source RSS / original summarynow supports. Vercel AI Gatewayrouting rulesRouting rules are firewall-style rules that control which models your team can use, applied at the gateway level instead of in your application code. When a model goes down or gets retired, you usually have to ship a code change to move off it. With routing rules, you push one rule and every request reroutes instantly. There are two types:Rules apply to every request made with your team's AI Gateway credentials. You manage them with the.
Vercel CLICreate a rewrite with a source and a destination model. The Gateway swaps in the destination transparently, so your application keeps requesting the source model:Create a deny rule to block a model. Requests for it return a:403Rules only change which model serves a request. Everything else you’ve configured still applies to the destination model, including:Routing rules are in beta. For more information, read the.
routing rules docsRead moreTypeWhat it doesUse it toRewriteServes a request for one model using anotherKeep traffic flowing when a model is unavailable, migrate off a retired model, standardize on one model, or route an expensive model to a cheaper oneDenyBlocks requests for a modelKeep your team off models you haven’t approvedRewriteDeny:,,, the, and. Request-levelBYOKmodel fallbackssortingfilterprovider optionsonly: and the. Team-levelZero Data Retentionprovider allowlist
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from Vercel AI
See more →
The Agent Stack
The Agent Stack by Vercel AI provides essential building blocks for creating production-grade agents, enabling seamless integration across multiple AI models and secure operations. It features components like AI Gateway for model routing, Workflow SDK for durable execution, and Vercel Connect for scoped access, streamlining agent development and deployment across various platforms.
