NVIDIA Vera CPU Boosts AI Factory Throughput to Accelerate Agentic Workloads

Agentic systems turn model reasoning into action through multi-step workflows that combine inference, tool use, code execution, retrieval, orchestration, and result handling. As these systems scale across the AI factory, performance depends not only on GPU acceleration, but also on the CPU work that happens between model steps.

Across the creation and deployment of an agentic system, the CPU ensures GPU resources and the entire AI factory deliver optimal performance. The CPU is on the critical path for reasoning, response time, and learning because it executes the work between model steps: sandboxed evaluations, tool calls, code execution, data processing, KV-cache coordination, and result handling. 

For agentic AI, one of the most important CPU metrics is sustained per-core performance under full socket load. The socket may be packed with concurrent sandboxes, tools, simulations, orchestration tasks, and data services, but each agent or RL workflow still depends on sequential steps that must be finished before the next step can begin. More cores increase parallelism across workflows; faster cores determine how quickly each workflow advances, generating tokens, updating models, or serving the next turn.

 If CPU-side execution slows, the GPU fleet suffers in three key ways:

1. RL receives fewer useful evaluations per cycle, leading to an inflated time-to-train.
2. The fleet takes longer to serve individual users.
3. KV-cache is evicted, and the fleet loses the compute savings of the cached context.

The following sections explore how NVIDIA Vera tackles these three challenges and how the NVIDIA Vera CPU design maximizes fleet performance.

Vera CPUs yield smarter and more efficient agents

RL alters the compute paradigm by introducing a continuous, interactive feedback loop between the GPU and CPU. Unlike AI training where data is simply streamed to accelerators, RL demands active generation of experience data. This data generation phase, often referred to as environment rollouts or simulations, heavily relies on sequential, logic-based processing. 

When the processor stalls during these highly parallelized rollout phases, the entire training pipeline suffers. A slower processor means fewer environment steps are completed per second, degrading the quality of the model’s policy updates, known as gradients.

These gradients act as the mathematical instructions sent to the neural network to tell it exactly how to improve its policy. In RL, they rely entirely on ‌training signals, such as feedback and rewards generated by these CPU environments. Because hard problems are sequentially deep, CPU per-core performance affects how many environments can complete evaluations within the training window. For example, a baseline CPU might only complete 45% of its evaluations, leaving the GPU waiting for actionable feedback.

As a max single-threaded CPU at scale, the Vera CPU solves this bottleneck by maximizing sustained per-core performance under full-socket load. With cores that are 1.8x faster, Vera CPU can complete up to 85% of those evaluations in the same window. Delivering more RL feedback, Vera CPU provides a richer, higher-quality training signal. This enables the GPU to compute more accurate gradients, accelerating time to convergence and yielding smarter models.

This improvement is achieved with the NVIDIA Olympus core, designed for the sequential, branch-heavy CPU work that dominates RL and agent execution. Table 1, outlines the Vera CPU features and influence on the RL workload.

Vera CPU features	RL impact
Neural branch predictor	Keeps pipelines moving through complex control flow in Python, simulators, reward logic, and tools.
10-wide decode front end	Feeds more instructions into execution each cycle, improving CPU throughput.
Deep out-of-order execution	Makes progress around long-latency operations, reducing stalls in irregular RL workloads.
NVIDIA Spatial Multithreading	Sustains high per-core performance even when scaling to thousands of parallel environments.

Vera CPUs help serve more users per GPU hour

Moving from offline training to live, interactive agentic deployments shifts the primary optimization metric from pure throughput to strict, predictable latency. In these environments, an AI agent operates autonomously to solve complex user queries by executing a chain of actions. 

Each step in this sequence requires launching a tool call, running code, compiling a test, or querying data before the model can continue.

To maintain this responsiveness at scale, low average latency isn’t sufficient. Agents require predictable latency when the socket is loaded with many concurrent sandboxes, services, and data processing tasks. 

The Vera CPU is designed to reduce that loaded latency penalty, as shown in Figure 2. Vera CPU avoids the performance cliffs common in multi-chiplet CPU designs, where workloads can spill across chiplets or sub-NUMA domains, by using a monolithic compute die for its 88 Olympus cores. This avoids complex cross-chiplet core-to-core hops.

Coupled with a large unified cache and NVIDIA Scalable Coherency Fabric (SCF), data moves more uniformly across the system. This deliberate, agentic-focused implementation enables Vera CPU to achieve 40% lower peak loaded latency compared to x86 CPU. This predictable fabric helps reduce tail latency, ensuring that tool responses stay strictly on schedule. 

Beyond latency, agents need massive memory bandwidth. Thousands of concurrent sandboxes, tool calls, retrieval operations, database queries, and data-processing tasks must move large, irregular datasets while keeping each agent’s state, context, and outputs close to the cores. The Vera CPU uses power-efficient LPDDR5x delivering up to 1.2 TB/s total memory bandwidth and up to 14 GB/s per core. This is more than 3x the per-core memory bandwidth at less than half the power of traditional data center CPUs—helping agents sustain throughput when every core is active. 

The result is a highly responsive agentic AI pipeline capable of maintaining strict service level agreements (SLAs) without hardware bottlenecks impeding the user experience.

Vera CPUs maximize GPU compute per session

In agentic inference, the GPU doesn’t work in one uninterrupted pass. A single session may include many model steps separated by CPU-side tool calls, code execution, retrieval, database queries, or sandboxed evaluation. In a fully subscribed datacenter, the GPU does not wait through those gaps. While the CPU is working, it takes on new requests and new active contexts. As the requests accumulate, the original trace’s KV cache becomes more challenging to preserve in place.

A slower CPU increases the gap between GPU compute steps. This makes it more likely that the system will need to evict the original request’s KV cache from the GPU memory to make room for other users whose work is ready to run. When the tool call completes, the GPU may need to rebuild that prior context by reprocessing the earlier sequence as a large input prompt rather than resuming from a cached state. 

The Vera CPUs help compress the CPU-side portion of the trace. Faster tool execution, stronger per-core performance under load, high memory bandwidth, and predictable latency reduce how long agents wait between GPU steps. This shortens the window in which the KV cache is exposed to eviction pressure, keeps more active context residents in HBM, and reduces the need to reconstruct prior states through expensive recompute. 

The result is that more of the GPU’s time goes to useful token generation instead of rebuilding context, making CPU performance a direct contributor to GPU efficiency in saturated agentic systems.

Accelerating the future of AI factories

Vera CPU brings together the CPU capabilities that matter most for agentic AI factories: sustained per-core performance, predictable low latency under load, massive memory bandwidth, and efficient data movement at scale. As agentic AI scales, the CPU is no longer background infrastructure. It is a direct driver of AI factory throughput, responsiveness, and GPU efficiency. By reducing stalls, limiting recomputation, and helping the entire system stay productive under load, NVIDIA Vera CPU is built to maximize performance across next-generation agentic AI factories.

To see how these capabilities translate to real-world performance, learn more about the Vera CPU,  NVIDIA Vera Rubin NVL72, and the Vera CPU benchmarking by Phoronix.

Relative performance based on measured data, and subject to change. NVIDIA Vera CPU with LPDDR5X performance baselined to the latest x86 CPU.