Scaling Agentic AI Factories Through Extreme Co-Design with NVIDIA BlueField

Agentic AI changes the infrastructure pattern for AI factories. One request can trigger many model calls, tool calls, memory lookups, policy checks, storage accesses, and network transfers before a final answer is produced. As more agents run at once and carry context across steps, users, tools, services, and sessions, infrastructure must move, protect, retrieve, and reuse data fast enough to keep GPUs and CPUs productive. 

The NVIDIA BlueField platform brings dedicated, programmable infrastructure processing in the AI factory data path. BlueField offloads infrastructure work from host CPUs, accelerates data movement, enforces policy inline, and enables context reuse. These capabilities help deliver production outcomes such as higher GPU utilization, more predictable latency, stronger isolation, lower cost per token, and more tokens per watt.

It combines purpose-built infrastructure and storage processors with NVIDIA DOCA software for AI factories. NVIDIA BlueField-4 DPUs offload, accelerate, and isolate networking, storage, security, telemetry, and control-plane services from host CPUs while accelerating data movement across GPU compute and CPU compute systems. NVIDIA Vera BlueField-4 STX storage processors power a new class of data platforms for context memory, high-performance storage infrastructure, and secure data services across AI factories.

Across these processors, NVIDIA DOCA provides the software foundation for building and operating these services across networking, storage, security, telemetry, and lifecycle management. In the NVIDIA Vera Rubin platform and the broader NVIDIA DSX architecture for AI factories, BlueField provides accelerated infrastructure while DOCA provides the programmable software model for deploying those services across the data center. 

This post explains how agentic AI and long-context inference drive new infrastructure demands, and how BlueField-4, Vera BlueField-4 STX, and DOCA address them by offloading, accelerating, and isolating infrastructure services across the AI factory data path.

Agentic AI makes infrastructure part of inference

Agentic AI extends inference beyond model execution into a distributed workflow spanning GPUs, CPUs, memory, networking, storage, and security. Each step depends on moving data, preserving context, enforcing policy, and coordinating services across the AI factory, making the infrastructure data path part of the inference pipeline. 

GPUs execute model inference and generate tokens, while CPUs orchestrate the agent runtime by executing tools, processing retrieval results, preparing prompts, validating outputs, and coordinating subsequent reasoning steps. The infrastructure must also preserve and retrieve the context that enables reasoning to continue across turns.

Preserving and retrieving context is especially important for KV cache. During prefill, an LLM creates KV cache data that stores intermediate attention state. As prompts, conversations, and agent workflows grow, cache state must increasingly persist across reasoning steps and be reused across requests. When GPU memory becomes constrained, systems evict and recompute KV cache, limit context length, or move state into another memory tier, introducing trade-offs in latency, throughput, or cost. This makes KV cache part of the infrastructure data path, where it must be moved, placed, protected, and retrieved without slowing inference.

As a result, infrastructure is no longer adjacent to inference. Infrastructure is now part of the inference pipeline. Networking, storage, security, telemetry, control-plane services, and context-memory management must process agent traffic without delaying GPU inference or consuming host CPU resources required for agent execution.

BlueField powers the operating system of the AI factory

Agentic inference depends on a fast, secure, and programmable infrastructure data path. BlueField provides the dedicated infrastructure processor for that path, connecting, securing, isolating, and accelerating services across the AI factory. 

BlueField-4 operates as the data processing unit (DPU) across Rubin GPUs and NVIDIA Vera CPUs, while the Vera BlueField-4 STX Storage Processor serves NVIDIA CMX for context memory and AI-native storage. Across these roles, BlueField combines high-speed networking, embedded infrastructure compute, local memory, PCIe connectivity, inline acceleration, isolation, and DOCA programmability into a coordinated AI factory data path.

The BlueField-4 DPU integrates up to 800 Gb/s Ethernet or InfiniBand connectivity, a 64-core NVIDIA Grace CPU, high-bandwidth LPDDR5X memory, PCIe Gen6, inline acceleration for networking, storage, security, and data movement, and the DOCA software platform. 

Compared with BlueField-3, it doubles networking bandwidth, delivers up to 6x more compute performance, 4x memory capacity, and more than 3x memory bandwidth. 

The BlueField-4 STX Storage Processor combines the NVIDIA Vera CPU, NVIDIA ConnectX-9 SuperNIC, up to 1.6 Tb/s of Spectrum-X Ethernet connectivity, high-performance NVMe storage access, accelerated data movement, in-silicon security, and DOCA programmability.

DOCA makes the BlueField infrastructure-processing domain programmable and provides the software foundation for building and deploying accelerated infrastructure services through libraries and microservices. It gives developers, operators, and ISVs a consistent way to create and operate BlueField and ConnectX-accelerated services as requirements shift across KV-cache reuse, tenant isolation, storage metadata, congestion control, secure provisioning, and new agentic runtime patterns.

System-level capabilities of BlueField

Networking improves interactivity only if embedded compute, memory bandwidth, PCIe, acceleration, and software can process traffic at the same pace. Additional compute improves throughput only if it has sufficient memory capacity, I/O bandwidth, and programmable services. BlueField was co-designed to balance these resources so infrastructure traffic can be processed where it arrives. 

High-speed connectivity brings AI workload, storage, security, and control traffic into the AI factory data path. Bluefield’s embedded compute and power-efficient LPDDR5X memory keep service logic and state close to the data being processed, including queues, policies, metadata, telemetry, and KV-cache placement. PCIe Gen6, VirtIO, and DOCA SNAP storage virtualization enable hosts to use standard host-visible network and storage device models backed by BlueField acceleration to reduce host CPU overhead. 

As agent requests move through the AI factory, they are carried by packets and trigger RDMA transfers, storage commands, policy checks, metadata lookups, and telemetry events. BlueField accelerates the associated flow steering, data movement, storage access, encryption, integrity, and policy enforcement so infrastructure services can keep pace with agent traffic without consuming host CPU resources.

DOCA turns this hardware foundation into programmable services for the AI factory, including:

• DOCA Host-Based Networking (HBN): Supports server-side Layer 3 routing, with BlueField acting as a BGP router for scalable multi-tenant designs.
• BlueField ASTRA: Enables Spectrum-X zero-trust, multi-tenant bare-metal deployments, with BlueField acting as the managed control point across multiple infrastructure planes with ConnectX-9.
• DOCA Memos: Helps manage and share KV cache across compute and storage nodes so context can be reused across long-context and agentic inference.
• DOCA security services: Support zero-trust access, policy enforcement, runtime visibility, and isolation to protect data, inference, and agents across the AI factory.

Together, these capabilities keep networking, storage, security, context management, and control services close to the data path instead of competing for host CPU resources. This helps BlueField improve GPU utilization, reduce inference latency, strengthen multi-tenant isolation, lower cost per token, and increase tokens per watt.

BlueField-4 across the Vera Rubin AI factory

Extreme co-design means the AI factory is engineered as an interdependent system. Each component has a defined role that complements the others, enabling the platform to sustain throughput, interactivity, isolation, and token efficiency under production workloads. 

The Vera Rubin platform is built for agentic AI workloads that require high-throughput inference, dense CPU execution, large-scale context memory, and secure data movement. Within this platform, Rubin GPUs provide accelerated compute, and Vera CPUs support tool calls, orchestration, and data movement. NVLink provides scale-up communication, while the ConnectX-9 and Spectrum-X support scale-out networking. BlueField operates across this entire system, spanning compute, networking, storage, and security (Figure 1).

GPU compute: Securing and accelerating Rubin GPUs

GPU performance depends on the full system’s ability to keep accelerators supplied with work, data, and secure connectivity. BlueField-4 operates as an infrastructure processor in the GPU compute tray. It supports frontend (north-south) networking, host CPU offload, secure data access, management, observability, and infrastructure isolation. By offloading and isolating these services, BlueField helps keep GPU compute from being gated by host CPU overhead, variable access controls, or management traffic.

The north-south path is critical for GPUs, because it brings user requests, retrieved data, context, storage traffic, and management services into the Rubin compute layer. BlueField-4 expands and secures that frontend path, while inline infrastructure processing handles the associated networking, storage, security, and telemetry work before it consumes host CPU cycles. Higher frontend bandwidth and lower host CPU contention help improve data and context availability to GPUs, supporting more predictable inference latency and higher tokens per second per user across the AI factory.

CPU execution: Isolating infrastructure from Vera agentic workloads

Agentic AI and reinforcement learning increase demand for CPU performance because agents execute tool calls, run code, browse or query data, transform inputs, parse outputs, evaluate results, and orchestrate workflows. The Vera CPU combines high per-core performance, concurrency, and power-efficient memory bandwidth to run this CPU execution layer at AI factory scale. BlueField-4 provides the infrastructure-processing layer around Vera CPU workloads, handling front-end networking, storage access, security and isolation, provisioning, policy, telemetry, and management.

This separation matters because agentic CPU workloads sit inside the reasoning loop. If infrastructure services consume host CPU cycles or add scheduling jitter, tool execution, retrieval, and validation can slow down even when GPU compute is available. BlueField processes networking, storage, security, and control-plane services in the DPU domain, enabling Vera CPUs to spend more time on agent execution and less time on infrastructure work.

AI-native storage and context memory

Agentic AI turns context into active infrastructure data. Multi-turn agents, long-context reasoning, retrieval-augmented workflows, and multi-agent systems generate reusable inference, including KV cache, that must be stored, shared, protected, and retrieved without stalling inference.

The NVIDIA CMX context memory storage platform creates a shareable, scalable, and power-efficient AI-native storage tier for inference context between GPU memory and scalable shared storage. It delivers an Ethernet-attached flash tier optimized for KV cache using the Vera BlueField-4 STX storage processor, which combines Vera CPU, ConnectX-9 networking, and DOCA Memos. 

The storage processor runs KV I/O, metadata management, data placement, security, and control operations close to the storage and network path. Instead of presenting flash as block storage, it tracks KV metadata, manages queues, supports cache recall and pre-staging, enforces tenant policy, and handles data protection. Placing these control-heavy, latency-sensitive operations in the storage processor helps preserve interactivity as context grows.

With DOCA Memos, CMX can manage and share KV cache across AI compute and CMX data nodes, making it an active context-memory data path rather than a passive storage tier. KV-cache reuse reduces repeated prefill, context recomputation, GPU idle time, and unnecessary data movement, improving tokens per second and power efficiency for long-context and agentic workloads.

In-silicon protection for data, inference, and agents 

Agentic AI changes the security model because agents repeatedly access data, models, tools, context memory, and inference services across AI factories. They may process proprietary or regulated data, model state, embeddings, and KV cache before producing a final answer. This makes data, inference, and agent behavior part of the security surface.

Security for AI factories can’t rely solely on host software. Host-resident security controls share resources and trust boundaries with the workloads they protect, which can expose them to tampering or evasion if the host is compromised. BlueField moves security processing in-silicon and outside the host workload domain, strengthening the control boundary while preserving CPU and GPU resources for AI work.

For multi-tenant AI factories, this enables inline protection without slowing the data path. BlueField provides a trusted infrastructure control point for tenant isolation, network policy enforcement, secure access, runtime detection, encryption, and telemetry across compute, storage, and inference infrastructure. DOCA extends this model through programmable services for zero-trust data access, inference protection, agent behavior visibility, and network-level isolation, helping protect models, datasets, context memory, and runtime interactions as agentic workloads scale.

Learn more

Agentic AI makes infrastructure part of the inference pipeline, requiring AI factories to move, protect, retrieve, and reuse data without slowing GPUs, CPUs, or context memory. BlueField provides accelerated infrastructure processing for networking, storage, security, telemetry, and context services, while DOCA makes those services programmable and deployable across the data center. Together, BlueField-4, Vera BlueField-4 STX, and DOCA help AI factories improve interactivity, isolation, GPU utilization, cost per token, and tokens per watt.

Get started

Download NVIDIA DOCA, install the latest release, and build the samples to start programming BlueField for accelerated networking, storage, security, telemetry, and lifecycle management.