QCon AI Boston: Production AI Moves Beyond Prompts to Platforms, Harnesses, and Evals

QCon AI Boston 2026 marked a turning point. We have spent the last couple of years learning to build AI agents. Now the question is how to run them, safely and reliably, once they are live. Almost every talk came back to the same theme: agents are forcing teams to build real production infrastructure around them.

OpenAI’s Martin Spier set the tone in the opening keynote. His talk was about performance, but not in the narrow "make inference faster" sense. There is a quiet stretch before inference where the product has to make the conversation usable for the model: enough context to help, enough trimming to keep it fast. In other words, there is still plenty of work to make the product fast, even when the model is fast.

"The basics became more important."
Martin Spier’s "Keeping ChatGPT Fast as AI Development Accelerates"

It turned out to be a good lens for the rest of the conference. The work around agents is getting less shiny. It is becoming the boring infrastructure work that decides whether a system survives contact with real users. The first recurring trend was context and agent infrastructure rising into a platform layer of its own. Teams are moving beyond single-purpose applications and toward shared systems for context, tool access, identity, and state. This is where ideas like context engineering, MCP gateways, and semantic tool catalogs start to look like core infrastructure. And as core building blocks, they need owners and contracts.

"Precision + Security + Cost"
Fabiane Nardon’s "Architecting the Data Layer for AI Agents: From Transactional Systems to MCP and Semantic Models"

"Context engineering isn’t a feature, it’s architecture. Get this right and everything else gets easier"
Ricardo Ferreira’s "Beyond Prompting: Context Engineering for Production-Grade AI"

"Own the state. Order the mutation. Prove the action"
Vinoth Govindarajan’s "The Agent Harness: Control Planes, Invariants, and Approval Boundaries for Production AI Agents"

The second trend was trust - a shift from prompt-level guardrails toward trustworthy execution, a harness. As agents gain access to tools and files, security can no longer depend on instructions in a prompt. The harness is the system that sits around the model. A tool can run while the user sees nothing, so production systems need clear ownership of state, ordered writes, approval boundaries, and a real audit trail. The problem is no longer whether an agent gives a good answer, but whether the system can prove what action was taken, by which component, and under which constraints and privileges.

"The most effective orgs do two things:

1. Thoroughly improve AI usage across SDLC
2. Resolve the bottlenecks that limit outcomes"

Lizzie Matusov’s "The Five Stages of AI Maturity in Engineering Organizations — Where and Why Teams Get Stuck"

"Write strategy early. Build around customers. Own company-fit surfaces"
Siddharth Kodwani and Swaroop Chitlur’s "Building GenAI Platform at DoorDash"

The third trend was AI adoption itself becoming an engineering operating model. Once usage spreads, the boring questions arrive quickly: who pays for this, who can call which tools, where do failures show up, and how do teams learn from them? Exposing a model through an API or handing engineers a chatbot is not enough. Teams need paved paths, shared policy surfaces, evaluation loops, observability, cost attribution, and feedback mechanisms that make the right behavior easier than the quick, risky one.

A standout topic is how engineering organizations should think about evaluation. A one-shot test can catch obvious failures, but agents do not always fail on the first turn. Single-turn tests and static benchmarks are a weak fit for systems that use tools, maintain state, carry context, and behave differently across turns. So the testing has to get closer to the shape of the product: conversations, traces, simulations, production feedback. Without that, the tests may report success while users hit failures the benchmark never exercised.

Taken together, QCon AI Boston 2026 suggested that production AI is becoming less about prompt engineering and more about a systems problem. The hard problems are shifting toward context, data contracts, LLM and MCP gateways, state, evaluation, latency, cost, observability, and, ultimately, security and trust. The harness around the model now matters as much as the model inside it. Agents may talk like coworkers, but they fail like software, and operating them well depends on old lessons from platform engineering and distributed systems.