How Datadog Used Claude and Cursor for Test-Driven Production Migration

In a recent article, Datadog engineer Arnold Wakim shared what worked, what didn't, and the lessons they learned while evolving a critical production system using AI to overcome hard limits in its storage backend and significantly improve performance.

The original version of Stream Router, an API for managing routing in the metrics pipeline, was built as an eventually consistent system using a key-value (KV) model. As the routing table expanded, however, the KV database began hitting transaction size limits, and the most demanding operations slowed dramatically, taking up to 45 minutes due to thousands of sequential round trips.

In the KV model, the code had to reconstruct these relationships. It would pull tens of thousands of entries into pod processes and effectively act as a relational database in-process—handling logic that foreign keys would normally enforce in a relational system.

To address this issue, Datadog engineers carefully redesigned the schema to reflect relationships between domain entities allowing foreign keys to replace logic that previously had to be reconstructed in application code. Once the new schema was defined, they needed to refactor the entire codebase to migrate from the old model to the new one, a task for which they turned to AI:

We used Claude and Cursor to accelerate a systematic, test-driven refactoring process. They weren’t generating code autonomously: For each method, we provided the old implementation, the new schema, and a failing test. The models would generate a first pass, and the tests told us whether it was correct.

According to Wakim, there were three key elements that made the migration possible: strong code modularity, which allowed the new Stream Router to implement the same API on PostgreSQL instead of FoundationDB without requiring changes elsewhere; a comprehensive test suite, that served as a clear pass/fail criterion for every AI-generated change; and a parallel infrastructure in which two independent instances of Stream Router ran side by side, handling the same requests while clients were routed between them via feature flags:

A dedicated validator service runs in every cluster, periodically comparing routing responses between the two and alerting the team immediately if anything diverges.

The migrations itself took place across three phases: first, using Claude to describe the intent of each key function; second, crafting focused prompts to fix failing tests by supplying the expected behavior along with the context from the previous step; and finally, deploying to production using a "blue/green" approach, where the new PostgreSQL-backed and the existing FoundationDB-backed Stream Routers ran in parallel and were continuously compared.

Wakim also notes a few areas where the approach fell short, including the limited effectiveness of higher-level prompts and the tendency of Claude to generate correct but suboptimal queries:

Niche optimizations such as batching, UNNEST tricks, and common table expressions required human input. The AI-generated queries returned the right results but issued far more round trips than necessary. We wrote the optimized versions ourselves, and once the models had seen the pattern, they could replicate it in subsequent methods. But they did not discover these patterns independently, even though they exist in the literature.

Another issue was high token consumption, driven by feeding the AI full test output dumps rather than trimmed excerpts, as well as the iterative loop of test output, code context, and schema information.

After the migration operations dropped from 45 minutes to about a second, and latencies fell from hundreds of milliseconds to just a few. Data storage became up to 40x smaller, while PostgreSQL and DuckDB simplified relationship handling and improved query efficiency. CPU and memory usage decreased and database costs were reduced by 90%.

In summary, Wakim concludes that the key success factor in Datadog's migration using Claude was the strength of their test suite, which ultimately determined how much they could trust AI-generated code.