Slack Introduces Agent Driven End-to-End Testing to Improve Resilience in UI Test Automation

Slack engineering has introduced an approach called agentic testing that explores how AI agents can be incorporated into end-to-end testing to improve resilience in dynamic software systems. to improve resilience in large distributed systems. The work targets a common challenge in continuous delivery environments, where end-to-end tests frequently fail due to user interface or service changes rather than actual functional regressions, leading to increased maintenance overhead for engineering teams.

Traditional end-to-end tests rely on fixed steps, stable selectors, and predictable flows across UI or APIs. In fast-changing systems, these assumptions often break, increasing maintenance effort. Slack engineers describe agentic testing as shifting part of this responsibility from static scripts to AI-driven agents that execute based on higher-level intent.

In this model, a test is expressed as an objective rather than a strict sequence of actions. An AI agent interprets the intent and attempts to complete the workflow by interacting with the application through UI or API surfaces. The agent evaluates the application state at each step and selects actions dynamically. When minor changes are encountered, such as modified UI structure or relocated elements, the agent attempts alternate paths to continue execution instead of failing immediately. The execution is then validated against expected assertions defined by engineers.

The workflow typically begins with a test intent being passed to the agent layer. The agent performs planning, executes actions against the system under test, observes results, and iterates until the objective is completed or a stopping condition is reached. The outcome is then recorded along with execution traces that capture the sequence of decisions and interactions taken during the run. Slack engineers blog that due to cost considerations, agent driven testing is currently better suited for targeted debugging and exploratory testing rather than frequent execution in continuous integration pipelines.

Traditional testing flow:

click → click → type → assert

Agentic testing flow:

goal → agent adapts → verify result

Slack engineers note that deterministic tests continue to serve as the primary mechanism for validating critical logic and contract correctness. Agentic testing is positioned within the end-to-end layer where workflows are more sensitive to UI and structural changes. The agent-based approach is used to reduce failures caused by superficial changes that do not reflect functional regressions.

The system also includes constraints to govern agent behavior during execution. These include limits on allowed actions, boundaries for exploration, and conditions under which execution should stop. Observability is a key requirement, and execution logs are structured to provide visibility into each step taken by the agent, enabling teams to replay and inspect failures.

Testing pyramid with four layers: Unit Tests, Integration Tests, E2E Testing, and Agentic Testing (Source: Slack Blog Post)

Slack engineering positions agentic testing as a complementary capability rather than a replacement for existing testing approaches. Deterministic end-to-end tests continue to support fast, repeatable regression validation in CI, while agent-driven execution is applied where UI changes introduce brittleness. Scripted or generated tests provide stable verification for predefined journeys, whereas agent-based execution operates from a goal-oriented model, observing application state and dynamically determining how to reach the desired outcome. This makes it useful for exploring complex UI behavior, debugging flaky workflows, and reproducing production issues, alongside traditional deterministic testing within the end-to-end layer.