Walmart’s AI workflows meet the realities of the balance sheet

Walmart has reportedly begun limiting employees’ use of an internal AI assistant called Code Puppy after demands placed on the LLM backing the tool were higher than expected. Employees of Walmart were encouraged to use Code Puppy without any stricture or stipulations as to the limits of use, but Walmart is now assigning employees a fixed number of AI tokens, which limits how much it can be used. Code Puppy was publicised as being able to help with tasks like spreadsheet analysis, creating presentations, and other automatable workplace activities.

The change in internal policy is a cost control measure, as LLMs are increasingly transitioning to pay-per-use, rather than the fixed-price, subscription model that gave near-limitless access to AI inference. Walmart has roughly 2.1 million employees, so even modest per-employee queries and task requests can create significant costs.

Walmart’s guidance to employees is to use AI where it can create value, and comes with guidance on how workers should choose the right AI tool for any given task. Reporting also says employees have access to other AI platforms paid for by the company.

Walmart has expanded AI tool use in the company and provided training for its employees in how to use an AI, encouraging workers to experiment and adopt successful uses. Now the costs of each interaction are being billed directly, it’s among other large enterprises struggling to balance reported improvements in productivity with the cost of achieving the same.

At least part of the issue may stem from the methods used to measure productivity in workflows based on AI. Previously, tracking the number and complexity of uses of AI tools as measure of productivity has led to many employees ‘gamifying’ their KPIs – so-called ‘token maxxing’. As recently as April this year, a partner at Sequoia Capital told The Wall Street Journal, “We all should be tokenmaxxing”, an approach that resulted the emergence of AI leaderboards in companies to celebrate those making best use of AI software.

Such performative practices at companies will increasingly incur costs relative to the number and complexity of AI tasks, and the model chosen to perform them. Larger models that perform recursive actions (‘thinking models’) use more tokens to process inputs introspectively, leading to higher bills for users. Walmart’s encouragement of workers to choose their model carefully is an attempt to limit spending on expensive, frontier models to achieve relatively trivial tasks, such as spreadsheet analysis and creating presentations.

Multi-agentic AI work may also create unexpected costs for employers. When employees instigate iterative loops running on multiple agents in order to create a desired outcome, the real cost of sub-optimal results (and the necessary refining and re-submission of prompts) is now measurable in hard cash.

While not all AI providers have changed the entirety of their billing models from fixed subscriptions to per-token, both Anthropic and OpenAI have already moved their higher tier enterprise plans to the new footing. Microsoft’s decision to charge for its GitHub Copilot software development tools as of June 1st is in line with what is rapidly becoming the new financial normality for model providers. Uber recently revealed that it had used up its 2026 budget for AI spend in the first four months of the year; a testament to changes in charging policy affecting end-users.

By setting limits on token use on a per-employee basis, Walmart is striving to keep a lid on its ongoing costs, enforce more thoughtful use of AI tools, and enable it to establish the metrics of return on investment in AI.

(Image source: Pixabay, under licence.)

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and co-located with other leading technology events. Click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.