Measuring the Inference Energy Efficiency of Open-Weight LLMs across Benchmarks, Reasoning Modes, and Hardware
Quick Answer
This study evaluates the energy efficiency of 86 open-weight LLMs across seven benchmarks and six hardware platforms, revealing that many configurations consume 2.6–4.8 times more energy than necessary for similar performance.
Quick Take
Reasoning increases energy use by an average of 3.7 times, while FP8 can reduce energy consumption by about 25% without affecting scores. The findings suggest that energy consumption should be reported alongside benchmark scores for a complete efficiency picture.
Key Points
- Energy efficiency varies significantly among models with similar capabilities.
- Most configurations use 2.6–4.8 times more energy than necessary for equivalent performance.
- Reasoning increases energy consumption by an average of 3.7 times.
- FP8 technology reduces energy usage by approximately 25% with minimal score impact.
- Benchmark scores alone do not provide a complete picture of model efficiency.
DeepSignal Analysis
What happened
A study assessed the energy efficiency of 86 open-weight LLMs across seven benchmarks and six hardware platforms. It found that many configurations consume significantly more energy than necessary for comparable performance, with reasoning tasks increasing energy use substantially.
Key evidence
- The study evaluated 86 open-weight model checkpoints on seven benchmarks and six hardware platforms, revealing large differences in energy efficiency.
- Most configurations used a median of 2.6 to 4.8 times more energy than other configurations that achieved at least similar benchmark scores.
- Enabling reasoning increased energy consumption by an average of 3.7 times, highlighting the trade-off between capability and energy efficiency.
Why it matters
Understanding the energy efficiency of LLMs is crucial as AI models become more prevalent. The findings indicate that energy consumption is a significant factor that should be reported alongside performance metrics to provide a comprehensive view of efficiency. This could influence model selection and deployment strategies in the industry.
What to watch
Paper Resources
📖 Reader Mode
~2 min readAbstract:Benchmark scores tell us what an LLM can do, but not how much energy is required to deliver that capability. We directly measure this trade-off for 86 open-weight model checkpoints, evaluated on seven benchmarks and six hardware platforms ranging from a consumer GPU to an eight-GPU server. We find large differences in energy efficiency even among models with similar capability. Most configurations lie away from the score--energy Pareto frontier: they use a median of 2.6--4.8$\times$ more energy than another configuration that scores at least as well, with differences reaching 27$\times$. Reasoning can improve capability, but enabling thinking increases energy by about 3.7$\times$ on average, and the benefit depends strongly on the model family and output budget. Generated-token count is also an imperfect proxy for energy, misordering 18--24\% of model pairs. In contrast, FP8 reduces energy by roughly a quarter with little change in score, while changing hardware largely rescales energy without changing the relative ranking of models. These results show that capability alone gives an incomplete picture of efficiency. We argue that benchmark scores should be reported together with measured energy consumption under a specified serving regime and output budget.
| Subjects: | Artificial Intelligence (cs.AI); Performance (cs.PF) |
| Cite as: | arXiv:2610.14389 [cs.AI] |
| (or arXiv:2610.14389v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.14389 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zeyu Yang [view email]
[v1]
Fri, 9 Oct 2026 23:15:10 UTC (626 KB)
— Originally published at arxiv.org
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →VEPA: Verified Evolving Formal Policy for Lifelong Agent Guardrails
VEPA introduces a framework for verified evolving formal policies for LLM agents, enhancing safety by adapting guardrails based on deployment feedback. It outperforms existing methods across benchmarks like ASSEBench-Safety and ASSEBench-Security, ensuring deterministic policy enforcement even under challenging conditions.