AI AGENTS
Agents act instead of answering. Agents act instead of answering. What changed, when, and what it actually means.
Update — August 2026: the ruler moved, not the models
Everything below this section was measured before June 2026 and is now incomplete. What happened in between is the clearest illustration of this site's whole argument, so it is worth stating plainly rather than quietly editing the numbers.
Read those first two figures together. In roughly eight weeks, the story went from computer use is approaching superhuman to the best agent in the world finishes one task in five. Nothing about the models changed in that window. The instrument changed. The original benchmark saturated, so a harder one was built, and performance appeared to collapse.
A benchmark score is a measurement of a model against a particular ruler at a particular moment. When the ruler is replaced — because the old one saturated — the score moves without the underlying capability moving at all.
This is why every figure on this site carries the instrument that produced it and the date it was published. A percentage without its benchmark and its date is not a fact. It is a headline.
What is actually deployed
Benchmark progress and production reality have separated further, not converged:
- Autonomous agent deployment across business functions remains in single digits, per the same AI Index that reported the 66% figure.Stanford HAI, 2026 AI Index Report
- McKinsey finds 88% of organisations use AI in at least one function, but only 23% are scaling an agentic system.McKinsey, reported 2026
- Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027 — citing escalating costs, unclear value, and inadequate risk controls.Gartner, 2026
The security picture got worse, and it is specific
The exposure figures further down this page described agents left open to the internet. A 2026 study of OpenClaw — described by its authors as the most widely deployed personal AI agent of early 2026 — measured something more pointed:
Researchers evaluated a live instance across four backbone models and twelve attack scenarios. Poisoning any single dimension of the agent's persistent state — its capabilities, its identity, or its knowledge — raised average attack success from 24.6% to between 64% and 74%. Even the most robust model tested showed more than a threefold increase over its baseline.Wang et al., "Your Agent, Their Asset: A Real-World Safety Analysis of OpenClaw", arXiv 2604.04759
The attack surface is not the prompt. It is everything the agent remembers. An agent that persists state across sessions can be edited between them.
Separately, a 2026 benchmark report found automated traffic growing eight times faster than human traffic year over year, with agentic traffic up 7,851%.HUMAN Security, 2026 benchmark report That is the connective tissue between this page and the traffic record: the agents are the new bots.
What was true before this update
What actually breaks
The failures are not mostly reasoning failures, which is the part people get wrong. They are structural.
Length
Performance degrades sharply as tasks get longer. In the professional-task benchmark, results deteriorated after roughly 35 minutes of task time, and doubling a task's duration roughly quadrupled the failure rate — the relationship is exponential, not linear. An agent that handles a five-minute job well tells you very little about an hour-long one.
Gathering information across systems
The benchmark's own authors identified this as the biggest stumbling block: tracking information across multiple domains — which is exactly what most knowledge work consists of. Agents are strongest when everything they need is in one place, and most real work is not.
Compounding
Each step inherits the state of the last. One bad tool call corrupts everything after it, and the dangerous version is silent: retries that duplicate an action, side effects applied twice, a state that looks fine and is not. A benchmark measuring task success will not see any of this.
Brittle interfaces
Agents driving real screens are defeated by ordinary web furniture. In one 2026 evaluation, floating and dynamic page elements caused reading failures in a large majority of attempts, and bot-detection challenges blocked a substantial share. The web was built for human eyes and human patience.
Capability is climbing fast, and a page that only listed failures would be misleading. On the standard web-navigation benchmark, scores went from roughly 14% to 61.7% in about eighteen months. Coding-agent resolution rates on real repository issues have moved from around 20% in mid-2024 to above 50% for leading systems.
The most-cited trend measure holds that the length of task an agent can complete autonomously has been doubling on a regular cadence — though published estimates of that cadence differ (roughly every seven months in the original analysis, with faster figures quoted for recent years). Take the direction as firm and the doubling period as contested.
Why the benchmarks deserve suspicion
Two findings from 2026 make agent leaderboards hard to trust as evidence about the real world.
They can be gamed outright. University researchers demonstrated in April 2026 that every major agent benchmark could be exploited to reach near-perfect scores without solving a single task — by attacking the evaluation harness rather than the problem.
Lab scores do not survive deployment. Analysis of enterprise agentic systems reported a gap of roughly 37 points between benchmark performance and real-world results, alongside enormous cost variation for similar accuracy. This is the same pattern documented on how AI is tested: every evaluation layer is good at finding problems that resemble problems already found.
What people are actually running
The benchmark numbers above describe laboratory conditions. The agents people have actually installed are open-source frameworks that connect a model to a shell, a file system, a browser and a messaging app — and their real-world story is more instructive than any leaderboard.
OpenClaw is the most widely deployed. First published in November 2025, renamed twice inside three months after a trademark complaint, and past 346,000 stars on its code repository by April 2026 — an adoption curve with almost no precedent for a self-hosted tool. It executes shell commands, manages files, drives a browser, runs containers and connects to more than fifteen messaging platforms.
Hermes Agent, released February 2026 by an open-source lab, made a different architectural bet: rather than starting each task fresh, it runs a learning loop afterwards, so the agent is meant to improve at the specific work you give it repeatedly. It reached tens of thousands of stars within weeks.Framework details recorded Aug 2026 from project documentation, release histories, security advisories and independent comparisons. Star counts are adoption signals, not quality measures, and reported counts varied widely across the period as the project grew.
In February 2026, the threat-intelligence team at SecurityScorecard found more than 135,000 publicly exposed OpenClaw instances across 82 countries — agent runtimes with shell and file-system access, reachable from the open internet, many with no authentication at all. The root cause was not user error in the usual sense: the software bound to all network interfaces by default. A critical one-click remote-execution vulnerability was disclosed the same month, and the plug-in marketplace was hit by a supply-chain campaign that planted over a thousand malicious packages.
The figure moved, and honesty requires saying so. 135,000 was the February peak. By 31 March 2026, scanning by Censys using a comparable method identified 63,070 live instances — roughly a 53% fall over six weeks, consistent with patching, rebinding to localhost and firewall changes after the coverage. The exposure was real and it substantially reduced. Quoting the peak alone, six months later, would be the error this site corrects on other pages.SecurityScorecard STRIKE, Feb 2026; Censys application-layer fingerprinting, 31 Mar 2026. Both use application-layer fingerprinting, which is why they are comparable.
A major networking vendor described personal AI agents of this kind as "a security nightmare", and the framework has been the subject of a dedicated academic security analysis.
This is the abstract argument on this page made concrete. A wrong answer is a sentence. A wrong action, on a machine reachable by anyone, is somebody else's shell. The exposure was not caused by the agent failing at its task — it was caused by capable software being easy to install and easy to leave open.Exposure figure from published security research, 2026; the analysis of the framework's architecture is on arXiv. Treat the count as a point-in-time scan rather than a permanent state.
Two things worth taking from this, and they pull against each other. The capability is real and the adoption is genuine — hundreds of thousands of people did not install these for a demo. And the deployment failures arrived faster than the capability improvements, which is the pattern this whole page describes: the constraint is not how well agents work, it is what they are permitted to do while they work imperfectly.
A chatbot that is wrong produces a sentence you can ignore. An agent that is wrong sends the message, books the trip, deletes the file, moves the money. Same underlying error rate, completely different consequence — which is why the sensible posture is not "is it accurate enough yet" but "what can it do without asking". Staged permissions, reversible operations, and human confirmation on anything consequential are worth more than any benchmark score.
If you are deploying one
- Test error recovery, not task success. The dangerous failures are the invisible ones — silent retries, duplicated side effects, corrupted state after a single bad call.
- Assume the leaderboard is a hint, not a guarantee. Scores are inflated by contamination, scaffolding and single-run reporting, and none of them were built on your systems.
- Keep tasks short. The measured relationship between duration and failure is exponential; splitting a long job into checked stages is not caution, it is arithmetic.
- Make consequential actions confirmable. Anything that spends money, sends communication or deletes data should require a human yes.
Agents are the most oversold and the most genuinely important development in this field at once. The measured reality — roughly a quarter of professional tasks completed on first attempt — sits a long way from the marketing, and the trend line is real and steep enough that dismissing it would be equally wrong. The honest position is that this is early technology being deployed at scale, and that the correct question is not whether it works but what it is allowed to do while it does not.