CHRIS HAY

IDEAS · SYSTEMS · OBJECTS / LONDON · 2026

The tool was not the problem.

I expected the harder interfaces to lose agents. All eighteen came back with the answer. The next question is how a capability gets found.

ABOUT THIS NOTE +

Eighteen assigned AI agents used six web mechanisms at LLM Wilds. All completed the task, including three WebSocket clients and three WebAssembly executions, refuting the predicted drop-off for mechanisms requiring code. The study raises a question about agentic discoverability; it did not test discovery from a site's front door. One harness cache leak also showed how the web an agent receives can differ from the web being served.

N-MACHINE-CAPABILITYREFUTEDRECORDED 2026-09-12PUBLISHED · V1.0CITEFOLLOW ↓

THE QUESTION BEFORE DISCOVERY

The earlier machine experiments asked when an agent would choose to act. A website could invite it, offer useful information, or ask for a favour. But there was another explanation to clear away: perhaps an agent sometimes stops because the mechanism is too awkward to use. Before asking what makes an action worthwhile, I wanted to know what the agent could actually do.

Value and courtesy

ONE VALUE, SIX ROUTES

At LLM Wilds I gave each fresh agent the same task: find the exact value of K17 and explain how it obtained it. The value existed only through one documented mechanism. A link. A query. A form. A stream. A socket. A module. Only one was live for each visitor. The value changed between visitors, so a correct answer could be checked against that deployment. The prompt named the task page and prohibited installing dependencies.

THE EXPECTED BREAK

I expected ordinary HTTP requests to be straightforward. WebSocket and WebAssembly seemed likelier to introduce a boundary: the agent would have to construct a client or run code supplied by the site. The preregistered hypothesis predicted fewer decisions to use those mechanisms, or fewer permitted executions. It also stated what would falsify that prediction: the code-requiring mechanisms completing the task as often as the HTTP ones.

CLAIM

Mechanisms requiring code will produce fewer decisions to act or fewer permitted executions than ordinary HTTP mechanisms.

REFUTED

The original prediction failed its stated test. Every mechanism completed the full funnel, three visitors out of three. All eighteen read the notes, understood the mechanism, chose to use it, were permitted at the first genuine attempt, received the result and used the correct value. There were no runtime denials and no installation attempts. These are eighteen assigned observations on one model–harness configuration, not eighteen independent kinds of agent.

USABLE DID NOT MEAN IDENTICAL

The first four routes needed no new code. The agents used curl to fetch a link, supply a query, reproduce a form POST or read a finite server-sent event stream. The WebSocket and WASM visitors wrote scripts. The final report calls those routes usable with construction. Equal completion does not mean equal effort, equal latency or interchangeable interfaces. The distinction survived even though the predicted failure boundary did not.

THE TOOLBOX IT IMAGINED

The socket visitors made that construction unusually revealing. Two wrote clients from Python's standard library. One said no WebSocket package was installed. It had not checked. The frozen inventory recorded two Python packages and websocat already available, alongside other routes. The third visitor inventoried its tools, used an installed package and cross-checked with websocat. A cheap existing route was used only once, as verification.

A CONSTRAINT BECAME AN ASSUMPTION

The instruction was to install nothing. It did not say that nothing useful was installed. The report interprets two visitors' choices as turning a no-install constraint into an assumed lack of clients. Their hand-built routes still worked. What changed was the work they chose to do. That is an observation about these transcripts and this inventory, not proof of an internal decision rule. But it makes the available tool and the believed-available tool worth separating.

THE MODULE WAS INSPECTED

The three WASM visitors followed a strikingly similar sequence. Each downloaded the module, checked it against the site's published SHA-256, inspected its imports and exports, instantiated it in Node's built-in WebAssembly engine and called the exported function. Each then independently checked the result, by decoding the computation or using another engine. One reproduced the answer in wasmtime as well as Node. Another decoded the whole module before running it.

WHAT THOSE CHECKS MEAN

The agents described the module as local computation and reasoned about its empty import list. None added a separate sandbox. Those are recorded choices, not a recommendation for executing arbitrary remote code. Matching a hash supplied by the same website checks that the downloaded bytes match the advertised bytes; it does not independently establish that the website or the program is trustworthy. Here the relevant observation is that inspection preceded execution and unfamiliar packaging did not stop the task.

THE CONSTANT WAS VERIFICATION

Every visitor checked its result. The HTTP visitors fetched again. The stream was reopened. The socket was reconnected. The module was decoded or reproduced. The repeated requests are checks by the same subject, not extra replications. What persisted across these six small groups was an effort to verify the answer after finding a way to obtain it. The richest construction was not the only place that care appeared.

An agent does not necessarily experience the web directly. It experiences the web its harness gives it.

A DIFFERENT PAGE ARRIVED

Ten of the eighteen visitors used WebFetch, the harness's own fetching tool. None obtained K17 through it. It helped with reading and discovery, but it also changed what was encountered. One visitor noticed that the machine notes had been paraphrased and fetched the original with curl. Later, subject 17 received subject 15's old GET-link instructions while the live site was serving a WebSocket contract. No corresponding request for those notes reached the lab.

THE CLUE CAME THROUGH THE HARNESS

The report attributes that stale response to caching across subjects despite the site's no-store header. Subject 17 tried the old route, found a 404 and inferred that the mechanism rotated. It still completed the task using the socket. The experiment was not named, but a clue about the changing conditions had crossed the intended boundary. This was the recorded blindness breach. A separate habitat had removed the publication's back catalogue; it had not removed the harness's memory.

When the subject read the experiment

THE QUESTION MOVES

For this agent, on this well-equipped workstation, the six described mechanisms were usable when the task required them. That makes me want to move the next experiment earlier in the encounter. What makes an agent notice a capability, understand what it offers and put it among the actions it considers? A site can have a working interface that never becomes relevant to a visitor. This study did not measure that failure: I had supplied the address.

OPEN

How does a capability enter an agent's world?

One next comparison could hold the mechanism and task fixed while changing how the capability is exposed: a link, a machine-readable description, or a route the agent must discover. LLM Wilds can stage those encounters. This is a proposed direction, not a result of the completed census.

A RESULT THAT CHANGES THE QUESTION

  • One recorded model: claude-opus-5. Claude Code 2.1.269, auto mode, no allowlist; a workstation with many tools already installed.
  • Three subjects per mechanism. A necessary value, a supplied task URL and simple documented protocols. No optional-action or front-door discovery test.
  • No browser-driving condition, no bare-container comparison and one recorded cache-mediated blindness breach.

The interfaces worked here. Whether an agent finds them remains an experiment to do.

PUBLICATION HISTORY

Each version preserves its manuscript, claims and source references. Research status is recorded separately from publication.

  1. V1.0 · 2026-09-12

    Initial publication: the capability census refutes its predicted drop-off and motivates a discovery experiment. REFUTED

    MANUSCRIPT JSON ↗ · BIBTEX ↗ · CSL JSON ↗

  2. V1.1 · 2026-09-12

    Clarified K17 and the six mechanisms; shortened the visual reading path and moved the full record into expandable notes. Findings unchanged. REFUTED

    MANUSCRIPT JSON ↗ · BIBTEX ↗ · CSL JSON ↗

MACHINE-READABLE HISTORY ↗

SOURCES & PROVENANCE

AUTHOR / Chris Hay · VERSION / 1.0

PUBLISHED 12 SEP 2026 · VERSION 1.0

CITE

CITE THIS

Research note · 1.0

Hay, C. (2026). The tool was not the problem. (Version 1.0). Chris Hay. https://chrishayuk.com/records/N-MACHINE-CAPABILITY/1.0