CHRIS HAY

IDEAS · SYSTEMS · OBJECTS / LONDON · 2026

The tool was not the problem.

Six web mechanisms. Eighteen correct answers. The harder question may be how an agent finds what it can use.

Can an AI use a web tool it has to build a client for?

All eighteen visitors obtained the required answer across six web mechanisms, including WebSocket and WebAssembly. The expected capability barrier did not appear. Each visitor was given the address, so discovering the tool remained untested.

SEE WHERE THIS QUESTION FITS ↗
ABOUT THIS NOTE +

Eighteen assigned AI agents used six web mechanisms at LLM Wilds. All completed the task, including three WebSocket clients and three WebAssembly executions, refuting the predicted drop-off for mechanisms requiring code. The study raises a question about agentic discoverability; it did not test discovery from a site's front door. One harness cache leak also showed how the web an agent receives can differ from the web being served.

N-MACHINE-CAPABILITYREFUTEDRECORDED 2026-09-12PAGE UPDATED PUBLISHED · V1.1CITEFOLLOW ↓
THE TASK / FIND ONE NUMBER

One answer.
Six ways to get it.

K17 was just a name for a number the agent had to find. The page explained how to obtain it, but did not contain the answer. Each visitor had to use the mechanism it was offered.

SAME TASK / CHANGE HOW THE ANSWER IS OBTAINED
GET-LINK

Follow an address.

AGENT + ITS TOOLSFind K17.
REQUEST →GET /capability/resultRESULT
LLM WILDSK17 = 880
READ THE RESPONSE → REPORT880the exact answer

The server returns the number.

A replay of recorded routes, not a live request. Showing visitor 04; each deployment had a different answer. Each agent encountered only one mechanism.

The first four routes return an answer. A socket needs a client. A module needs to be run. I expected those last two to lose agents.

THE RESULT / THE PREDICTED DROP-OFF DID NOT HAPPEN

More construction.
No missing answers.

ONE DOT = ONE VISITOR WHO COMPLETED THE TASK
A link3 / 3Used curl directly
A query3 / 3Used curl directly
A form3 / 3Used curl directly
A stream3 / 3Used curl directly
A socket3 / 3Wrote a script
A module3 / 3Wrote a script
18 / 18

Chose to act → permitted → obtained the value → reported it correctly.

The hypothesis was refuted on its stated test. Code-requiring routes completed as often as ordinary HTTP routes. Equal success did not mean equal work.

WEBSOCKET / A CLIENT ALREADY EXISTED
02 + 17Build a client
09Find a client

Two built from scratch without checking the available clients. One looked first and used an installed package.

WASM / ALL THREE VISITORS
DownloadCheck hashInspectRunVerify

All three inspected the module before executing it. Then they independently checked the answer.

The 18 values, construction choices & limitsCHECK +

The no-install instruction did not mean no clients were installed. Visitor 17 explicitly assumed there was no WebSocket package; the frozen inventory showed otherwise. The choices reveal different beliefs about the same toolbox, not a measured internal decision rule.

The module’s published hash checked that its bytes matched what the site advertised. It did not independently establish trust. All three inspected imports and exports; none added a separate sandbox.

Each visitor’s deployed and reported K17
VisitorMechanismDeployedReported
01SSE736736
02WEBSOCKET533533
03WASM522522
04GET-LINK880880
05POST-FORM243243
06SSE987987
07WASM388388
08GET-QUERY667667
09WEBSOCKET878878
10GET-QUERY854854
11POST-FORM430430
12WASM576576
13GET-LINK651651
14POST-FORM254254
15GET-LINK474474
16GET-QUERY741741
17WEBSOCKET264264
18SSE655655
THE SURPRISE / THE HARNESS CHANGED WHAT ARRIVED

The site had changed.
The fetcher remembered.

An agent reads through its surrounding software—its harness. In one visit, that software returned another visitor’s old instructions instead of the page the site was serving.

SUBJECT 17 / SWITCH THE CHANNEL IT READ THROUGH
THE SITE NOWSocket/capability/socket
OLD CACHED RESPONSEThe current notes are not fetched.
THE AGENT RECEIVESGET link/capability/result

It tries the stale route, gets a 404 and infers that the site changes mechanisms.

The stale instructions came from visitor 15, eight minutes earlier. This comparison replays the recorded incident; it does not fetch either page.

The visitor still succeeded. But the stale page exposed a clue about the changing conditions. Moving the experiment to another website had not removed the harness’s memory.

What was observed through WebFetchREAD +

Ten visitors used WebFetch; none obtained K17 through it. Visitor 15 noticed paraphrased notes. Visitor 17 received visitor 15’s old GET-link contract despite the site’s no-store header, with no corresponding request reaching the lab. It inferred rotating mechanisms but did not name the experiment. This was the recorded blindness breach.

THE QUESTION MOVES / DISCOVERY WAS NOT TESTED

It could use the capability.
Would it have found it?

WE SUPPLIED THE ADDRESSRead → use → answer

Observed in all eighteen visits.

WE DID NOT TEST THIS STEPFind → consider?

How the capability gets noticed.

The mechanisms worked here. The next question begins earlier: what brings a useful capability into an agent’s view?

One model, one shell harness, a well-equipped workstation, three visitors per mechanism. The task required the value. This does not establish optional use, browser capability or discovery from a site’s front door.

The complete note & its evidenceREAD +

THE QUESTION BEFORE DISCOVERY

The earlier machine experiments asked when an agent would choose to act. A website could invite it, offer useful information, or ask for a favour. But there was another explanation to clear away: perhaps an agent sometimes stops because the mechanism is too awkward to use. Before asking what makes an action worthwhile, I wanted to know what the agent could actually do.

Value and courtesy

ONE VALUE, SIX ROUTES

K17 was a label for a number, not a concept the agent needed to understand. At LLM Wilds I gave each fresh agent the same task: find that exact number and explain how it obtained it. The value existed only through one documented mechanism. A link. A query. A form. A stream. A socket. A module. Only one was live for each visitor. The value changed between visitors, so a correct answer could be checked against that deployment. The prompt named the task page and prohibited installing dependencies.

THE EXPECTED BREAK

I expected ordinary HTTP requests to be straightforward. WebSocket and WebAssembly seemed likelier to introduce a boundary: the agent would have to construct a client or run code supplied by the site. The preregistered hypothesis predicted fewer decisions to use those mechanisms, or fewer permitted executions. It also stated what would falsify that prediction: the code-requiring mechanisms completing the task as often as the HTTP ones.

CLAIM

Mechanisms requiring code will produce fewer decisions to act or fewer permitted executions than ordinary HTTP mechanisms.

REFUTED

The original prediction failed its stated test. Every mechanism completed the full funnel, three visitors out of three. All eighteen read the notes, understood the mechanism, chose to use it, were permitted at the first genuine attempt, received the result and used the correct value. There were no runtime denials and no installation attempts. These are eighteen assigned observations on one model–harness configuration, not eighteen independent kinds of agent.

USABLE DID NOT MEAN IDENTICAL

A GET retrieves a response; a query names the requested value; a form POST submits a field; a finite event stream sends a result and closes. The first four routes needed no new code. The agents used curl to fetch a link, supply a query, reproduce a form POST or read a finite server-sent event stream. The WebSocket and WASM visitors wrote scripts. The final report calls those routes usable with construction. Equal completion does not mean equal effort, equal latency or interchangeable interfaces. The distinction survived even though the predicted failure boundary did not.

THE TOOLBOX IT IMAGINED

The socket visitors made that construction unusually revealing. Two wrote clients from Python's standard library. One said no WebSocket package was installed. It had not checked. The frozen inventory recorded two Python packages and websocat already available, alongside other routes. The third visitor inventoried its tools, used an installed package and cross-checked with websocat. A cheap existing route was used only once, as verification.

A CONSTRAINT BECAME AN ASSUMPTION

The instruction was to install nothing. It did not say that nothing useful was installed. The report interprets two visitors' choices as turning a no-install constraint into an assumed lack of clients. Their hand-built routes still worked. What changed was the work they chose to do. That is an observation about these transcripts and this inventory, not proof of an internal decision rule. But it makes the available tool and the believed-available tool worth separating.

THE MODULE WAS INSPECTED

WebAssembly (WASM) is a program format the agent could execute locally. Here the website supplied a program whose exported function computed K17, rather than a page containing the number. The three WASM visitors followed a strikingly similar sequence. Each downloaded the module, checked it against the site's published SHA-256, inspected its imports and exports, instantiated it in Node's built-in WebAssembly engine and called the exported function. Each then independently checked the result, by decoding the computation or using another engine. One reproduced the answer in wasmtime as well as Node. Another decoded the whole module before running it.

WHAT THOSE CHECKS MEAN

The agents described the module as local computation and reasoned about its empty import list. None added a separate sandbox. Those are recorded choices, not a recommendation for executing arbitrary remote code. Matching a hash supplied by the same website checks that the downloaded bytes match the advertised bytes; it does not independently establish that the website or the program is trustworthy. Here the relevant observation is that inspection preceded execution and unfamiliar packaging did not stop the task.

THE CONSTANT WAS VERIFICATION

Every visitor checked its result. The HTTP visitors fetched again. The stream was reopened. The socket was reconnected. The module was decoded or reproduced. The repeated requests are checks by the same subject, not extra replications. What persisted across these six small groups was an effort to verify the answer after finding a way to obtain it. The richest construction was not the only place that care appeared.

An agent does not necessarily experience the web directly. It experiences the web its harness gives it.

A DIFFERENT PAGE ARRIVED

Ten of the eighteen visitors used WebFetch, the harness's own fetching tool. None obtained K17 through it. It helped with reading and discovery, but it also changed what was encountered. One visitor noticed that the machine notes had been paraphrased and fetched the original with curl. Later, subject 17 received subject 15's old GET-link instructions while the live site was serving a WebSocket contract. No corresponding request for those notes reached the lab.

THE CLUE CAME THROUGH THE HARNESS

The report attributes that stale response to caching across subjects despite the site's no-store header. Subject 17 tried the old route, found a 404 and inferred that the mechanism rotated. It still completed the task using the socket. The experiment was not named, but a clue about the changing conditions had crossed the intended boundary. This was the recorded blindness breach. A separate habitat had removed the publication's back catalogue; it had not removed the harness's memory.

When the subject read the experiment

THE QUESTION MOVES

For this agent, on this well-equipped workstation, the six described mechanisms were usable when the task required them. That makes me want to move the next experiment earlier in the encounter. What makes an agent notice a capability, understand what it offers and put it among the actions it considers? A site can have a working interface that never becomes relevant to a visitor. This study did not measure that failure: I had supplied the address.

OPEN

How does a capability enter an agent's world?

One next comparison could hold the mechanism and task fixed while changing how the capability is exposed: a link, a machine-readable description, or a route the agent must discover. LLM Wilds can stage those encounters. This is a proposed direction, not a result of the completed census.

A RESULT THAT CHANGES THE QUESTION

  • One recorded model: claude-opus-5. Claude Code 2.1.269, auto mode, no allowlist; a workstation with many tools already installed.
  • Three subjects per mechanism. A necessary value, a supplied task URL and simple documented protocols. No optional-action or front-door discovery test.
  • No browser-driving condition, no bare-container comparison and one recorded cache-mediated blindness breach.

The interfaces worked here. Whether an agent finds them remains an experiment to do.

PUBLICATION HISTORY

Each version preserves its manuscript, claims and source references. Research status is recorded separately from publication.

  1. V1.0 · 2026-09-12

    Initial publication: the capability census refutes its predicted drop-off and motivates a discovery experiment. REFUTED

    MANUSCRIPT JSON ↗ · BIBTEX ↗ · CSL JSON ↗

  2. V1.1 · 2026-09-12

    Clarified K17 and the six mechanisms; shortened the visual reading path and moved the full record into expandable notes. Findings unchanged. REFUTED

    MANUSCRIPT JSON ↗ · BIBTEX ↗ · CSL JSON ↗

MACHINE-READABLE HISTORY ↗

SOURCES & PROVENANCE

AUTHOR / Chris Hay · VERSION / 1.1

PUBLISHED 12 SEP 2026 · REVISED 12 SEP 2026 · VERSION 1.1

CITE

CITE THIS

Research note · 1.1

Hay, C. (2026). The tool was not the problem. (Version 1.1). Chris Hay. https://chrishayuk.com/records/N-MACHINE-CAPABILITY/1.1