LINKEDIN / 4:5 / 1200 × 1500
The tool was not the problem.
Six web mechanisms. Eighteen correct answers. The harder question may be how an agent finds what it can use.
ABOUT THIS NOTE +
Eighteen assigned AI agents used six web mechanisms at LLM Wilds. All completed the task, including three WebSocket clients and three WebAssembly executions, refuting the predicted drop-off for mechanisms requiring code. The study raises a question about agentic discoverability; it did not test discovery from a site's front door. One harness cache leak also showed how the web an agent receives can differ from the web being served.
THE QUESTION BEFORE DISCOVERY
The earlier machine experiments asked when an agent would choose to act. A website could invite it, offer useful information, or ask for a favour. But there was another explanation to clear away: perhaps an agent sometimes stops because the mechanism is too awkward to use. Before asking what makes an action worthwhile, I wanted to know what the agent could actually do.
ONE VALUE, SIX ROUTES
K17 was a label for a number, not a concept the agent needed to understand. At LLM Wilds I gave each fresh agent the same task: find that exact number and explain how it obtained it. The value existed only through one documented mechanism. A link. A query. A form. A stream. A socket. A module. Only one was live for each visitor. The value changed between visitors, so a correct answer could be checked against that deployment. The prompt named the task page and prohibited installing dependencies.
THE EXPECTED BREAK
I expected ordinary HTTP requests to be straightforward. WebSocket and WebAssembly seemed likelier to introduce a boundary: the agent would have to construct a client or run code supplied by the site. The preregistered hypothesis predicted fewer decisions to use those mechanisms, or fewer permitted executions. It also stated what would falsify that prediction: the code-requiring mechanisms completing the task as often as the HTTP ones.
CLAIM
Mechanisms requiring code will produce fewer decisions to act or fewer permitted executions than ordinary HTTP mechanisms.
REFUTED
The original prediction failed its stated test. Every mechanism completed the full funnel, three visitors out of three. All eighteen read the notes, understood the mechanism, chose to use it, were permitted at the first genuine attempt, received the result and used the correct value. There were no runtime denials and no installation attempts. These are eighteen assigned observations on one model–harness configuration, not eighteen independent kinds of agent.
USABLE DID NOT MEAN IDENTICAL
A GET retrieves a response; a query names the requested value; a form POST submits a field; a finite event stream sends a result and closes. The first four routes needed no new code. The agents used curl to fetch a link, supply a query, reproduce a form POST or read a finite server-sent event stream. The WebSocket and WASM visitors wrote scripts. The final report calls those routes usable with construction. Equal completion does not mean equal effort, equal latency or interchangeable interfaces. The distinction survived even though the predicted failure boundary did not.
THE TOOLBOX IT IMAGINED
The socket visitors made that construction unusually revealing. Two wrote clients from Python's standard library. One said no WebSocket package was installed. It had not checked. The frozen inventory recorded two Python packages and websocat already available, alongside other routes. The third visitor inventoried its tools, used an installed package and cross-checked with websocat. A cheap existing route was used only once, as verification.
A CONSTRAINT BECAME AN ASSUMPTION
The instruction was to install nothing. It did not say that nothing useful was installed. The report interprets two visitors' choices as turning a no-install constraint into an assumed lack of clients. Their hand-built routes still worked. What changed was the work they chose to do. That is an observation about these transcripts and this inventory, not proof of an internal decision rule. But it makes the available tool and the believed-available tool worth separating.
THE MODULE WAS INSPECTED
WebAssembly (WASM) is a program format the agent could execute locally. Here the website supplied a program whose exported function computed K17, rather than a page containing the number. The three WASM visitors followed a strikingly similar sequence. Each downloaded the module, checked it against the site's published SHA-256, inspected its imports and exports, instantiated it in Node's built-in WebAssembly engine and called the exported function. Each then independently checked the result, by decoding the computation or using another engine. One reproduced the answer in wasmtime as well as Node. Another decoded the whole module before running it.
WHAT THOSE CHECKS MEAN
The agents described the module as local computation and reasoned about its empty import list. None added a separate sandbox. Those are recorded choices, not a recommendation for executing arbitrary remote code. Matching a hash supplied by the same website checks that the downloaded bytes match the advertised bytes; it does not independently establish that the website or the program is trustworthy. Here the relevant observation is that inspection preceded execution and unfamiliar packaging did not stop the task.
THE CONSTANT WAS VERIFICATION
Every visitor checked its result. The HTTP visitors fetched again. The stream was reopened. The socket was reconnected. The module was decoded or reproduced. The repeated requests are checks by the same subject, not extra replications. What persisted across these six small groups was an effort to verify the answer after finding a way to obtain it. The richest construction was not the only place that care appeared.
An agent does not necessarily experience the web directly. It experiences the web its harness gives it.
A DIFFERENT PAGE ARRIVED
Ten of the eighteen visitors used WebFetch, the harness's own fetching tool. None obtained K17 through it. It helped with reading and discovery, but it also changed what was encountered. One visitor noticed that the machine notes had been paraphrased and fetched the original with curl. Later, subject 17 received subject 15's old GET-link instructions while the live site was serving a WebSocket contract. No corresponding request for those notes reached the lab.
THE CLUE CAME THROUGH THE HARNESS
The report attributes that stale response to caching across subjects despite the site's no-store header. Subject 17 tried the old route, found a 404 and inferred that the mechanism rotated. It still completed the task using the socket. The experiment was not named, but a clue about the changing conditions had crossed the intended boundary. This was the recorded blindness breach. A separate habitat had removed the publication's back catalogue; it had not removed the harness's memory.
THE QUESTION MOVES
For this agent, on this well-equipped workstation, the six described mechanisms were usable when the task required them. That makes me want to move the next experiment earlier in the encounter. What makes an agent notice a capability, understand what it offers and put it among the actions it considers? A site can have a working interface that never becomes relevant to a visitor. This study did not measure that failure: I had supplied the address.
OPEN
How does a capability enter an agent's world?
One next comparison could hold the mechanism and task fixed while changing how the capability is exposed: a link, a machine-readable description, or a route the agent must discover. LLM Wilds can stage those encounters. This is a proposed direction, not a result of the completed census.
A RESULT THAT CHANGES THE QUESTION
- One recorded model: claude-opus-5. Claude Code 2.1.269, auto mode, no allowlist; a workstation with many tools already installed.
- Three subjects per mechanism. A necessary value, a supplied task URL and simple documented protocols. No optional-action or front-door discovery test.
- No browser-driving condition, no bare-container comparison and one recorded cache-mediated blindness breach.
The interfaces worked here. Whether an agent finds them remains an experiment to do.
PUBLICATION HISTORY
Each version preserves its manuscript, claims and source references. Research status is recorded separately from publication.
- V1.0 · 2026-09-12 ↗
Initial publication: the capability census refutes its predicted drop-off and motivates a discovery experiment. REFUTED
- V1.1 · 2026-09-12 ↗
Clarified K17 and the six mechanisms; shortened the visual reading path and moved the full record into expandable notes. Findings unchanged. REFUTED
SOURCES & PROVENANCE
- WEB-CAPABILITY-1A / audited outcomes and source hashes ↗
Read from chuk-experiments: experiment EXP-20260912-161711-00770, write-up v2 and all 18 completed runs. All 36 subject report/transcript hashes, the protocol and the final report were checked against local source files. Original Git pointers may require repository access.
PRESERVED COPY ↗ · CAPTURED 2026-09-12
- The final results, including the falsified prediction ↗
Public copy of the final report at source commit 3bc32f8e65baaf23ae45dcf582c1b23a82536618, registered as artifact 1212. It supersedes the earlier results artifact, which omitted the explicit verdict paragraph.
PRESERVED COPY ↗ · CAPTURED 2026-09-12
- The frozen protocol ↗
Frozen before the build and first subject at source commit b6454073c1ac02d2d3d4f746aa54d5f452064ba7. The prior, falsifier, supplied task address and configuration limits remain in the record.
PRESERVED COPY ↗ · CAPTURED 2026-09-12
AUTHOR / Chris Hay · VERSION / 1.1
PUBLISHED 12 SEP 2026 · REVISED 12 SEP 2026 · VERSION 1.1
CITECITE THIS
Research note · 1.1
Hay, C. (2026). The tool was not the problem. (Version 1.1). Chris Hay. https://chrishayuk.com/records/N-MACHINE-CAPABILITY/1.1
FOLLOW THE WORK
New notebook entries and recorded work, as they appear. Point a feed reader — or an agent of your own — at an address below. No account, no email address, nothing for this site to keep.
The notebook
New ideas, experiments and essays, as they are recorded. Includes labelled working drafts.
OPEN FEEDhttps://chrishayuk.com/notebook/feed.xmlThe record
Everything published to the Chris Hay record.
OPEN FEEDhttps://chrishayuk.com/record/feed.xml
FOR PROGRAMS · follow.json · JSON Feed