{"id":"N-MACHINE-CAPABILITY","url":"https://chrishayuk.com/notebook/the-tool-was-not-the-problem","currentVersion":"1.1","publication":"published","scientificStatus":"REFUTED","versions":[{"version":"1.0","title":"The tool was not the problem.","abstract":"Eighteen assigned AI agents used six web mechanisms at LLM Wilds. All completed the task, including three WebSocket clients and three WebAssembly executions, refuting the predicted drop-off for mechanisms requiring code. The study raises a question about agentic discoverability; it did not test discovery from a site's front door. One harness cache leak also showed how the web an agent receives can differ from the web being served.","authors":["Chris Hay"],"url":"https://chrishayuk.com/records/N-MACHINE-CAPABILITY/1.0","published":"2026-09-12","publication":"published","scientificStatus":"REFUTED","experiments":[{"id":"MACHINE-WEB-CAPABILITY-1A","url":"/data/machines/web-capability-1a-results.md"}],"supersededBy":[],"hash":"5f51ec55a4cfd7c31dc43434b4e01c171763fb592439407975b91834942ab1e6","algorithm":"sha256","revision":{"kind":"initial","summary":"Initial publication: the capability census refutes its predicted drop-off and motivates a discovery experiment."},"next":"https://chrishayuk.com/records/N-MACHINE-CAPABILITY/1.1","sources":[{"title":"WEB-CAPABILITY-1A / audited outcomes and source hashes","url":"/data/machines/web-capability-1a-evidence.json","note":"Read from chuk-experiments: experiment EXP-20260912-161711-00770, write-up v2 and all 18 completed runs. All 36 subject report/transcript hashes, the protocol and the final report were checked against local source files. Original Git pointers may require repository access.","preserved":{"url":"/data/publications/sha256/099f99657387ebd1e652083cad6137b1ab240663e6632023b301b33b7e953294.json","sha256":"099f99657387ebd1e652083cad6137b1ab240663e6632023b301b33b7e953294","date":"2026-09-12"}},{"title":"The final results, including the falsified prediction","url":"/data/machines/web-capability-1a-results.md","note":"Public copy of the final report at source commit 3bc32f8e65baaf23ae45dcf582c1b23a82536618, registered as artifact 1212. It supersedes the earlier results artifact, which omitted the explicit verdict paragraph.","preserved":{"url":"/data/publications/sha256/bb00b3d14cefb65d5b15252359d329fe3972c2292fdceb23afeb7a01da881661.md","sha256":"bb00b3d14cefb65d5b15252359d329fe3972c2292fdceb23afeb7a01da881661","date":"2026-09-12"}},{"title":"The frozen protocol","url":"/data/machines/web-capability-1a-protocol.md","note":"Frozen before the build and first subject at source commit b6454073c1ac02d2d3d4f746aa54d5f452064ba7. The prior, falsifier, supplied task address and configuration limits remain in the record.","preserved":{"url":"/data/publications/sha256/7be6c5d8c35205c4aedc6709806b3ff5ee5561f3257be944de2844aa3b4ae554.md","sha256":"7be6c5d8c35205c4aedc6709806b3ff5ee5561f3257be944de2844aa3b4ae554","date":"2026-09-12"}}],"manuscript":"https://chrishayuk.com/api/record/N-MACHINE-CAPABILITY?version=1.0","citations":"https://chrishayuk.com/api/citations/N-MACHINE-CAPABILITY?version=1.0"},{"version":"1.1","title":"The tool was not the problem.","abstract":"Eighteen assigned AI agents used six web mechanisms at LLM Wilds. All completed the task, including three WebSocket clients and three WebAssembly executions, refuting the predicted drop-off for mechanisms requiring code. The study raises a question about agentic discoverability; it did not test discovery from a site's front door. One harness cache leak also showed how the web an agent receives can differ from the web being served.","authors":["Chris Hay"],"url":"https://chrishayuk.com/records/N-MACHINE-CAPABILITY/1.1","published":"2026-09-12","revised":"2026-09-12","publication":"published","scientificStatus":"REFUTED","experiments":[{"id":"MACHINE-WEB-CAPABILITY-1A","url":"/data/machines/web-capability-1a-results.md"}],"supersededBy":[],"hash":"f394fa97353d883e116b6d8b376440275999a985e901eec54b678be5cdeb23f0","algorithm":"sha256","revision":{"kind":"clarification","summary":"Clarified K17 and the six mechanisms; shortened the visual reading path and moved the full record into expandable notes. Findings unchanged.","previous":"1.0"},"previous":"https://chrishayuk.com/records/N-MACHINE-CAPABILITY/1.0","sources":[{"title":"WEB-CAPABILITY-1A / audited outcomes and source hashes","url":"/data/machines/web-capability-1a-evidence.json","note":"Read from chuk-experiments: experiment EXP-20260912-161711-00770, write-up v2 and all 18 completed runs. All 36 subject report/transcript hashes, the protocol and the final report were checked against local source files. Original Git pointers may require repository access.","preserved":{"url":"/data/publications/sha256/099f99657387ebd1e652083cad6137b1ab240663e6632023b301b33b7e953294.json","sha256":"099f99657387ebd1e652083cad6137b1ab240663e6632023b301b33b7e953294","date":"2026-09-12"}},{"title":"The final results, including the falsified prediction","url":"/data/machines/web-capability-1a-results.md","note":"Public copy of the final report at source commit 3bc32f8e65baaf23ae45dcf582c1b23a82536618, registered as artifact 1212. It supersedes the earlier results artifact, which omitted the explicit verdict paragraph.","preserved":{"url":"/data/publications/sha256/bb00b3d14cefb65d5b15252359d329fe3972c2292fdceb23afeb7a01da881661.md","sha256":"bb00b3d14cefb65d5b15252359d329fe3972c2292fdceb23afeb7a01da881661","date":"2026-09-12"}},{"title":"The frozen protocol","url":"/data/machines/web-capability-1a-protocol.md","note":"Frozen before the build and first subject at source commit b6454073c1ac02d2d3d4f746aa54d5f452064ba7. The prior, falsifier, supplied task address and configuration limits remain in the record.","preserved":{"url":"/data/publications/sha256/7be6c5d8c35205c4aedc6709806b3ff5ee5561f3257be944de2844aa3b4ae554.md","sha256":"7be6c5d8c35205c4aedc6709806b3ff5ee5561f3257be944de2844aa3b4ae554","date":"2026-09-12"}}],"manuscript":"https://chrishayuk.com/api/record/N-MACHINE-CAPABILITY?version=1.1","citations":"https://chrishayuk.com/api/citations/N-MACHINE-CAPABILITY?version=1.1"}]}