Repository navigation
feat(abilities): add datamachine/web-fetch for public pages - #3922
Merged
Merged
Conversation
Contributor
Homeboy Results —
|
Contributor
Homeboy Results —
|
Contributor
Homeboy Results —
|
chubes4
force-pushed
the
fix/issue-3917-data-machine
branch
from
November 6, 2026 22:30
e81232c to
00787a8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3917. Gardner (editor, ability tools only) had no way to read a web page for show or bio research. The
web_fetchchat and pipeline tool existed, but it was not an ability, so REST, MCP and person-scoped sessions couldn't reach it.What's added
datamachine/web-fetch(inc/Abilities/WebFetchAbilities.php): inputurl, outputtitle, readablecontent,final_url,content_length,content_truncatedandfetch_timestamp. It isshow_in_rest, readonly and idempotent, and lives in thedatamachine-contentcategory.PermissionHelper::can( 'web_fetch' )maps topublish_posts, the same author level the chat tool already required (access_level => author). It is also implied bycan_manage().web_fetchchat tool now delegates toWebFetchAbilities::fetch(). Its HTML-to-text extraction was moved over unchanged, so chat output is the same apart from the newfinal_url.Safety (new, applies to the existing chat tool too)
The old tool fetched any URL with WordPress following redirects, including internal addresses. The shared fetch now:
http/httpson ports 80 and 443, and rejects embedded credentials.localhost,*.local,*.internal) and any host that resolves (A and AAAA) to a private, reserved, loopback, link-local (169.254.169.254), CGNAT, unique-local, or IPv4-mapped address. If any resolved address is non-public, the host is refused.redirection = 0on the WP request, max 5 hops) and re-validates every hop, so a public page can't redirect into a metadata endpoint.limit_response_size) and the returned text at 50,000 characters. The timeout is 20s.Residual risk: DNS can change between validation and the request (rebinding). Closing that would require pinning the resolved IP into the transport, which WP's HTTP API doesn't expose. Noted, not addressed here.
Verification
fetch()on production with the new class required (no deploy):https://example.comreturns its title and text.http://extrachill.com/follows the redirect tohttps://extrachill.com/.127.0.0.1,169.254.169.254,[::1]andlocalhost:8080are blocked; so are credentials in the URL andfile://. None of them issued a request.tests/web-fetch-ability-smoke.php(plainphp, stubbed DNS and HTTP) covers: extraction, the size cap, 18 blocked or invalid targets with no request issued, relative and absolute redirect chains, a redirect into the metadata IP blocked before it is requested, redirect loops, truncation, upstream errors, the IP classifier, and the tool delegating to the shared fetch.global-tool-pipeline-modesandglobal-ability-projectionspass.adjacent-handler-tool-policy-smokefails identically on main.Notes
PermissionHelperaction (search_content). If one merges first, the other needs a trivial rebase of the same map and list lines.Provenance: the Homeboy cooks for this issue failed before starting (lab transport retries were exhausted), so Extra Chill Bot wrote this directly. AI-generated (Extra Chill Bot).