My agents read the web constantly, and for a long time they did it badly: one search provider’s opinion of a query, raw HTML soup for page content, or a full browser automation stack for what should have been a text lookup. webctx is the tool I actually wanted them holding: three verbs, terminal-native, output an agent can paste, diff, store, or cite. search asks Brave, Tavily, and Exa the same question at once and returns one ranked list. read-link turns a URL into clean markdown. map-site returns a sitemap-style list of URLs so an agent can survey a docs site before deciding what to read. That’s the whole surface, and every command exists because an agent loop of mine needed exactly it.
No single search provider is right often enough
The reason search is multi-provider isn’t redundancy. It’s that Brave, Tavily, and Exa are good at genuinely different things, and for agent research the cost of a missed result is higher than the cost of three API calls. The interesting work is turning three opinions into one list. URLs get normalized first, with lowercased scheme and host, trailing slashes trimmed, and tracking parameters (utm_*, fbclid, gclid, ref) stripped, so the same page from two providers is recognized as the same page. Then scoring: position points on a declining table (30 for a provider’s first result down to 1), plus a duplicate bonus when multiple providers independently surface the same normalized URL. Cross-provider agreement is the strongest relevance signal available, with a small penalty past three appearances so agreement doesn’t compound into distortion. Ties sort lexicographically, because deterministic output is a feature when your consumer is a test suite or an agent comparing runs. Top 35 results out, excluded domains and keyword filters applied on the way.
Don’t scrape what you can just fetch
read-link is built around a cost gradient. A huge share of what agents read is GitHub files and markdown pages, and scraping those through a headless-browser service is paying rendering prices for plain text. So there are fast paths: GitHub URLs get checked against the raw-content path first, markdown-style URLs against their direct .md form, and only when neither works does the request fall back to Firecrawl for real extraction. Docs and repos, the common case, come back instantly and free; the open web still works. Credentials load from env vars, a .env.local beside the binary, or the macOS Keychain, same pattern as spaceship-cli, and a missing key fails with the exact variable name instead of a generic auth error. An agent can read that and tell me what to fix.
The second version, ported on purpose
webctx is a rewrite. The first version was TypeScript, and it worked. The ranking model here deliberately preserves parity with the old TS scorer, down to a vestigial provider weight. But a context tool for agents gets installed on fresh sandboxes and remote machines constantly, and “first install Node” is real friction for a single-purpose binary. The port to pure Go, for the same reasons as agentbox and procoder, made it a small self-contained executable, delivered via npm i -g webctx (the npm package just unpacks the right platform binary) or straight from GitHub releases.
It’s also documented like a real tool rather than a weekend script: a full Astro docs site covering the ranking internals, agent workflow recipes, credentials, and release checks, built on the shared zuedocs template, so keeping the docs current is a dependency update, not a chore.