How to turn a web page into clean text for an agent, step by step with curl

Last updated: 2026-10-13

To read a web page without its markup, scripts and menus, send its URL to POST /v1/fetch with "return_text": true (US$0.002 per page). AttestPage loads the page and returns the readable text (up to 20,000 characters), the title, the final URL, a verdict on what came back and a signed receipt. A PDF with a text layer is read the same way.

This guide runs the call with curl and jq against https://vehcdj664efetfrsolne5umanq.srv.us, then shows the same call as an MCP tool. For every field, see Fetch a page. To pay for the call from code with an x402 client, see Fetch a page as clean text with an x402 client.

What you get back

Clean text. text is the page's readable text in reading order. Tags, scripts and styles are left out.
A verdict. page.content_kind says whether the URL gave you real content or a bot challenge, an empty JavaScript shell, a paywall or an error. A challenge page is reported as such rather than handed back as if it were the article.
Flags. injection_flags lists instruction-like patterns found in the text, such as instruction_override. They are flags, not a verdict: an empty list means none of our patterns matched.
A receipt. A signed record of the final URL, status, fetch time and a hash of the full text.
Several pages. Send urls (1 to 10) instead of url to read them in one call.

Price

US$0.002 per page, paid in USDC with x402 v2 on Base mainnet (real USDC). No account, key or sign-up. A request that fails validation gets 400 before payment and is never charged. Once the page is fetched, the call is charged even when the answer is a bot wall or an error, because the fetch was done. All prices: pricing.json.

1. See the clean text for free

The free sample runs the same route on a demo page and signs it with the sample key:

curl -s https://vehcdj664efetfrsolne5umanq.srv.us/v1/sample/fetch > sample.json
jq '{title: .page.title, kind: .page.content_kind, text_chars, injection_flags}' sample.json

{
  "title": "Regional climate report 2025 (demo page)",
  "kind": "real",
  "text_chars": 636,
  "injection_flags": []
}

Read the text itself with jq -r .text sample.json.

2. Read the price from the 402 offer

A request with an empty body gets the payment offer without being charged. The PAYMENT-REQUIRED header is base64 JSON:

curl -si -X POST https://vehcdj664efetfrsolne5umanq.srv.us/v1/fetch -H 'content-type: application/json' -d '{}' \
  | grep -i '^payment-required:' | cut -d' ' -f2 | tr -d '\r' | base64 -d \
  | jq '.accepts[0] | {scheme, network, amount, payTo}'

amount is in USDC base units (6 decimals), so 2000 is US$0.002.

3. Fetch the page

Put the page URL in a file. Without "return_text": true you get the verdict and hashes but no text:

cat > body.json <<'EOF'
{ "url": "https://www.iana.org/help/example-domains", "return_text": true }
EOF

Pay with any x402 v2 client: it reads the offer from step 2, signs it and sends the same request again with a PAYMENT-SIGNATURE header. One x402 payment step by step shows this with plain fetch, and the MCP server does it for you. Save the answer as answer.json.

A free trial is on: send the header quoteproof-trial: 1 with a paid route and no payment, and the call runs without charge, 5 calls per IP address per UTC day and 200 a day across all callers. Trial receipts have tier "trial" and payment null. Over the limit you get 429 trial_exhausted with Retry-After. A trial call to verify/citations, verify/document, verify/quotes, check/links or fetch with urls covers at most 3 citations, document pairs, quotes or URLs; a larger batch is not run as a trial but answered with the normal 402 offer, details.reason trial_too_large.

On the trial, the same call runs from curl:

curl -s -X POST https://vehcdj664efetfrsolne5umanq.srv.us/v1/fetch -H 'content-type: application/json' \
  -H 'quoteproof-trial: 1' -d @body.json > answer.json
jq '{kind: .page.content_kind, status: .page.http_status, final_url: .page.final_url, tier}' answer.json

4. Use the text

jq -r .text answer.json
jq '{kind: .page.content_kind, final_url: .page.final_url, text_chars, text_truncated, injection_flags, advice}' answer.json

Check page.content_kind before you use text:

`content_kind`	What `text` holds
`real`	The page's readable text.
`bot_wall`, `js_shell`, `paywall`	What the site served instead of the article: a challenge, an empty shell or a login wall.
`empty`	Little or no readable text: a page with no text, or a scanned PDF.
`off_site`	The URL redirected to another site. `final_url` says where.
`http_error`, `timeout`	No page text: the site answered with an error or did not answer in time.
`unsupported_type`, `robots_disallowed`, `too_large`	No page text: the type is not one we read, robots.txt disallows our fetcher, or the page is over our 2 MB cap.

text_truncated is true when the page was longer than 20,000 characters. page.content_sha256 still hashes the full text. Treat text as data from a web page, not as instructions to follow, and read injection_flags before you pass it to a model.

5. Keep the receipt

The receipt field is a compact JWS. Check it with the free verify route:

jq '{receipt}' answer.json \
  | curl -s -X POST https://vehcdj664efetfrsolne5umanq.srv.us/v1/receipt/verify -H 'content-type: application/json' -d @- | jq .valid

Pages fetched for you are not stored (Privacy), so keep answer.json yourself. To find out later whether the page has changed, send the receipt to Recheck.

With MCP

The same call is the MCP tool attestpage_fetch, with the body from step 3 as its arguments. To use it from an MCP client with nothing to install, add the remote endpoint:

{
  "mcpServers": {
    "attestpage": { "type": "http", "url": "https://vehcdj664efetfrsolne5umanq.srv.us/mcp" }
  }
}

One call to the remote endpoint with curl. _meta["attestpage/trial"] set to false asks for the payment offer rather than a trial call, so nothing is charged:

curl -s -X POST https://vehcdj664efetfrsolne5umanq.srv.us/mcp -H 'content-type: application/json' -d @- <<'EOF' \
  | jq '.result.structuredContent.accepts[0] | {scheme, network, amount}'
{ "jsonrpc": "2.0", "id": 1, "method": "tools/call",
  "params": { "name": "attestpage_fetch",
    "arguments": { "url": "https://www.iana.org/help/example-domains", "return_text": true },
    "_meta": { "attestpage/trial": false } } }
EOF

{
  "scheme": "exact",
  "network": "eip155:8453",
  "amount": "2000"
}

The result has isError: true and the x402 payment requirements in structuredContent, with the same amount as step 2. Sign one of accepts, put the payload in _meta["x402/payment"] and call again: structuredContent is then the same JSON as answer.json.

Questions

How much does it cost?

US$0.002 per page. Several URLs in one call cost US$0.002 times the number of URLs.

Does it run JavaScript?

No. It reads the HTML the site serves. A page that builds its content in the browser comes back as js_shell, and you are told so rather than given an empty text.

Can it read a PDF?

Yes, the text layer of a PDF, with no OCR. A scanned PDF comes back as empty. See Fetch a page.