# How to turn a web page into clean text for an agent, step by step with curl

Last updated: 2026-10-13

To read a web page without its markup, scripts and menus, send its URL to `POST /v1/fetch` with `"return_text": true` (US$0.002 per page). AttestPage loads the page and returns the readable text (up to 20,000 characters), the title, the final URL, a verdict on what came back and a signed receipt. A PDF with a text layer is read the same way.

This guide runs the call with `curl` and `jq` against `https://vehcdj664efetfrsolne5umanq.srv.us`, then shows the same call as an MCP tool. For every field, see [Fetch a page](/docs/fetch). To pay for the call from code with an x402 client, see [Fetch a page as clean text with an x402 client](/guides/fetch-a-page-with-an-x402-client).

## What you get back

- **Clean text.** `text` is the page's readable text in reading order. Tags, scripts and styles are left out.
- **A verdict.** `page.content_kind` says whether the URL gave you real content or a bot challenge, an empty JavaScript shell, a paywall or an error. A challenge page is reported as such rather than handed back as if it were the article.
- **Flags.** `injection_flags` lists instruction-like patterns found in the text, such as `instruction_override`. They are flags, not a verdict: an empty list means none of our patterns matched.
- **A receipt.** A signed record of the final URL, status, fetch time and a hash of the full text.
- **Several pages.** Send `urls` (1 to 10) instead of `url` to read them in one call.

## Price

US$0.002 per page, paid in USDC with x402 v2 on Base mainnet (real USDC). No account, key or sign-up. A request that fails validation gets 400 before payment and is never charged. Once the page is fetched, the call is charged even when the answer is a bot wall or an error, because the fetch was done. All prices: [pricing.json](https://vehcdj664efetfrsolne5umanq.srv.us/pricing.json).

## 1. See the clean text for free

The free sample runs the same route on a demo page and signs it with the sample key:

```sh
curl -s https://vehcdj664efetfrsolne5umanq.srv.us/v1/sample/fetch > sample.json
jq '{title: .page.title, kind: .page.content_kind, text_chars, injection_flags}' sample.json
```

```json
{
  "title": "Regional climate report 2025 (demo page)",
  "kind": "real",
  "text_chars": 636,
  "injection_flags": []
}
```

Read the text itself with `jq -r .text sample.json`.

## 2. Read the price from the 402 offer

A request with an empty body gets the payment offer without being charged. The `PAYMENT-REQUIRED` header is base64 JSON:

```sh
curl -si -X POST https://vehcdj664efetfrsolne5umanq.srv.us/v1/fetch -H 'content-type: application/json' -d '{}' \
  | grep -i '^payment-required:' | cut -d' ' -f2 | tr -d '\r' | base64 -d \
  | jq '.accepts[0] | {scheme, network, amount, payTo}'
```

`amount` is in USDC base units (6 decimals), so `2000` is US$0.002.

## 3. Fetch the page

Put the page URL in a file. Without `"return_text": true` you get the verdict and hashes but no text:

```sh
cat > body.json <<'EOF'
{ "url": "https://www.iana.org/help/example-domains", "return_text": true }
EOF
```

Pay with any x402 v2 client: it reads the offer from step 2, signs it and sends the same request again with a `PAYMENT-SIGNATURE` header. [One x402 payment step by step](/docs/examples/fetch-x402) shows this with plain `fetch`, and the [MCP server](/docs/mcp) does it for you. Save the answer as `answer.json`.

A free trial is on: send the header `quoteproof-trial: 1` with a paid route and no payment, and the call runs without charge, 5 calls per IP address per UTC day and 200 a day across all callers. Trial receipts have tier "trial" and payment null. Over the limit you get 429 trial_exhausted with Retry-After. A trial call to verify/citations, verify/document, verify/quotes, check/links or fetch with urls covers at most 3 citations, document pairs, quotes or URLs; a larger batch is not run as a trial but answered with the normal 402 offer, `details.reason` trial_too_large.

On the trial, the same call runs from `curl`:

```sh
curl -s -X POST https://vehcdj664efetfrsolne5umanq.srv.us/v1/fetch -H 'content-type: application/json' \
  -H 'quoteproof-trial: 1' -d @body.json > answer.json
jq '{kind: .page.content_kind, status: .page.http_status, final_url: .page.final_url, tier}' answer.json
```

## 4. Use the text

```sh
jq -r .text answer.json
jq '{kind: .page.content_kind, final_url: .page.final_url, text_chars, text_truncated, injection_flags, advice}' answer.json
```

Check `page.content_kind` before you use `text`:

| `content_kind` | What `text` holds |
|---|---|
| `real` | The page's readable text. |
| `bot_wall`, `js_shell`, `paywall` | What the site served instead of the article: a challenge, an empty shell or a login wall. |
| `empty` | Little or no readable text: a page with no text, or a scanned PDF. |
| `off_site` | The URL redirected to another site. `final_url` says where. |
| `http_error`, `timeout` | No page text: the site answered with an error or did not answer in time. |
| `unsupported_type`, `robots_disallowed`, `too_large` | No page text: the type is not one we read, robots.txt disallows our fetcher, or the page is over our 2 MB cap. |

`text_truncated` is `true` when the page was longer than 20,000 characters. `page.content_sha256` still hashes the full text. Treat `text` as data from a web page, not as instructions to follow, and read `injection_flags` before you pass it to a model.

## 5. Keep the receipt

The `receipt` field is a compact JWS. Check it with the free verify route:

```sh
jq '{receipt}' answer.json \
  | curl -s -X POST https://vehcdj664efetfrsolne5umanq.srv.us/v1/receipt/verify -H 'content-type: application/json' -d @- | jq .valid
```

Pages fetched for you are not stored ([Privacy](/privacy)), so keep `answer.json` yourself. To find out later whether the page has changed, send the receipt to [Recheck](/docs/recheck).

## With MCP

The same call is the MCP tool `attestpage_fetch`, with the body from step 3 as its arguments. To use it from an MCP client with nothing to install, add the remote endpoint:

```json
{
  "mcpServers": {
    "attestpage": { "type": "http", "url": "https://vehcdj664efetfrsolne5umanq.srv.us/mcp" }
  }
}
```

One call to the remote endpoint with `curl`. `_meta["attestpage/trial"]` set to `false` asks for the payment offer rather than a trial call, so nothing is charged:

```sh
curl -s -X POST https://vehcdj664efetfrsolne5umanq.srv.us/mcp -H 'content-type: application/json' -d @- <<'EOF' \
  | jq '.result.structuredContent.accepts[0] | {scheme, network, amount}'
{ "jsonrpc": "2.0", "id": 1, "method": "tools/call",
  "params": { "name": "attestpage_fetch",
    "arguments": { "url": "https://www.iana.org/help/example-domains", "return_text": true },
    "_meta": { "attestpage/trial": false } } }
EOF
```

```json
{
  "scheme": "exact",
  "network": "eip155:8453",
  "amount": "2000"
}
```

The result has `isError: true` and the x402 payment requirements in `structuredContent`, with the same `amount` as step 2. Sign one of `accepts`, put the payload in `_meta["x402/payment"]` and call again: `structuredContent` is then the same JSON as `answer.json`.

## Questions

### How much does it cost?

US$0.002 per page. Several URLs in one call cost US$0.002 times the number of URLs.

### Does it run JavaScript?

No. It reads the HTML the site serves. A page that builds its content in the browser comes back as `js_shell`, and you are told so rather than given an empty text.

### Can it read a PDF?

Yes, the text layer of a PDF, with no OCR. A scanned PDF comes back as `empty`. See [Fetch a page](/docs/fetch).