Skip to content

POST /v1/fetch opens a page in a headless browser (Cloudflare Browser Run), waits until the network is quiet (networkidle2, at most 30 seconds), and converts the page as it is after its JavaScript has run. It costs 5 credits per fetch ($1.00 per 1,000 fetches), whatever the length of the page. For a PDF link, use /v1/fetch-pdf.

Terminal window
curl -X POST https://api.md.tlelabs.com/v1/fetch \
-H "Authorization: Bearer <YOUR_API_KEY>" \
-H "Content-Type: application/json" \
-d '{"url": "https://example.com/article"}'

Content-Type: application/json, a JSON object of at most 2 MiB:

Field Required Value
url yes An http or https URL without user name or password, on a public host (not localhost or a private address).
preserve_images no true (default) or false, as for /v1/convert.

A field of the wrong type gets 400 invalid_request with details.field; a URL that is refused gets 400 invalid_url.

  1. 5 credits are reserved before the page is opened; with fewer available you get 402 insufficient_credits and nothing is fetched.
  2. The browser opens the page without cookies or a login. It identifies itself as a bot, so a site that blocks bots may show a challenge page, and that page is what you get.
  3. A page that answers an HTTP error (404, 500, …) gets 422 fetch_failed with details.http_status.
  4. A page that is not HTML gets 415 unsupported_content; a PDF names /v1/fetch-pdf. A page without a Content-Type is read as HTML.
  5. Scripts and styles are removed. Relative links and image sources are made absolute from the final URL (after redirects) and the page’s <base>; #anchors, mailto: and data: values are kept as written.
  6. The page is converted like an HTML file sent to /v1/convert with ocr=off.

Any failure after the reservation releases it: a failed fetch is not charged.

200 OK:

{
"markdown": "# Article title\n\n…",
"format": "web",
"usage": { "pages": 3, "ocr_pages": 0, "credits_charged": 5, "credits_remaining": 495 },
"metadata": {
"url": "https://example.com/article",
"final_url": "https://example.com/article",
"http_status": 200,
"byte_size": 81234,
"duration_ms": 5400
},
"warnings": []
}
Field Meaning
usage.pages Pages of the converted HTML (how pages are counted); the price does not depend on it.
metadata.final_url The page URL after redirects; null when the browser did not report it, in which case relative links are kept as in the page and a warning says so.
metadata.http_status HTTP status of the page.
metadata.byte_size UTF-8 size of the page HTML after rendering, before scripts and styles are removed.

With Accept: text/markdown the body is the Markdown, with the headers X-Credits-Charged, X-Credits-Remaining, X-Http-Status, X-Duration-Ms and X-Final-Url (when known).

Status Code When
400 invalid_request Not application/json, invalid JSON, missing url, or a field of the wrong type.
400 invalid_url The URL is not http/https, has credentials, or points to a local or private address.
402 insufficient_credits Fewer than 5 credits available.
413 request_too_large JSON body over 2 MiB.
413 file_too_large The rendered page is larger than the HTML converter accepts.
415 unsupported_content The page is not HTML (a PDF names /v1/fetch-pdf).
422 fetch_failed The page answered an HTTP error (details.http_status).
502 fetch_failed The browser could not open the page (DNS, connection, refused URL).
503 service_unavailable Web fetch is busy (retry after Retry-After) or unavailable.
504 fetch_timeout The page did not finish loading within 30 seconds.

For all of them the reservation is released: an error is not charged. Sending the same URL again is a new fetch and is charged again.