POST /api/extract

Extract raw Terms of Service text from a URL using the Themys extraction API.

Endpoint

code

POST /api/extract

Extracts Terms of Service text from a URL. Returns the cleaned, readable text along with metadata.

Authentication

Include your Clerk session token in the Authorization header:

code

Authorization: Bearer <your_session_token>

Request Body

json

{
  "url": "https://example.com/terms-of-service"
}

Field	Type	Required	Description
`url`	`string`	Yes	URL of the Terms of Service page to extract

Response

json

{
  "text": "Terms of Service\n\n1. Acceptance of Terms\nBy accessing or using...",
  "wordCount": 4823,
  "source": "jina",
  "lastModified": "2026-01-15T10:30:00.000Z"
}

Field	Type	Description
`text`	`string`	The extracted Terms of Service text, cleaned and formatted
`wordCount`	`number`	Word count of the extracted text
`source`	`string`	Extraction method used: `"cache"`, `"jina"`, or `"fallback"`
`lastModified`	`string \`	null	Last-modified header from the source page, if available

Extraction Sources

Themys uses multiple methods to extract text from URLs, falling back automatically if the primary method fails:

Source	Method	When Used
`cache`	Cached result	Previously extracted URL within the cache window
`jina`	Jina Reader API	Primary method. Renders JavaScript and extracts clean text.
`fallback`	Direct HTTP fetch + HTML parsing	Fallback when Jina fails or returns insufficient content.

The source field in the response tells you which method was used.

Code Examples

JavaScript (fetch)

javascript

const response = await fetch("https://themys.ca/api/extract", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    "Authorization": "Bearer YOUR_SESSION_TOKEN",
  },
  body: JSON.stringify({
    url: "https://example.com/terms-of-service",
  }),
});

const data = await response.json();

console.log(`Extracted ${data.wordCount} words via ${data.source}`);
console.log(data.text);

cURL

bash

curl -X POST https://themys.ca/api/extract \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_SESSION_TOKEN" \
  -d '{"url": "https://example.com/terms-of-service"}'

Error Responses

Missing URL (400)

json

{
  "error": "Missing or invalid 'url' in request body"
}

Extraction Failed (500)

json

{
  "error": "Failed to extract text: <reason>"
}

The page may not contain readable text, be behind authentication, or block automated access. Try submitting the text directly to /api/analyze instead.

Unauthorized (401)

json

{
  "error": "Unauthorized"
}

Check that your Authorization header is set correctly and the token is valid.

Best Practices

Prefer URL input: The API handles JavaScript-rendered pages and cleanup automatically.
Cache results: ToS text doesn't change often. Cache extraction results to avoid redundant requests.
Handle failures gracefully: Always check for error responses and implement retry logic with exponential backoff.
Pass text to /api/analyze: After extracting, send the text field directly to the analyze endpoint for scoring.

Was this page helpful?