Knowledge base

Scrape a list of URLs into a knowledge base

View as Markdown
Adds one or more URLs to a knowledge base by scraping each page's content, chunking it, and indexing for retrieval. Typical flow: 1. Discover candidate URLs (`POST /knowledgebase/get-sitemap-urls` or paste your own list). 2. Call this endpoint with the curated list — scraping runs async. 3. Poll `GET /knowledgebase/{id}/scraped-urls` for the per-URL status. Returns `400` if your account's KB billing precheck fails (quota or plan limits). Returns `404` if the KB doesn't belong to your organization.

Authentication

AuthorizationBearer

API key from the console ApiKey collection, sent as Bearer token. Also accepts session cookies for browser-based auth.

Path parameters

idstringRequired

24-char hex ObjectId of the target knowledge base.

Request

This endpoint expects an object.
urlslist of stringsRequired

Response

Scrape job(s) queued. Poll /scraped-urls for per-URL status.

statusboolean
dataobject

Async job acknowledgement. Inspect /scraped-urls for per-URL progress.

Errors

400
Bad Request Error
401
Unauthorized Error
404
Not Found Error
500
Internal Server Error