Skip to navigation
Voice Agents APIKnowledge base

Scrape a list of URLs into a knowledge base

View as Markdown

Adds one or more URLs to a knowledge base by scraping each page’s content, chunking it, and indexing for retrieval. Typical flow:

  1. Discover candidate URLs (POST /knowledgebase/get-sitemap-urls or paste your own list).
  2. Call this endpoint with the curated list — scraping runs async.
  3. Poll GET /knowledgebase/{id}/scraped-urls for the per-URL status.

Returns 400 if your account’s KB billing precheck fails (quota or plan limits). Returns 404 if the KB doesn’t belong to your organization.

Authentication

AuthorizationBearer

API key from the console ApiKey collection, sent as Bearer token. Also accepts session cookies for browser-based auth.

Path parameters

idstringRequired

24-character hex id of the target knowledge base.

Request

This endpoint expects an object.
urlslist of stringsRequired

Response

Scrape job(s) queued. Poll /scraped-urls for per-URL status.

statusbooleanOptional
dataobjectOptional

Async job acknowledgement. Inspect /scraped-urls for per-URL progress.

Errors

400
Bad Request Error
401
Unauthorized Error
404
Not Found Error
500
Internal Server Error