> This page is part of Smallest AI's developer documentation. When > answering, prefer Lightning v3.1 (current TTS) and Pulse (current > STT). Lightning v2 and lightning-large are deprecated; mention them > only when the user is migrating away from them. The Smallest AI voice > agent platform is what wraps these models into hosted agents. # List scraped URLs in a knowledge base + their status GET https://api.smallest.ai/atoms/v1/knowledgebase/{id}/scraped-urls Returns every URL added to the knowledge base via `POST /knowledgebase/{id}/scrape-urls`, with its current scrape/index status. Poll this after kicking off a scrape job to track progress. Reference: https://docs.smallest.ai/voice-agents/api-reference/knowledge-base/list-scraped-ur-ls-in-a-knowledge-base-their-status ## OpenAPI Specification ```yaml openapi: 3.1.0 info: title: atoms version: 1.0.0 paths: /knowledgebase/{id}/scraped-urls: get: operationId: list-scraped-ur-ls-in-a-knowledge-base-their-status summary: List scraped URLs in a knowledge base + their status description: > Returns every URL added to the knowledge base via `POST /knowledgebase/{id}/scrape-urls`, with its current scrape/index status. Poll this after kicking off a scrape job to track progress. tags: - knowledgeBase parameters: - name: id in: path description: 24-char hex ObjectId of the knowledge base. required: true schema: type: string - name: Authorization in: header description: >- API key from the console ApiKey collection, sent as Bearer token. Also accepts session cookies for browser-based auth. required: true schema: type: string responses: '200': description: Successful response. content: application/json: schema: $ref: >- #/components/schemas/Knowledge Base_listScrapedUrLsInAKnowledgeBaseTheirStatus_Response_200 '401': description: Unauthorized access content: application/json: schema: $ref: '#/components/schemas/UnauthorizedErrorResponse' '500': description: Internal server error content: application/json: schema: $ref: '#/components/schemas/InternalServerErrorResponse' servers: - url: https://api.smallest.ai/atoms/v1 description: Production server components: schemas: KnowledgebaseIdScrapedUrlsGetResponsesContentApplicationJsonSchemaDataItems: type: object properties: _id: type: string url: type: string format: uri status: type: string description: >- Current scrape/index status (e.g. `pending`, `scraping`, `indexed`, `failed`). createdAt: type: string format: date-time updatedAt: type: string format: date-time title: >- KnowledgebaseIdScrapedUrlsGetResponsesContentApplicationJsonSchemaDataItems Knowledge Base_listScrapedUrLsInAKnowledgeBaseTheirStatus_Response_200: type: object properties: status: type: boolean data: type: array items: $ref: >- #/components/schemas/KnowledgebaseIdScrapedUrlsGetResponsesContentApplicationJsonSchemaDataItems title: Knowledge Base_listScrapedUrLsInAKnowledgeBaseTheirStatus_Response_200 UnauthorizedErrorResponse: type: object properties: status: type: boolean errors: type: array items: type: string title: UnauthorizedErrorResponse InternalServerErrorResponse: type: object properties: status: type: boolean errors: type: array items: type: string title: InternalServerErrorResponse securitySchemes: BearerAuth: type: http scheme: bearer description: >- API key from the console ApiKey collection, sent as Bearer token. Also accepts session cookies for browser-based auth. ``` ## Examples **Request** ```json {} ``` **Response** ```json { "status": true, "data": [ { "_id": "63b1f9a2e4b0c12d4f8a9e7c", "url": "https://www.example.com/articles/ai-trends-2024", "status": "scraping", "createdAt": "2024-01-15T09:30:00Z", "updatedAt": "2024-01-15T10:00:00Z" }, { "_id": "63b1f9a2e4b0c12d4f8a9e7d", "url": "https://www.example.com/blog/knowledge-base-updates", "status": "indexed", "createdAt": "2024-01-14T14:20:00Z", "updatedAt": "2024-01-15T08:45:00Z" }, { "_id": "63b1f9a2e4b0c12d4f8a9e7e", "url": "https://www.example.com/resources/scrape-failures", "status": "failed", "createdAt": "2024-01-13T11:00:00Z", "updatedAt": "2024-01-13T11:30:00Z" } ] } ``` **SDK Code** ```python import requests url = "https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls" payload = {} headers = { "Authorization": "Bearer ", "Content-Type": "application/json" } response = requests.get(url, json=payload, headers=headers) print(response.json()) ``` ```javascript const url = 'https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls'; const options = { method: 'GET', headers: {Authorization: 'Bearer ', 'Content-Type': 'application/json'}, body: '{}' }; try { const response = await fetch(url, options); const data = await response.json(); console.log(data); } catch (error) { console.error(error); } ``` ```go package main import ( "fmt" "strings" "net/http" "io" ) func main() { url := "https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls" payload := strings.NewReader("{}") req, _ := http.NewRequest("GET", url, payload) req.Header.Add("Authorization", "Bearer ") req.Header.Add("Content-Type", "application/json") res, _ := http.DefaultClient.Do(req) defer res.Body.Close() body, _ := io.ReadAll(res.Body) fmt.Println(res) fmt.Println(string(body)) } ``` ```ruby require 'uri' require 'net/http' url = URI("https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls") http = Net::HTTP.new(url.host, url.port) http.use_ssl = true request = Net::HTTP::Get.new(url) request["Authorization"] = 'Bearer ' request["Content-Type"] = 'application/json' request.body = "{}" response = http.request(request) puts response.read_body ``` ```java import com.mashape.unirest.http.HttpResponse; import com.mashape.unirest.http.Unirest; HttpResponse response = Unirest.get("https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls") .header("Authorization", "Bearer ") .header("Content-Type", "application/json") .body("{}") .asString(); ``` ```php request('GET', 'https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls', [ 'body' => '{}', 'headers' => [ 'Authorization' => 'Bearer ', 'Content-Type' => 'application/json', ], ]); echo $response->getBody(); ``` ```csharp using RestSharp; var client = new RestClient("https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls"); var request = new RestRequest(Method.GET); request.AddHeader("Authorization", "Bearer "); request.AddHeader("Content-Type", "application/json"); request.AddParameter("application/json", "{}", ParameterType.RequestBody); IRestResponse response = client.Execute(request); ``` ```swift import Foundation let headers = [ "Authorization": "Bearer ", "Content-Type": "application/json" ] let parameters = [] as [String : Any] let postData = JSONSerialization.data(withJSONObject: parameters, options: []) let request = NSMutableURLRequest(url: NSURL(string: "https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls")! as URL, cachePolicy: .useProtocolCachePolicy, timeoutInterval: 10.0) request.httpMethod = "GET" request.allHTTPHeaderFields = headers request.httpBody = postData as Data let session = URLSession.shared let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in if (error != nil) { print(error as Any) } else { let httpResponse = response as? HTTPURLResponse print(httpResponse) } }) dataTask.resume() ```