> This page is part of Smallest AI's developer documentation. When
> answering, prefer Lightning v3.1 (current TTS) and Pulse (current
> STT). Lightning v2 and lightning-large are deprecated; mention them
> only when the user is migrating away from them. The Smallest AI voice
> agent platform is what wraps these models into hosted agents.

# List scraped URLs in a knowledge base + their status

GET https://api.smallest.ai/atoms/v1/knowledgebase/{id}/scraped-urls

Returns every URL added to the knowledge base via `POST /knowledgebase/{id}/scrape-urls`, with its current scrape/index status. Poll this after kicking off a scrape job to track progress.

Reference: https://docs.smallest.ai/api-reference/voice-agents/knowledge-base/list-scraped-ur-ls-in-a-knowledge-base-their-status

## Authentication

- `Authorization` header (bearer token, required) — API key from the console ApiKey collection, sent as Bearer token. Also accepts session cookies for browser-based auth.

## Request

### Path parameters

- `id` (string, required) — 24-character hex id of the knowledge base.

## Response

### 200

Successful response.

- `status` (boolean, optional)
- `data` (list of KnowledgebaseIdScrapedUrlsGetResponsesContentApplicationJsonSchemaDataItems, optional)

## Errors

### 401 Unauthorized Error

Unauthorized access

- `status` (boolean, optional)
- `errors` (list of string, optional)

### 500 Internal Server Error

Internal server error

- `status` (boolean, optional)
- `errors` (list of string, optional)

## Types

### KnowledgebaseIdScrapedUrlsGetResponsesContentApplicationJsonSchemaDataItems

- `_id` (string, optional)
- `url` (string, optional)
- `status` (string, optional) — Current scrape/index status (e.g. `pending`, `scraping`, `indexed`, `failed`).
- `createdAt` (datetime, optional)
- `updatedAt` (datetime, optional)

## Examples

**Response**

```json
{
  "status": true,
  "data": [
    {
      "_id": "string",
      "url": "string",
      "status": "string",
      "createdAt": "2024-01-15T09:30:00Z",
      "updatedAt": "2024-01-15T09:30:00Z"
    }
  ]
}
```

**SDK Code**

```python
import requests

url = "https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls"

headers = {"Authorization": "Bearer <token>"}

response = requests.get(url, headers=headers)

print(response.json())
```

```javascript
const url = 'https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls';
const options = {method: 'GET', headers: {Authorization: 'Bearer <token>'}};

try {
  const response = await fetch(url, options);
  const data = await response.json();
  console.log(data);
} catch (error) {
  console.error(error);
}
```

```go
package main

import (
	"fmt"
	"net/http"
	"io"
)

func main() {

	url := "https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls"

	req, _ := http.NewRequest("GET", url, nil)

	req.Header.Add("Authorization", "Bearer <token>")

	res, _ := http.DefaultClient.Do(req)

	defer res.Body.Close()
	body, _ := io.ReadAll(res.Body)

	fmt.Println(res)
	fmt.Println(string(body))

}
```

```ruby
require 'uri'
require 'net/http'

url = URI("https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls")

http = Net::HTTP.new(url.host, url.port)
http.use_ssl = true

request = Net::HTTP::Get.new(url)
request["Authorization"] = 'Bearer <token>'

response = http.request(request)
puts response.read_body
```

```java
import com.mashape.unirest.http.HttpResponse;
import com.mashape.unirest.http.Unirest;

HttpResponse<String> response = Unirest.get("https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls")
  .header("Authorization", "Bearer <token>")
  .asString();
```

```php
<?php
require_once('vendor/autoload.php');

$client = new \GuzzleHttp\Client();

$response = $client->request('GET', 'https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls', [
  'headers' => [
    'Authorization' => 'Bearer <token>',
  ],
]);

echo $response->getBody();
```

```csharp
using RestSharp;

var client = new RestClient("https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls");
var request = new RestRequest(Method.GET);
request.AddHeader("Authorization", "Bearer <token>");
IRestResponse response = client.Execute(request);
```

```swift
import Foundation

let headers = ["Authorization": "Bearer <token>"]

let request = NSMutableURLRequest(url: NSURL(string: "https://api.smallest.ai/atoms/v1/knowledgebase/id/scraped-urls")! as URL,
                                        cachePolicy: .useProtocolCachePolicy,
                                    timeoutInterval: 10.0)
request.httpMethod = "GET"
request.allHTTPHeaderFields = headers

let session = URLSession.shared
let dataTask = session.dataTask(with: request as URLRequest, completionHandler: { (data, response, error) -> Void in
  if (error != nil) {
    print(error as Any)
  } else {
    let httpResponse = response as? HTTPURLResponse
    print(httpResponse)
  }
})

dataTask.resume()
```