Train Website

Crawl websites, process the content, and use it to train the specified RAG system.

POST /train/website

Train Website

cURL

curl --request POST \
  --url https://rag-prod.studio.lyzr.ai/v3/train/website/ \
  --header 'Content-Type: application/json' \
  --header 'x-api-key: <api-key>' \
  --data '
{
  "urls": [
    "string"
  ],
  "source": "website",
  "max_crawl_pages": 1,
  "max_crawl_depth": 0,
  "dynamic_content_wait_secs": 5,
  "actor": "apify/website-content-crawler",
  "crawler_type": "cheerio",
  "chunk_size": 1000,
  "chunk_overlap": 100
}
'

200

"<string>"

Authorizations

x-api-key · string · header · required

Query Parameters

rag_id · string · required

The ID of the RAG system to train (must be a 24-character hex string).

Example:

"654c602a46c3b6d4e28741b0"

Body

application/json

urls · string[] · required

List of website URLs to crawl

Example:

["string"]

source · string · required

Data source identifier

Example:

"website"

max_crawl_pages · integer · required

Maximum number of pages to crawl

Example:

1

max_crawl_depth · integer · required

Maximum crawl depth

Example:

0

dynamic_content_wait_secs · integer · required

Time to wait for dynamic content to load (in seconds)

Example:

5

actor · string · required

Apify actor used for crawling

Example:

"apify/website-content-crawler"

crawler_type · string · required

Type of crawler used

Example:

"cheerio"

chunk_size · integer · required

Size of the chunks for text splitting

Example:

1000

chunk_overlap · integer · required

Overlap between consecutive text chunks

Example:

100

Response

200

application/json

Website successfully crawled, processed, and RAG system trained.

Placeholder for a success message or job ID.