Webhook callbacks

Webhooks push status updates during async tasks so you can track progress in real time.

XCrawl currently supports webhook on:

POST /v1/scrape
POST /v1/crawl

Configuration reference

{
  "webhook": {
    "url": "https://your-domain.com/xcrawl/webhook",
    "headers": {
      "X-Webhook-Secret": "YOUR_SHARED_SECRET"
    },
    "events": [
      "started",
      "completed",
      "failed"
    ]
  }
}

Fields:

Field	Type	Required	Default	Description
`url`	string	No	-	Callback URL
`headers`	object map	No	-	Custom headers for callback
`events`	string[]	No	`["started","completed","failed"]`	Events to send

events options:

started: task started
completed: task completed successfully
failed: task failed

Callback payload

Webhook payloads follow the same response envelope as the corresponding async result APIs:

Scrape task events align with Scrape Result API
Crawl task events align with Crawl Result API
started events usually include task identifiers and status only
completed events include the final data payload
failed events include identifiers, status, and timestamps; data may be omitted