Web Scraper IDE API
Trigger a scraper for batch collection method
API Endpoint: POST
/dca/trigger?collector={ID_COLLECTOR}
POST
Body
Authorization
string
requiredraw JSON -d '[{"url":"https://targetwebsite.com/product_id/"}]'
Query Parameters
collector
string
requiredA unique identification of scraper
version
string
Set to dev
to trigger the development version of the scraper
name
string
human_name
- A human readable name for the batch
Choose trigger_behavior
parameters:
queue_next
int
default: "1"If there’s already something in the crawl queue, push this job into the queue
queue
string
Send another batch of requests that will start after the last one is finished
confirm_cancel
int
default: "1"Cancel running job and run instead, submit the request and cancel running one
no_downloads
int
default: "1"Disable media file download
deadline
string
Set job time deadline, job will be terminated after specified time
- h : hours
- m : minutes
- s : seconds
Headers
Authorization
string
requiredAuthorization Bearer Key -H "Authorization: Bearer API_TOKEN"
Content-Type
string
requiredJSON/CSV/Multipart -H "Content-Type: application/json"
Was this page helpful?