Sync /scrape | Response wait | 1 minute | The API answers HTTP 202 with a snapshot_id and the job continues. The retry-after header says how long to wait before polling, 10 seconds at the time of writing. See How to handle a 202 response |
Async /trigger | Input data per job | 1 GB | About tens of thousands of URLs. Split larger inputs across several jobs |
Async /trigger | Minimum inputs | Set per scraper | Some scrapers need more than one input. A request with too few returns HTTP 400 Should be at least LIMIT inputs |
/trigger and /scrape | Running jobs | 5,000 active jobs or snapshots | New requests return HTTP 429 You have too many running jobs for this dataset |
/trigger and /scrape | HTTP 429 responses per IP | 25 within 5 minutes | Bright Data blocks the IP on every API request until support clears it |
| Webhook delivery | Payload size | 1 GB | Use storage delivery for larger results |
| Webhook delivery | Response deadline | 30 seconds | Bright Data treats the delivery as failed and retries it. Return HTTP 200 before processing the payload. See How to handle webhooks in production |
| Download snapshot | Snapshot retention | 30 days after the job completes | The snapshot can no longer be downloaded. Deliver results to storage or a webhook to keep them longer |
| Download snapshot | Download size per request | 5 GB | Download the snapshot in parts with batch_size and part |
| Download snapshot | Minimum batch_size for parts | 1,000 records | Use a batch_size of 1,000 or more |
| Deliver snapshot | Delivered file size | 5 GB per file | HTTP 400. Lower batch_size so each file stays under 5 GB |
| Streamed delivery | Lines per batch | 10 to 100,000 | Set stream_max_lines within this range. Streamed delivery needs a storage or webhook destination |
| Instagram | Media link lifetime | 24 hours after collection | Media URLs in the records stop working. Download the media within 24 hours |
| LinkedIn | Posts per profile | The posts a profile shows publicly, typically about 10 | Older posts are not collected |
