Skip to main content
This guide shows you how to use two advanced delivery features for scrapers: streamed delivery for large snapshots, and file delivery to retrieve raw page files with your data.
Prerequisites:
  • A Bright Data account with an active scraper
  • Delivery method set to Storage or Webhook (both features require this)

​
Streamed delivery

When a snapshot is large, streamed delivery lets you receive results immediately as they’re collected, in batches, rather than waiting for the full snapshot to complete.

​
How to enable streamed delivery

Option 1: Control Panel
  1. Open your scraper’s Delivery settings tab.
  2. Toggle Stream results on.
  3. Enter how many data lines each batch should contain.
Stream results toggle in Delivery settings
Option 2: WSAPI parameter Add &stream_max_lines=10 to your WSAPI request: 
curl "https://api.brightdata.com/datasets/v3/trigger?dataset_id=<DATASET_ID>&stream_max_lines=10" \
  -H "Authorization: Bearer API_TOKEN"

​
Limits

SettingValue
Minimum batch size10 lines
Maximum batch size100,000 lines
Streamed delivery requires a Storage or Webhook delivery method. It is not compatible with API download.

​
File delivery

File delivery lets you retrieve raw page files (HTML snapshots, WARC archives, or screenshots) alongside your scraped data.

​
How to enable file delivery

Add &download_fields= to your WSAPI request with one or more of the available file types: 
curl "https://api.brightdata.com/datasets/v3/trigger?dataset_id=<DATASET_ID>&download_fields=html" \
  -H "Authorization: Bearer API_TOKEN"
To request multiple file types, pass them as a comma-separated list: 
curl "https://api.brightdata.com/datasets/v3/trigger?dataset_id=<DATASET_ID>&download_fields=html,screenshot" \
  -H "Authorization: Bearer API_TOKEN"

​
Available file types

TypeAvailabilityDescription
htmlAlways availableRaw HTML of the scraped page
warcNot always availableFull WARC archive including request and response
screenshotNot always availableScreenshot of the page at time of scraping
File delivery works only when the delivery method is set to Storage or Webhook.

Delivery options

Configure your storage destination and output format

API reference

Full API parameters for triggering and managing snapshots