How to stream results and deliver files

This guide shows you how to use two advanced delivery features for scrapers: streamed delivery for large snapshots and file delivery to retrieve raw page files with your data.

Prerequisites:

A Bright Data account with an active scraper
Delivery method set to Storage or Webhook (both features require this)

How to stream delivery in batches

When a snapshot is large, streamed delivery lets you receive results immediately as they’re collected, in batches, rather than waiting for the full snapshot to complete.

Enable streamed delivery

Control panel

Open your scraper’s Delivery settings tab.
Toggle Stream results on.
Enter how many data lines each batch should contain.

Stream results toggle in Delivery settings

Scraper API parameter

Add &stream_max_lines=1000 to your Scraper API request:

curl -X POST \
  "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l7q7dkf244hwjntr0&stream_max_lines=1000" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[{"url": "https://www.amazon.com/dp/B0FQFB8FMG"}]'

Limits

Setting	Value
Minimum batch size	10 lines
Maximum batch size	100,000 lines

Streamed delivery requires a Storage or Webhook delivery method. It is not compatible with API download.

How to deliver raw page files

File delivery lets you retrieve raw page files (HTML snapshots, WARC archives, or screenshots) alongside your scraped data.

How to enable file delivery

Add &download_fields= to your Scraper API request with one or more of the available file types:

curl -X POST \
  "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l7q7dkf244hwjntr0&download_fields=html" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[{"url": "https://www.amazon.com/dp/B0FQFB8FMG"}]'

To request multiple file types, pass them as a comma-separated list:

curl -X POST \
  "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l7q7dkf244hwjntr0&download_fields=html,screenshot" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[{"url": "https://www.amazon.com/dp/B0FQFB8FMG"}]'

Available file types

Type	Availability	Description
`html`	Always available	Raw HTML of the scraped page
`warc`	Not always available	Full WARC archive including request and response
`screenshot`	Not always available	Screenshot of the page at time of scraping

File delivery works only when the delivery method is set to Storage or Webhook.

Deliver results

Configure your storage destination and output format

API reference

Full API parameters for triggering and managing snapshots

Get started

Data extraction

AI training data

Proxy networks

How to stream results and deliver files

How to stream delivery in batches

Enable streamed delivery

Control panel

Scraper API parameter

Limits

How to deliver raw page files

How to enable file delivery

Available file types

Deliver results

API reference

​How to stream delivery in batches

​Enable streamed delivery

​Control panel

​Scraper API parameter

​Limits

​How to deliver raw page files

​How to enable file delivery

​Available file types

​Related

Deliver results

API reference

How to stream delivery in batches

Enable streamed delivery

Control panel

Scraper API parameter

Limits

How to deliver raw page files

How to enable file delivery

Available file types

Related