> ## Documentation Index
> Fetch the complete documentation index at: https://docs.brightdata.com/llms.txt
> Use this file to discover all available pages before exploring further.
# How to deliver Scraper API results
> Deliver Bright Data Scraper API results to a webhook, Amazon S3, Google Cloud Storage or Snowflake. Destination table, IAM setup and the 14 webhook IPs.
This guide shows how to have Bright Data Scraper API jobs push results to your infrastructure instead of waiting for you to download them. Add an `endpoint` parameter for webhook delivery, or configure a cloud destination once and have every job write to it. The setup is identical for all scrapers; only the `dataset_id` changes.
The examples use the LinkedIn profiles scraper (`gd_l1viktl72bvl7bjuj0`). Swap in the `dataset_id` for your platform from the [async requests table](/products/scrapers/scrapers-library/async-requests#which-dataset-id-do-i-use).
## Prerequisites
* A [Bright Data account](https://brightdata.com/?hs_signup=1\&utm_source=docs) with an active API key
* Familiarity with the [async request workflow](/products/scrapers/scrapers-library/async-requests)
* For webhooks, a publicly accessible HTTPS endpoint (or a testing tool like [webhook.site](https://webhook.site))
* For a cloud destination, permission to create credentials in that account: an IAM role for Amazon S3, a service account key for Google Cloud, a user and role for Snowflake
## Which destination should I use?
Set a cloud destination once on the scraper's **Delivery settings** tab in the Control Panel. Set a webhook per request with the `endpoint` parameter. Both can be active for the same job.
| Destination | Best for | Setup |
| :---------------------- | :------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------- |
| API download | Testing and small jobs | Nothing. Fetch the snapshot with [Download snapshot](/api-reference/scrapers/delivery-apis/download-snapshot) once it is ready |
| Webhook | Event-driven pipelines, payloads under 1 GB | The `endpoint` query parameter on the trigger request. See [webhook delivery](#how-to-deliver-results-to-a-webhook) |
| Amazon S3 | Large payloads, data lakes, retained history | A one-time IAM policy and role. See [S3 delivery](#how-to-deliver-results-to-amazon-s3) |
| Google Cloud Storage | Google Cloud data platforms | A one-time service account key. See [Google Cloud Storage delivery](#how-to-deliver-results-to-google-cloud-storage) |
| Snowflake | Loading straight into a warehouse | A one-time database, stage, role and user. See [Snowflake delivery](#how-to-deliver-results-to-snowflake) |
| Google Cloud Pub/Sub | Message-driven pipelines | The topic and credentials in the Control Panel |
| Microsoft Azure Storage | Azure data platforms | The container and credentials in the Control Panel |
| SFTP or FTP | Servers without a cloud API | The host, path and credentials in the Control Panel |
The file format is `json`, `ndjson` (also labelled JSON lines) or `csv`.
Two options change what a delivery contains. Streamed delivery sends results in batches of 10 to 100,000 lines while the job is still running, and file delivery adds the raw HTML, WARC or screenshot of each page next to the records. Both need a storage or webhook destination, not API download. See [How to stream results and deliver files](/products/scrapers/scrapers-library/stream-and-file-delivery).
## How to deliver results to a webhook
When you trigger an async collection with an `endpoint` URL, Bright Data sends a `POST` request to that URL with the scraped data once the job completes. No polling required.
```text theme={null}
Your app --> POST /trigger (with webhook URL) --> Bright Data scrapes --> POST to your webhook
```
### Step 1: Set up a test webhook
For testing, use [webhook.site](https://webhook.site) to get a temporary public URL:
1. Open [webhook.site](https://webhook.site) in your browser
2. Copy the unique URL displayed (e.g., `https://webhook.site/abc-123-def`)
3. Keep the page open to monitor incoming requests
### Step 2: Trigger a collection with the webhook URL
Add the `endpoint` query parameter to your async `/trigger` request:
```bash cURL theme={null}
curl -X POST \
"https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l1viktl72bvl7bjuj0&format=json&endpoint=https://webhook.site/abc-123-def&uncompressed_webhook=true" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '[
{"url": "https://www.linkedin.com/in/satyanadella"},
{"url": "https://www.linkedin.com/in/jeffweiner08"}
]'
```
```python Python theme={null}
import requests
WEBHOOK_URL = "https://webhook.site/abc-123-def"
response = requests.post(
"https://api.brightdata.com/datasets/v3/trigger",
params={
"dataset_id": "gd_l1viktl72bvl7bjuj0",
"format": "json",
"endpoint": WEBHOOK_URL,
"uncompressed_webhook": "true",
},
headers={
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
json=[
{"url": "https://www.linkedin.com/in/satyanadella"},
{"url": "https://www.linkedin.com/in/jeffweiner08"},
],
)
print("Snapshot ID:", response.json()["snapshot_id"])
```
```javascript Node.js theme={null}
const WEBHOOK_URL = "https://webhook.site/abc-123-def";
const response = await fetch(
`https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l1viktl72bvl7bjuj0&format=json&endpoint=${encodeURIComponent(WEBHOOK_URL)}&uncompressed_webhook=true`,
{
method: "POST",
headers: {
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
body: JSON.stringify([
{ url: "https://www.linkedin.com/in/satyanadella" },
{ url: "https://www.linkedin.com/in/jeffweiner08" },
]),
}
);
const data = await response.json();
console.log("Snapshot ID:", data.snapshot_id);
```
Key parameters:
| Parameter | Description |
| :--------------------- | :----------------------------------------------------------- |
| `endpoint` | Your HTTP endpoint URL that receives the `POST` payload |
| `uncompressed_webhook` | Set to `true` to receive uncompressed JSON (default is gzip) |
| `format` | Output format: `json`, `ndjson`, or `csv` |
### Step 3: Verify delivery
Once the collection completes (typically 30-60 seconds for a few records), check your webhook.site page. You should see a `POST` request with the scraped data.
The payload is the same JSON array you would receive from a direct API download:
```json theme={null}
[
{
"name": "Satya Nadella",
"city": "Redmond",
"country_code": "US",
"current_company": { "name": "Microsoft" },
"followers": 10842560
},
{
"name": "Jeff Weiner",
"city": "San Francisco Bay Area",
"country_code": "US",
"current_company": { "name": "Next Chapter" },
"followers": 1200000
}
]
```
### How to handle webhooks in production
For production, point the `endpoint` URL to your own server endpoint.
```javascript server.js theme={null}
const express = require("express");
const app = express();
app.use(express.json({ limit: "100mb" }));
app.post("/webhook/scraper", (req, res) => {
const records = req.body;
console.log(`Received ${records.length} records`);
for (const record of records) {
console.log(`- ${record.name}`);
}
res.status(200).json({ received: true });
});
app.listen(3000, () => console.log("Webhook server running on port 3000"));
```
```python server.py theme={null}
from flask import Flask, request, jsonify
app = Flask(__name__)
@app.route("/webhook/scraper", methods=["POST"])
def handle_webhook():
records = request.get_json()
print(f"Received {len(records)} records")
for record in records:
print(f"- {record['name']}")
return jsonify({"received": True}), 200
if __name__ == "__main__":
app.run(port=3000)
```
Return a `200` status code within 30 seconds to acknowledge receipt. If your endpoint fails or times out, Bright Data retries delivery.
### How to authenticate webhook deliveries
If your endpoint requires authentication, add the `auth_header` parameter. Bright Data sends its value as the `Authorization` header on every delivery to that endpoint. Verified on September 7, 2026. Earlier versions of this page named the parameter `webhook_header_Authorization`; the API ignores that name and delivers with no `Authorization` header, so replace it if you copied it.
```bash theme={null}
curl -X POST \
"https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l1viktl72bvl7bjuj0&format=json&endpoint=https://your-server.com/webhook&auth_header=Bearer+YOUR_SECRET_TOKEN" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '[{"url": "https://www.linkedin.com/in/satyanadella"}]'
```
### Which headers does a delivery carry?
Every webhook delivery is a `POST` with `Content-Type: application/json; charset=utf-8`, a `User-Agent` starting with `BRD dca-stage-deliver`, a `dca-collection-id` header holding the snapshot ID and a `dca-filename` header naming the file. Match deliveries to jobs with `dca-collection-id` rather than parsing the body. Observed on September 7, 2026.
### Which IPs do webhooks come from?
If your server uses an IP allowlist, add the following 14 Bright Data webhook source IPs. Every asynchronous webhook delivery from Bright Data, across the Scraper API, the AI scrapers and the SERP API, comes from one of them.
```text theme={null}
54.175.27.69
34.225.9.175
100.28.38.247
100.29.18.195
52.72.185.255
35.174.112.248
54.165.183.124
3.91.140.7
52.202.75.37
98.82.225.117
100.27.150.189
18.214.10.85
35.169.71.210
44.194.183.74
```
## How to deliver results to Amazon S3
S3 delivery is configured once per scraper in the Control Panel. After setup, every job writes its results to your bucket automatically.
### Step 1: Create the IAM policy and role
Bright Data writes to your bucket by assuming an IAM role in your AWS account. Create a policy that allows the writes, a role that Bright Data's delivery role can assume, and attach the role to the bucket.
In the IAM console, open **Policies** and create a policy that allows `PutObject` and `GetObject` on the bucket.
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:PutObject", "s3:GetObject"],
"Resource": "arn:aws:s3:::NAME-OF-YOUR-BUCKET/*"
}
]
}
```
Open **Roles**, create a role, and attach the policy from step 1 under **Permission policies**. Note the role's ARN; it looks like `arn:aws:iam:::role/` and goes into the delivery settings.
The role's trust policy lets Bright Data's delivery role assume it. The external ID in the condition is the value you enter in the delivery settings:
```json theme={null}
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::422310177405:role/brd.ec2.zs-dca-delivery"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "YOUR EXTERNAL ID"
}
}
}
]
}
```
On the bucket, grant the role from step 2 access by its ARN.
### Step 2: Configure the delivery destination
1. Navigate to your [scraper configuration](https://brightdata.com/cp/scrapers)
2. Click the **Delivery settings** tab
3. Select **Amazon S3** as the delivery destination
4. Enter your credentials:
* **Bucket name**: Your S3 bucket name
* **Role ARN**: The IAM role ARN from Step 1
* **External ID**: The value from the role's trust policy
* **Region**: Your S3 bucket region
* **Path prefix** (optional): A folder path within the bucket (e.g., `linkedin/profiles/`)
5. Select your preferred file format (JSON, NDJSON, or CSV)
6. Click **Save**
### Step 3: Trigger a collection
Trigger an async collection as normal. Results are delivered to your S3 bucket without any extra parameters:
```bash theme={null}
curl -X POST \
"https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l1viktl72bvl7bjuj0&format=json" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '[
{"url": "https://www.linkedin.com/in/satyanadella"},
{"url": "https://www.linkedin.com/in/jeffweiner08"},
{"url": "https://www.linkedin.com/in/rbranson"}
]'
```
### Step 4: Verify delivery
Once the collection completes, check your S3 bucket for the delivered file:
```bash theme={null}
aws s3 ls s3://your-bucket-name/linkedin/profiles/
```
You should see a file named with the snapshot ID (e.g., `sd_m1a2b3c4d5e6f7g8h.json`). Download and inspect it:
```bash theme={null}
aws s3 cp s3://your-bucket-name/linkedin/profiles/sd_m1a2b3c4d5e6f7g8h.json ./results.json
cat results.json | python -m json.tool | head -20
```
You can also verify delivery status using the [Monitor Delivery API](/api-reference/scrapers/management-apis/monitor-delivery).
## How to deliver results to Google Cloud Storage
Google Cloud Storage delivery authenticates with a service account key. Create the key, then paste its private key into the scraper's delivery settings.
1. Open the [Google Cloud console](https://console.cloud.google.com/), expand the menu and open **IAM & Admin**.
2. Click **Service accounts**.
3. Choose an existing service account or create one. If the button is missing, create a project first.
4. Enter the name, ID and description, grant the account access to the bucket, and create it.
5. Click the service account's email address.
6. Open the **Keys** tab, click **Add key**, then **Create new key**.
7. Choose **JSON**. The key file downloads to your machine.
8. Copy the `private_key` value from the downloaded file into the scraper's delivery settings, with the bucket name and the file format.
Trigger a collection as normal. Results land in the bucket without extra parameters.
## How to deliver results to Snowflake
Bright Data loads data through an internal named stage, using a user and role you create for it. Run the following in your Snowflake account, then enter the values in the scraper's delivery settings.
```sql theme={null}
CREATE DATABASE ;
```
Every database has a `PUBLIC` schema. To use another one:
```sql theme={null}
CREATE SCHEMA ;
```
Snowflake's guidance on sizing a warehouse for data loading applies; see [Warehouses overview](https://docs.snowflake.com/en/user-guide/warehouses-overview#impact-on-data-loading).
```sql theme={null}
CREATE WAREHOUSE ;
```
```sql theme={null}
CREATE STAGE ;
```
```sql theme={null}
CREATE ROLE ;
GRANT OPERATE ON WAREHOUSE TO ROLE ;
GRANT WRITE ON STAGE TO ROLE ;
```
```sql theme={null}
CREATE USER
PASSWORD = ''
LOGIN_NAME =
MUST_CHANGE_PASSWORD = FALSE
DISABLED = FALSE
COMMENT = 'user for Bright Data to upload data directly into Snowflake';
GRANT ROLE TO USER ;
```
If a [network policy](https://docs.snowflake.com/en/user-guide/network-policies) is active on the account, add these four addresses to it:
```sql theme={null}
ALTER NETWORK POLICY
SET ALLOWED_IP_LIST=(
,
'35.169.71.210',
'34.233.211.38',
'44.194.183.74',
'54.243.177.151');
```
Enter the database, schema, warehouse, stage, user and password in the scraper's delivery settings. For help with the setup, contact [support](mailto:support@brightdata.com).
## Troubleshooting
### Webhook not receiving data?
* Verify the URL is publicly accessible (not `localhost`)
* Check that your endpoint returns a `200` status code within 30 seconds
* Verify the 14 webhook IPs above are allowlisted if you have firewall rules
### Receiving compressed data?
If you omit `uncompressed_webhook=true`, data arrives gzip-compressed. Add `uncompressed_webhook=true` to your trigger URL, or decompress the payload on your server.
### Payload too large for your server?
Large collections can produce payloads up to 1 GB. Set `express.json({ limit: "100mb" })` in Express.js or equivalent in your framework. For very large datasets, use S3 delivery instead.
### Files not appearing in S3?
* Verify the IAM role ARN and external ID are correct
* Check that the bucket policy allows `s3:PutObject` from Bright Data's account
* Ensure the bucket region matches your configuration
* Review delivery status in the Bright Data dashboard under **Logs**
### Access denied errors on S3?
Verify the trust policy on your IAM role names Bright Data's delivery role in account `422310177405` and that the external ID in the trust policy matches the one in the delivery settings.
## FAQ
### Does delivery configuration differ per platform?
No. Webhook parameters and cloud destination setup are identical for every Bright Data scraper. Only the `dataset_id` in the trigger request changes.
### Can I use a webhook and cloud storage at the same time?
Yes. Cloud delivery is configured per scraper in the Control Panel, and the `endpoint` parameter is set per request, so a single job can do both.
### What happens if my webhook is down when a job finishes?
Bright Data retries delivery. If your endpoint stays unavailable, download the snapshot directly using the `snapshot_id` returned by the trigger call; snapshots stay available for 30 days.
## Related
* [How to scrape in bulk with async requests](/products/scrapers/scrapers-library/async-requests)
* [How to stream results and deliver files](/products/scrapers/scrapers-library/stream-and-file-delivery)