Synchronous requests

curl -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input":[{"user_name":"zoobarcelona"}],"limit_per_input":null}' \
  "https://api.brightdata.com/datasets/v3/scrape?dataset_id=gd_l1vikfch901nx3by4&notify=false&include_errors=true&type=discover_new&discover_by=user_name"

import requests

response = requests.post(
    "https://api.brightdata.com/datasets/v3/scrape",
    params={
        "dataset_id": "gd_l1vikfch901nx3by4",
        "notify": "false",
        "include_errors": "true",
        "type": "discover_new",
        "discover_by": "user_name",
    },
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={"input": [{"user_name": "zoobarcelona"}], "limit_per_input": None},
)

print(response.status_code, response.json())

import axios from "axios";

const response = await axios.post(
  "https://api.brightdata.com/datasets/v3/scrape",
  { input: [{ user_name: "zoobarcelona" }], limit_per_input: null },
  {
    params: {
      dataset_id: "gd_l1vikfch901nx3by4",
      notify: false,
      include_errors: true,
      type: "discover_new",
      discover_by: "user_name",
    },
    headers: { Authorization: "Bearer YOUR_API_KEY" },
  }
);

console.log(response.status, response.data);

OK

{
  "snapshot_id": "sd_m1a2b3c4d5e6f7g8h",
  "message": "Your request is still in progress and cannot be retrieved in this call. Use the provided Snapshot ID to track progress via the Monitor Snapshot endpoint and download it once ready via the Download Snapshot endpoint."
}

{
  "error": "Invalid input provided",
  "code": "validation_error",
  "type": "validation",
  "line": "{\"url\":\"not-a-url\",\"asin\":\"\",\"zipcode\":\"\",\"language\":\"\"}",
  "index": 1,
  "errors": [
    [
      "url",
      "This field must be a valid url"
    ]
  ]
}

Invalid credentials

dataset does not exist

POST

datasets

scrape

curl -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input":[{"user_name":"zoobarcelona"}],"limit_per_input":null}' \
  "https://api.brightdata.com/datasets/v3/scrape?dataset_id=gd_l1vikfch901nx3by4&notify=false&include_errors=true&type=discover_new&discover_by=user_name"

import requests

response = requests.post(
    "https://api.brightdata.com/datasets/v3/scrape",
    params={
        "dataset_id": "gd_l1vikfch901nx3by4",
        "notify": "false",
        "include_errors": "true",
        "type": "discover_new",
        "discover_by": "user_name",
    },
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={"input": [{"user_name": "zoobarcelona"}], "limit_per_input": None},
)

print(response.status_code, response.json())

import axios from "axios";

const response = await axios.post(
  "https://api.brightdata.com/datasets/v3/scrape",
  { input: [{ user_name: "zoobarcelona" }], limit_per_input: null },
  {
    params: {
      dataset_id: "gd_l1vikfch901nx3by4",
      notify: false,
      include_errors: true,
      type: "discover_new",
      discover_by: "user_name",
    },
    headers: { Authorization: "Bearer YOUR_API_KEY" },
  }
);

console.log(response.status, response.data);

OK

{
  "snapshot_id": "sd_m1a2b3c4d5e6f7g8h",
  "message": "Your request is still in progress and cannot be retrieved in this call. Use the provided Snapshot ID to track progress via the Monitor Snapshot endpoint and download it once ready via the Download Snapshot endpoint."
}

{
  "error": "Invalid input provided",
  "code": "validation_error",
  "type": "validation",
  "line": "{\"url\":\"not-a-url\",\"asin\":\"\",\"zipcode\":\"\",\"language\":\"\"}",
  "index": 1,
  "errors": [
    [
      "url",
      "This field must be a valid url"
    ]
  ]
}

Invalid credentials

dataset does not exist

curl -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input":[{"user_name":"zoobarcelona"}],"limit_per_input":null}' \
  "https://api.brightdata.com/datasets/v3/scrape?dataset_id=gd_l1vikfch901nx3by4&notify=false&include_errors=true&type=discover_new&discover_by=user_name"

import requests

response = requests.post(
    "https://api.brightdata.com/datasets/v3/scrape",
    params={
        "dataset_id": "gd_l1vikfch901nx3by4",
        "notify": "false",
        "include_errors": "true",
        "type": "discover_new",
        "discover_by": "user_name",
    },
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={"input": [{"user_name": "zoobarcelona"}], "limit_per_input": None},
)

print(response.status_code, response.json())

import axios from "axios";

const response = await axios.post(
  "https://api.brightdata.com/datasets/v3/scrape",
  { input: [{ user_name: "zoobarcelona" }], limit_per_input: null },
  {
    params: {
      dataset_id: "gd_l1vikfch901nx3by4",
      notify: false,
      include_errors: true,
      type: "discover_new",
      discover_by: "user_name",
    },
    headers: { Authorization: "Bearer YOUR_API_KEY" },
  }
);

console.log(response.status, response.data);

How It Works

This synchronous API endpoint allows users to send a scraping request and receive the results in real-time directly in the response, at the point of request - such as a terminal or application - without the need for external storage or manual downloads. This approach streamlines the data collection process by eliminating additional steps for retrieving results. Set the output format with the format query parameter. If no format is provided, the response is NDJSON, one record per line. Set format=json to get a JSON array.

Request body

Send the inputs as an object with an input array:

{
  "input": [{"user_name": "zoobarcelona"}],
  "limit_per_input": null
}

Field	Type	Description
`input`	array of objects, required	The inputs to scrape. The fields depend on the scraper, for example `url` or `user_name`
`limit_per_input`	integer or `null`	Maximum records returned per input, or `null`
`custom_output_fields`	string	Output fields to keep, separated by `\|`, for example `title\|final_price\|url`

limit_per_input matters for discovery requests, whose result count is open-ended. This request discovers posts in a subreddit and stops at three:

{
  "input": [{"url": "https://www.reddit.com/r/learnpython/"}],
  "limit_per_input": 3
}

Send it with type=discover_new&discover_by=subreddit_url on the query string. Put limit_per_input in the body, next to input. A limit_per_input query parameter also caps the results (verified September 30, 2026), but the body is the recommended place. A bare JSON array of input objects is also accepted and returns the same records. It cannot carry limit_per_input or custom_output_fields:

[{"url": "https://www.amazon.com/dp/B0FQFB8FMG"}]

POST /datasets/v3/trigger takes the same object body. On that endpoint the object is also where a deliver block goes. See Asynchronous requests.

Timeout Limit

Please note that this synchronous request is subject to a 1 minute timeout limit. If the data retrieval process exceeds this limit, the API will return an HTTP 202 response, indicating that the request is still being processed. In such cases, you will receive a snapshot ID to monitor and retrieve the results asynchronously via the Monitor Snapshot and Download Snapshot endpoints. The 202 response carries a retry-after header (10 seconds at the time of writing) telling you how long to wait before polling. Example response on timeout:

202

{
  "snapshot_id": "sd_m1a2b3c4d5e6f7g8h",
  "message": "Your request is still in progress and cannot be retrieved in this call. Use the provided Snapshot ID to track progress via the Monitor Snapshot endpoint and download it once ready via the Download Snapshot endpoint."
}

How to handle a 202 response

A /scrape request that runs past 1 minute answers HTTP 202 with a snapshot_id instead of records, and the job continues. Branch on the status code, then poll Monitor progress and fetch the records with Download snapshot. The polling loop is in How to scrape in bulk with async requests.

Python

import requests

response = requests.post(
    "https://api.brightdata.com/datasets/v3/scrape",
    params={"dataset_id": "gd_l7q7dkf244hwjntr0", "format": "json"},
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={"input": [{"url": "https://www.amazon.com/dp/B0FQFB8FMG"}]},
)

if response.status_code == 200:
    print(f"Got {len(response.json())} results")
elif response.status_code == 202:
    print(f"Job running. Snapshot ID: {response.json()['snapshot_id']}")
else:
    response.raise_for_status()

A 200 carries the records; a 202 carries the snapshot ID. Any other status is an error, listed under Response below. The fix for each error is on Scraper API error codes.

Custom inputs

You can add custom fields to the input schema. Whatever you send in those fields is returned in the results for each record. Use this to:

Keep a unified output structure across different scrapers and datasets.
Pass an id, row_index or any internal key so you can match results back to your original input rows.

Authorizations

Authorization

string

header

required

Use your Bright Data API Key as a Bearer token in the Authorization header.

How to authenticate:

Obtain your API Key from the Bright Data account settings at https://brightdata.com/cp/setting/users
Include the API Key in the Authorization header of your requests
Format: Authorization: Bearer YOUR_API_KEY

Example:

Authorization: Bearer YOUR_API_KEY

Learn how to get your Bright Data API key: https://docs.brightdata.com/api-reference/authentication

Query Parameters

dataset_id

string

required

Dataset ID for which data collection is triggered.

custom_output_fields

string

List of output columns, separated by | (e.g., url|about.updated_on). Filters the response to include only the specified fields.

Example:

"url|about.updated_on"

include_errors

boolean

Include errors report with the results.

format

enum<string>

default:ndjson

Format of the response: ndjson (default, one record per line), json (a JSON array) or csv. See output formats.

Available options:

ndjson,

json,

csv

Body

application/json

Inputs with output filter · object
Only inputs · object[]

input

object[]

required

Array of input objects. The fields depend on the scraper, for example url or user_name.

Show child attributes

custom_output_fields

string

List of output columns, separated by | (e.g., url|about.updated_on). Filters the response to include only the specified fields.

Example:

"url|about.updated_on"

limit_per_input

integer | null

Maximum number of records to return per input, or null. Applies to discovery requests, whose result count is otherwise open-ended.

Required range: x >= 1

Example:

null

Response

A JSON array of records. An empty array means the inputs produced no records; check the input URLs.

The response is of type string.

Example:

"OK"

Essentials

Data extraction

Web search

AI training data

Proxy

Administrative APIs

How It Works

Request body

Timeout Limit

How to handle a 202 response

Custom inputs

Authorizations

Query Parameters

Body

Response

​How It Works

​Request body

​Timeout Limit

​How to handle a 202 response

​Custom inputs

Authorizations

Query Parameters

Body

Response

How It Works

Request body

Timeout Limit

How to handle a 202 response

Custom inputs