Skip to main content
POST
/
datasets
/
filter
cURL
curl --request POST \
  --url https://api.brightdata.com/datasets/filter \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: multipart/form-data' \
  --form 'filter={
  "name": "name",
  "operator": "=",
  "value": "John"
}'
{
  "snapshot_id": "<string>"
}
Paste your API key into the authorization field. To get an API key, Create an account and learn how to generate an API key.

​
General Description

  • This endpoint filters a dataset and creates a snapshot of the filtered data in your account.
  • The job runs asynchronously and can take up to 5 minutes to complete. If it exceeds this time, it will be cancelled.
  • Charges apply based on the size of the snapshot and per-record pricing.
  • Filters can have up to 3 levels of nested filter groups.
  • You can upload CSV or JSON files for efficient filtering when handling large sets of values.

​
File Format Requirements

  • First line must be a header matching the field name in your filter.
  • Each following line contains a single value.
Example: industries.csv
industries:value
Accounting
Ad Network
Advertising

​
Filter Syntax with File References

When using file uploads, reference the filename in the filter’s value field.
Example
{
  "operator": "and",
  "filters": [
    {
      "name": "industries:value",
      "operator": "includes",
      "value": "industries.csv"
    }
  ]
}

​
Supported Operators for File References

OperatorField TypesDescription
inAnyField value equals any value in file
not_inAnyField value does not equal any value in file
includesArray, TextField value contains any value in file
not_includesArray, TextField value does not contain any value in file
array_includesArrayAny value in file exists in field value
not_array_includesArrayNo values in file exist in field value

​
Example: Filtering with Multiple Files

curl \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: multipart/form-data" \
  -F "files[]=@/path/to/industries.csv" \
  -F "files[]=@/path/to/regions.csv" \
  -F "filter={\"operator\":\"and\",\"filters\":[{\"name\":\"industries:value\",\"operator\":\"includes\",\"value\":\"industries.csv\"},{\"name\":\"region\",\"operator\":\"in\",\"value\":\"regions.csv\"}]}" \
  "api.brightdata.com/datasets/filter?dataset_id=gd_l1vijqt9jfj7olije"

​
Troubleshooting

IssuePossible Solution
”File not found”Make sure the filename in your filter exactly matches the uploaded file name.
”Invalid file format”Check CSV header matches the filter field name, or ensure JSON is an array of objects.
”Field not found”Verify field exists in dataset. Use Get Dataset Metadata.

Authorizations

​
Authorization
string
header
required

Use your Bright Data API Key as a Bearer token in the Authorization header.

How to authenticate:

  1. Obtain your API Key from the Bright Data account settings at https://brightdata.com/cp/setting/users
  2. Include the API Key in the Authorization header of your requests
  3. Format: Authorization: Bearer YOUR_API_KEY

Example:

Authorization: Bearer b5648e1096c6442f60a6c4bbbe73f8d2234d3d8324554bd6a7ec8f3f251f07df

Learn how to get your Bright Data API key: https://docs.brightdata.com/api-reference/authentication

Query Parameters

​
dataset_id
string
required

ID of the dataset to filter (required in multipart/form-data mode)

Example:

"gd_l1viktl72bvl7bjuj0"

​
records_limit
integer

Limit the number of records to be included in the snapshot

Example:

1000

Body

multipart/form-data
​
filter
Single field filter · object
required
Example:
{
  "name": "name",
  "operator": "=",
  "value": "John"
}

Response

Job of creating the snapshot successfully started

​
snapshot_id
string

ID of the snapshot