Skip to main content
POST
/
datasets
/
filter
cURL
curl --request POST \
  --url https://api.brightdata.com/datasets/filter \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: multipart/form-data' \
  --form 'filter={
  "name": "name",
  "operator": "=",
  "value": "John"
}'
{
  "snapshot_id": "<string>"
}
The Filter endpoint of the Bright Data Marketplace Dataset API can filter a dataset against thousands of values stored in a CSV or JSON file. Upload one or more files in multipart mode and reference each filename in your filter.
Paste your API key into the authorization field. To get an API key, Create an account and learn how to generate an API key.

​
How does file-based filtering work?

Use file uploads when you need to filter against large value lists, such as including or excluding 100k+ company IDs:
  • Upload CSV or JSON files in multipart/form-data mode and reference each filename in your filter.
  • Each file holds up to 10,000 data rows, and the whole request can be up to 200 MiB.
  • The Filter job runs asynchronously and returns a snapshot_id to download once it completes.
For the async job flow, limits, pricing and error codes, see Filter dataset (async).

​
How do I format the CSV or JSON file?

  • First line must be a header matching the field name in your filter.
  • Each following line contains a single value.
Example: industries.csv
industries:value
Accounting
Ad Network
Advertising

​
How do I reference a file in the filter?

When using file uploads, set the filter’s value field to the filename:
Example
{
  "operator": "and",
  "filters": [
    {
      "name": "industries:value",
      "operator": "includes",
      "value": "industries.csv"
    }
  ]
}
File references work only with the operators in, not_in, includes, not_includes, array_includes and not_array_includes. For the full operator table and field types, see the filter syntax reference.

​
Filter with multiple files

curl \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: multipart/form-data" \
  -F "files[]=@/path/to/industries.csv" \
  -F "files[]=@/path/to/regions.csv" \
  -F "filter={\"operator\":\"and\",\"filters\":[{\"name\":\"industries:value\",\"operator\":\"includes\",\"value\":\"industries.csv\"},{\"name\":\"region\",\"operator\":\"in\",\"value\":\"regions.csv\"}]}" \
  "https://api.brightdata.com/datasets/filter?dataset_id=gd_l1vijqt9jfj7olije"

​
Troubleshoot file uploads

IssuePossible Solution
”File not found”Make sure the filename in your filter exactly matches the uploaded file name.
”Invalid file format”Check CSV header matches the filter field name, or ensure JSON is an array of objects.
”Field not found”Verify field exists in dataset. Use Get Dataset Metadata.

Authorizations

​
Authorization
string
header
required

Use your Bright Data API Key as a Bearer token in the Authorization header.

How to authenticate:

  1. Obtain your API Key from the Bright Data account settings at https://brightdata.com/cp/setting/users
  2. Include the API Key in the Authorization header of your requests
  3. Format: Authorization: Bearer YOUR_API_KEY

Example:

Authorization: Bearer b5648e1096c6442f60a6c4bbbe73f8d2234d3d8324554bd6a7ec8f3f251f07df

Learn how to get your Bright Data API key: https://docs.brightdata.com/api-reference/authentication

Query Parameters

​
dataset_id
string
required

ID of the dataset to filter (required in multipart/form-data mode)

Example:

"gd_l1viktl72bvl7bjuj0"

​
records_limit
integer

Limit the number of records to be included in the snapshot

Example:

1000

Body

multipart/form-data
​
filter
Single field filter · object
required
Example:
{
  "name": "name",
  "operator": "=",
  "value": "John"
}

Response

Job of creating the snapshot successfully started

​
snapshot_id
string

ID of the snapshot