Overview

Our Archive API allows you to access and retrieve Data Snapshots from Bright Data’s cached data collections in a seamless and efficient method.

To access this API, you will need a Bright Data API key

Run a Search

To initiate a search of our Archive, use the following /search endpoint. Endpoint: POST api.brightdata.com/webarchive/search

If the search takes longer than 30 seconds, the response returns only a search_id and you should poll the status asynchronously. If the search completes within 30 seconds, the response returns the full search result object (same as GET /webarchive/search/<search_id>).

Request
Response
Dictionary

POST api.brightdata.com/webarchive/search
{
  "filters": {
    "max_age": "Duration", // mandatory: use max_age OR (min_date + max_date)
    "min_date": "YYYY-MM-DD", // mandatory if max_age is not set
    "max_date": "YYYY-MM-DD", // mandatory if max_age is not set
    "domain_whitelist": ["example.com"],
    "domain_blacklist": ["example.com"],
    "domain_regex_whitelist": [".*example..*"],
    "domain_regex_blacklist": [".*example..*"],
    "domain_like_whitelist": ["%.example.%", "example%"],
    "domain_like_blacklist": ["%.example.ca"],
    "category_whitelist": ["Automotive"],
    "category_blacklist": ["Automotive"],
    "url_regex_whitelist": [".*/products/.*"],
    "url_regex_blacklist": [".*/products/.*"],
    "url_like_whitelist": ["%/products/%", "%/search%"],
    "url_like_blacklist": ["%/review/%"],
    "language_whitelist": ["eng"], // ISO 639-3 letter language codes
    "language_blacklist": ["eng"],
    "ip_country_whitelist": ["us", "ie", "in"],
    "ip_country_blacklist": ["mx", "ae", "ca"],
    "captcha": true,
    "robots_block": true
  }
}

{
  "search_id": "ucd_abc123xyz"
}

Here is a brief explanation of each of the parameters you are able to use in your requests:

Parameter	Description
max_age	Limits results to records collected within a specified time range.
min_date	Returns records collected on or after the specified date.
max_date	Returns records collected on or before the specified date.
domain_whitelist	Includes results only from listed domains.
domain_blacklist	Excludes results from listed domains.
domain_like_whitelist	Includes domains matching LIKE pattern (% = any chars, _ = single char). Case-insensitive.
domain_like_blacklist	Excludes domains matching LIKE pattern. Case-insensitive.
category_whitelist	Includes results only from specified categories.
category_blacklist	Excludes results from specified categories.
url_regex_whitelist	Includes results only matching the specified URL regex pattern.
url_regex_blacklist	Excludes results matching the specified URL regex pattern.
url_like_whitelist	Includes URLs matching LIKE pattern (% = any chars, _ = single char). Case-insensitive.
url_like_blacklist	Excludes URLs matching LIKE pattern. Case-insensitive.
language_whitelist	Includes results only for specific language codes (ISO 639-3).
language_blacklist	Excludes results for specific language codes.
ip_country_whitelist	Includes results collected through IPs or peers only from specified countries.
ip_country_blacklist	Excludes results collected through IPs or peers from specified countries.
captcha	Return only results with captcha triggered
robots_block	Return only results with robots block

You can run up to 100 searches per day without triggering a dump. Once you trigger a dump, that search no longer count against your limit.

LIKE vs Regex Filters: Use LIKE filters (domain_like_*, url_like_*) for simple pattern matching with % (any sequence) and _ (single character). LIKE patterns are case-insensitive and often faster than regex for simple prefix/suffix matching like %.com or amazon%. Use regex filters (domain_regex_*, url_regex_*) for complex patterns requiring full regex syntax. LIKE patterns use backslash escaping: \% for literal %, \_ for literal _.

Get Search Status

To check the status of a specific query that was made. Endpoint: GET api.brightdata.com/webarchive/search/<search_id> When successful it will retrieve:

The number of entries for your query
The estimated size and cost of the full Data Snapshot

Pricing & size: estimate_batch_size is measured in bytes. dump_cost_usd is an estimated total cost based on files_count and your current cache/archive pricing tier. The cost_breakdown object shows separate costs for cache vs archive pages.

Request
Response
Response Dictionary

GET api.brightdata.com/webarchive/search/<search_id>

{
  "search_id": "ucd_abc123xyz",
  "status": "in_progress"
}

Field	Description
search_id	Unique identifier for the search
status	Current status: `in_progress`, `done`, or `failed`
filters	The filters used for this search (echoed back)
files_count	Total number of matching files found
estimate_batch_count	Estimated number of batches for the dump
estimate_batch_size	Estimated total size in bytes
dump_cost_usd	Estimated total cost to create a dump
cost_breakdown	Breakdown of costs between cache and archive pages
estimate_dump_duration_sec	Estimated time to complete the dump in seconds
duration	How long the search took to complete
error	Error message (only present when status is `failed`)

Get All Search Statuses

Check the status of all current searches. Endpoint: GET api.brightdata.com/webarchive/searches

Request
Response

GET api.brightdata.com/webarchive/searches

200 OK

[
  {
    "search_id": "ucd_abc123xyz",
    "status": "done",
    "filters": {
      "max_age": "1d",
      "domain_whitelist": ["example.com"],
      "min_date": "2026-02-05T11:00:00.000Z"
    },
    "files_count": 2885,
    "estimate_batch_count": 1,
    "estimate_batch_size": 1132300,
    "dump_cost_usd": 0.58,
    "cost_breakdown": {
      "archive_pages_count": 0,
      "archive_pages_cost": 0,
      "cache_pages_count": 2885,
      "cache_pages_cost": 0.58
    },
    "estimate_dump_duration_sec": 37.79,
    "duration": "2s425ms"
  },
  {
    "search_id": "ucd_def456abc",
    "status": "in_progress"
  }
]

How data range affects delivery time

If your query is matching data within last 24 hours - your snapshot will start processing/delivering immediately. If some of your matched data is older than 24 hours - it needs to be retrieved from S3 Glacier Deep Archive storage tier before delivery, which may take up to 72 hours.

Starting February 9, 2026, the hot storage retention period changed from 72 hours to 24 hours.

We recommend using max_age = 12h for initial testing to ensure fast delivery.

Warning: Avoid queries that span the retention boundary (approximately 24 hours from now). Requests with max_age or time ranges that fall within ~24h ± 2h of the current time may include files that have already been migrated to archive storage tier. Attempting a dump for such queries can cause the dump to stall or remain incomplete because of files storage class transition.Recommendations:

For real-time data needs: use max_age: "12h" or a narrower window to avoid the retention edge.
For historical data (older than 24h): use explicit min_date/max_date filters rather than max_age.
If a dump appears stalled: we usually retry automatically, please open a ticket if it didn’t happen.

Deliver Snapshot to Amazon S3 Storage

To use S3 storage delivery, you will first need to do the following:

Create an AWS role which gives Bright Data access to your system.
- During this setup, you will be asked by Amazon for an “external ID” that is used with the role.
- Your external ID for S3 is your Bright Data Account ID that can be found within Account Settings
Once a role is created, you will need to allow our system delivery role to AssumeRole that role.
- Our system delivery role is: arn:aws:iam::422310177405:role/brd.ec2.zs-dca-delivery

To deliver a specific Snapshot from a specific search_id to S3 storage, use the following /dump endpoint. Endpoint: POST api.brightdata.com/webarchive/dump

Common dump parameters:

search_id (required): The search ID from a completed search
max_entries (optional): Limit the number of files to include in the dump
delivery (required): Delivery configuration (S3, Azure, or webhook)

Request
Response

{
  "search_id": "ucd_abc123xyz",
  "max_entries": 1000000,
  "delivery": {
    "strategy": "s3",
    "settings": {
      "bucket": "your-bucket-name",
      "prefix": "optional/custom/prefix",
      "assume_role": {
        "role_arn": "arn:aws:iam::YOUR_ACCOUNT:role/your-role"
      }
    }
  }
}

200 OK

{"dump_id": "ucd_abc123-1234567890"}

Deliver Snapshot to Azure Blob Storage

Deliver a specific Snapshot from a specific search_id directly into an Azure Blob Storage container using the same /dump endpoint. Endpoint: POST api.brightdata.com/webarchive/dump

Request
Response

{
  "search_id": "ucd_abc123xyz",
  "max_entries": 1000000,
  "delivery": {
    "strategy": "azure",
    "settings": {
      "container": "your-container-name",
      "prefix": "optional/custom/prefix",
      "credentials": {
        "account": "your-storage-account",
        "key": "your-account-key"
      }
    }
  }
}

200 OK

{"dump_id": "ucd_abc123-1234567890"}

Collect Snapshot via Webhook

Collect a Data Snapshot via webhook from a specific search_id Endpoint: POST api.brightdata.com/webarchive/dump

Request
Response

{
  "search_id": "ucd_abc123xyz",
  "max_entries": 1000000,
  "delivery": {
    "strategy": "webhook",
    "settings": {
      "url": "https://your-domain.com/webhook",
      "auth": "Bearer your-optional-auth-token"
    }
  }
}

200 OK

{"dump_id": "ucd_abc123-1234567890"}

If you’re running a linux/macos machine, you can simulate one of our delivery webhooks with the code on this page.

Get Status of Data Snapshot

Check the status of a specific Data Snapshot (dump) using the dump_id. Endpoint: GET api.brightdata.com/webarchive/dump/<dump_id>

Request
Response
Response Dictionary

GET api.brightdata.com/webarchive/dump/<dump_id>

The status code of all three following responses is 200 OK

{
  "dump_id": "ucd_abc123-1234567890",
  "id": "ucd_abc123-1234567890",
  "status": "in_progress",
  "created": "2026-02-05T07:57:50.947Z",
  "readiness": "22.31%",
  "search_id": "ucd_xyz789",
  "batches_total": 130,
  "batches_uploaded": 29,
  "files_total": 1241241251,
  "files_uploaded": 276987432
}

Field	Description
dump_id	Unique identifier for the dump
id	Same as dump_id (included for compatibility)
status	Current status: `in_progress`, `done`, `failed`, or `canceled`
created	ISO timestamp when the dump was created
done	ISO timestamp when the dump completed (only when status is `done`)
readiness	Percentage of completion (e.g., “50.00%“)
search_id	ID of the search this dump is based on
batches_total	Total number of batches to process
batches_uploaded	Number of batches uploaded so far
files_total	Total number of files in the dump
files_uploaded	Number of files uploaded so far
error	Error message (only present when status is `failed`)

Get the Status of all Data Snapshots

Endpoint: GET api.brightdata.com/webarchive/dumps

Request
Response

GET api.brightdata.com/webarchive/dumps

200 OK

[
  {
    "dump_id": "ucd_abc123-1234567890",
    "id": "ucd_abc123-1234567890",
    "status": "done",
    "created": "2026-02-05T07:57:50.947Z",
    "done": "2026-02-05T08:21:55.245Z",
    "readiness": "100.00%",
    "search_id": "ucd_xyz789",
    "filters": {
      "max_age": "2h",
      "min_date": "2026-02-05T05:00:00.000Z"
    },
    "batches_total": 1,
    "batches_uploaded": 1,
    "files_total": 10,
    "files_uploaded": 10
  },
  {
    "dump_id": "ucd_def456-9876543210",
    "id": "ucd_def456-9876543210",
    "status": "in_progress",
    "created": "2026-02-04T13:35:54.607Z",
    "readiness": "50.00%",
    "search_id": "ucd_qwe321",
    "filters": {
      "max_age": "1d",
      "domain_whitelist": ["example.com"]
    },
    "batches_total": 10,
    "batches_uploaded": 5,
    "files_total": 1000,
    "files_uploaded": 500
  }
]

Introduction

Product Guides

Run a Search

Get Search Status

Get All Search Statuses

How data range affects delivery time

Deliver Snapshot to Amazon S3 Storage

Deliver Snapshot to Azure Blob Storage

Collect Snapshot via Webhook

Get Status of Data Snapshot

Get the Status of all Data Snapshots

High-level process flow diagram

Introduction

Product Guides

​Run a Search

​Get Search Status

​Get All Search Statuses

​How data range affects delivery time

​Deliver Snapshot to Amazon S3 Storage

​Deliver Snapshot to Azure Blob Storage

​Collect Snapshot via Webhook

​Get Status of Data Snapshot

​Get the Status of all Data Snapshots

​High-level process flow diagram

Run a Search

Get Search Status

Get All Search Statuses

How data range affects delivery time

Deliver Snapshot to Amazon S3 Storage

Deliver Snapshot to Azure Blob Storage

Collect Snapshot via Webhook

Get Status of Data Snapshot

Get the Status of all Data Snapshots

High-level process flow diagram