Management APIs for Web Scraper API
Explore Bright Data’s Management APIs to monitor, retrieve, and deliver web data snapshots. Manage data collection status, delivery, and batch processing efficiently.
Monitor Progress
Endpoint: GET /datasets/v3/progress/:snapshot_id
Returns the status of a specific collection that has previously been triggered.
Parameters
The ID that was returned when the collection was triggered
Example:
s_lynh132v19n82v81kx
Get snapshot list
Endpoint: GET /datasets/v3/snapshots
Returns a list of triggered collections, the list contains only snapshots created for a specific dataset
Query Parameters
The dataset identifier (can be found in the specific API page)
Example:
dataset_id=gd_l1vikfnt1wgvvqz95w
List only snapshots with a specific status
Example:
status=ready
Options:
running,ready,failed
List only snapshots that were created after a specific date
Example:
from_date=2024-01-01
List only snapshots that were created before a specific date
Example:
from_date=2024-04-01
Monitor delivery
Endpoint: GET /datasets/v3/delivery/:delivery_id
The call returns the delivery status.
Parameters
The unique delivery id returned from the delivery API endpoint
Example:
delivery_id=d_lysxl9vf2dobrb6h31
Calculate the number of parts for delivery
Endpoint: GET /datasets/v3/snapshot/:snapshot_id/parts
When requesting a delivery in batches (see available delivery APIs) use this endpoint to check how many parts were created. format, compress and batch_size should exactly match what was sent to the delivery/download API calls.
Was this page helpful?