Features

​

Data scraper Dashboard

Any scraper you create using a template or a custom scraper will appear on your Data scraper dashboard.

​

Overview

• Free trial: As part of the 7-day free trial, you’re entitled to 1,000-page loads

• Update available: A new version of the scraper is available. If there is no update button, you have the latest version.

• Properties: This is where you’ll see all the scraper properties. Learn more

• Delivery preferences: Choose your desired file format, delivery method, and notification settings. Learn more

• Output configuration/ Schema: Here, you can go back to edit your output definitions. Learn more

• Limitations - Our collectors have a limitation of 100 parallel-running jobs. When more than 100 jobs are triggered, the additional jobs are placed in a queue and wait until the earlier ones finish.  

​

Scraper action menu

The scraper action menu allows performing different actions with the scraper.

• Initiate by API - start a data collection without having to enter the control panel

• Initiate manually - Bright Data’s control panel makes it easy to get started collecting data

• Run on schedule - select precisely when to collect the data you need

• Versions - review the modified versions of the scraper

• Report an issue - You can use this form to communicate any problems you have with the platform, the scraper, or the dataset results

• Copy link - copy the link of the scraper to share it with your colleagues

• Tickets - view the status of your tickets

• Advanced options:

  • Edit the code - edit the scraper’s code within the IDE.

  • Disable scraper - temporarily disable the scraper, but you can reactivate it if needed.

  • Delete scraper - permanently delete the scraper.

​

Properties:

• Maintainer of the scraper :

  • Self-serve: scraper is maintained by you

  • Full-service: scraper is maintained by Bright Data Developers

• Type of the scraper :

  • Search : The scraper input is a keyword (i.e., iPhone)

  • PDP : The scraper input is a product page URL

  • Discovery : The scraper input is a category URL

  • Other

• Use case of the scraper (Social media, eCommerce, Travel, etc.)

• Last modified : indicates when the scraper was last updated

• Price of CPM : 1 CPM = 1,000 page loads.

• Avg. Page-load per input : Average number of loaded pages to process 1 input set

​

Initiate scraper and get collection results

Initiate scraper

To start collecting the data, you have three options:

A. Initiate by API B. Initiate manually C. Schedule a scraper

Get collection results

Once the data collection is completed, click the “three dots” icon and select “Statistics” to access the results and download the data.

​

Statistics

The statistics page presents essential information about the success of the data collection. Below is a list of all the terms included in the statistics table:

Statistics actions menu

• 3 dots Here you can perform different functions with the data collection job: The statistics page presents essential information about the success of the data collection. Below is a list of all the terms included in the statistics table:

• Job ID - The unique id of the collection

• Trigger - The person who initiated the data collection and how (API, manually or scheduled)

• Inputs - The number of inputs inserted into the collection

• Records - The number of results collected

• Failed - The number of pages failed to be crawled

• Success rate - The percentage of the results that were successfully collected

• Queued at - The queue timestamp

• Started at - The date and time when the scraper began collecting

• Finished at - The date and time when the scraper finished collecting

• Job time - The length of time it took to complete

• Estimated time left - The amount of time left until collection is complete

• Queue - The name of the job given in the trigger behavior (Queue name)

• Usage - The total amount of page loads used