Features

​

Data scraper Dashboard

Any scraper you create using a template or a custom scraper will appear on your Data scraper dashboard.

​

Overview

• Free trial: As part of the 7-day free trial, you’re entitled to 1,000-page loads
• Update available: A new version of the scraper is available. If there is no update button, you have the latest version.
• Properties: This is where you’ll see all the scraper properties. Learn more
• Delivery preferences: Choose your desired file format, delivery method, and notification settings. Learn more
• Output configuration/ Schema: Here, you can go back to edit your output definitions. Learn more

​

Scraper action menu

The scraper action menu allows performing different actions with the scraper.

• Initiate by API - start a data collection without having to enter the control panel
• Initiate manually - Bright Data’s control panel makes it easy to get started collecting data
• Run on schedule - select precisely when to collect the data you need
• Versions - review the modified versions of the scraper
• Report an issue - You can use this form to communicate any problems you have with the platform, the scraper, or the dataset results
• Copy link - copy the link of the scraper to share it with your colleagues
• Tickets - view the status of your tickets
• Advanced options:
  • Edit the code - edit the scraper’s code within the IDE.
  • Disable scraper - temporarily disable the scraper, but you can reactivate it if needed.
  • Delete scraper - permanently delete the scraper.

​

Properties:

• Maintainer of the scraper :
  • Self-serve: scraper is maintained by you
  • Full-service: scraper is maintained by Bright Data Developers
• Type of the scraper :
  • Search : The scraper input is a keyword (i.e., iPhone)
  • PDP : The scraper input is a product page URL
  • Discovery : The scraper input is a category URL
  • Other
• Use case of the scraper (Social media, eCommerce, Travel, etc.)
• Last modified : indicates when the scraper was last updated
• Price of CPM : 1 CPM = 1,000 page loads.
• Avg. Page-load per input : Average number of loaded pages to process 1 input set

​

Initiate scraper and get collection results

Initiate scraper

To start collecting the data, you have three options:

A. Initiate by API
B. Initiate manually
C. Schedule a scraper

Get collection results

Once the data collection is completed, click the “three dots” icon and select “Statistics” to access the results and download the data.

​

Statistics

The statistics page presents essential information about the success of the data collection. Below is a list of all the terms included in the statistics table:

Statistics actions menu

• 3 dots Here you can perform different functions with the data collection job: The statistics page presents essential information about the success of the data collection. Below is a list of all the terms included in the statistics table:

• Job ID - The unique id of the collection
• Trigger - The person who initiated the data collection and how (API, manually or scheduled)
• Inputs - The number of inputs inserted into the collection
• Records - The number of results collected
• Failed - The number of pages failed to be crawled
• Success rate - The percentage of the results that were successfully collected
• Queued at - The queue timestamp
• Started at - The date and time when the scraper began collecting
• Finished at - The date and time when the scraper finished collecting
• Job time - The length of time it took to complete
• Estimated time left - The amount of time left until collection is complete
• Queue - The name of the job given in the trigger behavior (Queue name)
• Usage - The total amount of page loads used