Scraping shield provides data about the outgoing proxy traffic of the customers so they can have better control on their traffic, additionally, it also allows to trigger predefined actions when certain conditions are met, such as: send email alerts, block requests, etc.

Scraping shield features

Domain Classification

The domain classification feature categorizes outgoing traffic from the customer’s proxy operations into groups like Shopping, Social Media, and Travel. Customers can see how much traffic goes to each category and drill down to see specific domains. They can also set rules to block specific categories or domains from proxy usage, and alert designated users when there is an access attempt.

Adding and using rules

Rules provide the ability for the customer to set specific actions or alerts based on the classification or domains of outgoing traffic. These rules include the ability to block requests to certain classifications or domains and to set alerts when a specific class or domain has been targeted.

There are 2 ways to add rules:

  • Login to your account and go to Domain Classification page.
  • Choose the ‘Rules’ tab in the top of the page.
  • Click on ‘Add rule’
  • Choose if the rule should apply to an entire classification or a specific domain
  • The “Block requests” checkbox will block all requests to the chosen classification/domain when enabled.
  • The “Notify..” checkbox will allow you to send alerts to designated users when a minimum number of requests have been sent to the chosen classification/domain.
Create rule from Scratch

Scraping Shield - Billing

Scraping Shields pricing is $10k/month at the minimum. The price for customers with usage costs of more than $200k/month will be an additional 5% of their usage costs instead. (Actual usage costs, not the monthly commitment itself).

For example, a customer with usage costs of $150k/month will pay the minimum $10k/month, but a customer with $220k/month will pay $11k/month.

The usage cost is the total amount of money that was utilized by using the service, and only for products that are relevant for Scraping Shield usage are taken into account: All proxy products, Web-Scraper IDE and Custom Datasets. (all products but marketplace datasets)

Click here to learn about Scraping Shield API Reference