Scraping Shield

Scraping shield provides data about the outgoing proxy traffic of the customers so they can have better control on their traffic, additionally, it also allows to trigger predefined actions when certain conditions are met, such as: send email alerts, block requests, etc.

​

Scraping shield features

​

Domain Classification

The domain classification feature categorizes outgoing traffic from the customer’s proxy operations into groups like Shopping, Social Media, and Travel. Customers can see how much traffic goes to each category and drill down to see specific domains. They can also set rules to block specific categories or domains from proxy usage, and alert designated users when there is an access attempt.

What can I see in the Domain Classification page?

On the domain classification page, you can view a chart that displays all outgoing proxy traffic from your account. This chart can be broken down by classification, domain, or zone.

Additionally, there is a table that shows all traffic broken down by classification. Customers can expand each row of this table to see the specific domains or zones used, along with the number of requests and the amount of traffic to these domains/zones.

Customers can also add rules to perform specific actions when a specific class/domain is targeted.

​

Adding and using rules

Rules provide the ability for the customer to set specific actions or alerts based on the classification or domains of outgoing traffic. These rules include the ability to block requests to certain classifications or domains and to set alerts when a specific class or domain has been targeted.

There are 2 ways to add rules:

• Create rule from scratch
• Create rule from stats table

• Login to your account and go to Domain Classification page.
• Choose the ‘Rules’ tab in the top of the page.
• Click on ‘Add rule’
• Choose if the rule should apply to an entire classification or a specific domain
• The “Block requests” checkbox will block all requests to the chosen classification/domain when enabled.
• The “Notify..” checkbox will allow you to send alerts to designated users when a minimum number of requests have been sent to the chosen classification/domain.

​

Scraping Shield - Billing

Scraping Shields pricing is $10k/month at the minimum. The price for customers with usage costs of more than $200k/month will be an additional 5% of their usage costs instead. (Actual usage costs, not the monthly commitment itself).

For example, a customer with usage costs of $150k/month will pay the minimum $10k/month, but a customer with $220k/month will pay $11k/month.

The usage cost is the total amount of money that was utilized by using the service, and only for products that are relevant for Scraping Shield usage are taken into account: All proxy products, Web-Scraper IDE and Custom Datasets. (all products but marketplace datasets)