Features
Disable CAPTCHA Solving
By default, as part of our full proxy unblocking solution, Web Unlocker also solves CAPTCHAs that are encountered while returning your proxy request.
When disabling CAPTCHA solver, our intelligent algorithm still takes care of the entire ever-changing flow of finding the best proxy network, customizing headers, fingerprinting, and more, but intentionally does not solve CAPTCHAs automatically, giving your team a lightweight, streamlined solution, that broadens the scope of your potential scraping opportunities.
Best for:
- Scraping data from websites without getting blocked
- Emulating real-user web behavior
- Teams that don’t have an unblocking infrastructure in-house and don’t want their scraper to solve CAPTCHAs automatically
Web Unlocker Premium Domains
Premium domains are a part of Bright Data’s two-tiered website classification system and these are websites that are more challenging to unblock than others and require additional Web Unlocker resources.
In this article, we will see the current list of Premium domains, understand how to target them, and go over the special pricing.
Current List of Premium Domains
This list is frequently changing based on our website classification logic. You can see the most up-to-date list within your proxy zone
Enable Premium Domains
When creating your Web Unlocker zone, check the ‘Premium Domains’ box under Special features
Pricing
Once enabled, the premium price of $5 more per CPM ($8 total), will be reflected in the “Estimated cost” section.
Even after enabled, only specific requests to these domains will be priced at the higher rate. Requests to other domains will be kept at the default lower tier.
Monitor Web Unlocker Usage
To review your current Web Unlocker CPM, navigate to My Proxies page, and review the Traffic column.
Understanding usage
The number displayed in the traffic column above is the number of successful requests. In the example above, 115k requests are equal to 115 CPM, so you’ll be billed according to the rate of 10 CPMs for that billing cycle.
How is usage calculated?
Your Web Unlocker usage is measured by CPM. CPM is the cost of 1000 successful requests, meaning only successful Web Unlocker requests will count toward your billing.
See our Billing & Pricing page to learn more.
Headers & Cookies management
Web Unlocker automatically manages all headers and cookies in your request to get the best results. Therefore, any header/cookies sent along with the request are disregarded.
If your use case requires additional headers or cookies, please create a ticket and send us the complete details about your request and what you expect in the response, including which headers/cookies you want to add.
Be advised:
- We do not allow cookies for login/authentication purposes
- Adding special headers/cookies to the request may result in blocking and a drop in the success rate.
- Performance drops due to headers added to the request are not been monitored by bright data
systems and handling blocking events might take longer to resolve
Common error codes
Occaisianly, for a number of reasons, you might receive an unexpected error code in response to your Web Unlocker request.
The following list will provide you with a deeper understanding of what the source of the issue may be.
| Error | Description |
|---|---|
404 | Page not found. Invalid URL, which suggests the URL might be broken or dead. |
| 403 | The URL you’re trying to access is valid, but you are forbidden from accessing that URL. |
| 502 | Error code 502 is the most common error for Web Unlocker users, the descriptive part is under the x-luminati-error-code. |
407 | This error code suggests one of your account credentials is incorrect (password or zone’s name). |
429 | This error code implies a rate limit (rare). In such cases, if the response appears as below, Bright Data is applying auto-throttling to the request, and you should open a ticket or email support@brightdata.com for assistance (you can read more about Auto-Throttling here) |
401411444 | Bad request, usually happens in API requests when headers or cookies missing. |
503 | HTTP error code 503 means “Service Unavailable”. Browser check failed or browser check wasn’t completed |
< HTTP/1.1 429 The request was auto-throttled due to low success rate
< x-luminati-error-code: sr_rate_limit
< x-luminati-error: The request was auto-throttled due to low success rate
< x-brd-error-code: sr_rate_limit
< x-brd-error: The request was auto-throttled due to low success rate
< date: Tue, 23 Jan 2024 17:07:19 GMT
< connection: keep-alive
< keep-alive: timeout=5
< transfer-encoding: chunked
<
* Connection #0 to host brd.superproxy.io left intact