- Residential
- Data Center
- ISP
- Mobile
- Proxy Manager
Proxy products FAQ
Find answers to common questions about integrating, configuring, and using Bright Data’s proxy products, including IP types, geotargeting, and error codes.
In bright data, a “zone” represents a specific product and its configuration settings. You can think of it as an instance in the context of cloud computing.
For example, you can have a zone for Datacenter proxy in Germany and you can have a separate zone for Datacenter proxies in France.
Every zone has a name and one or more password, allowing you to interact with it.
The zone name itself cannot be changed once it’s configured, so you may want to use an easy to remember name that represents what you’re trying to achieve. Keep in mind, that you can add a description to the zone, and you can change that description anytime. You can find it under the zone name at the top of the page.
The zone name is used to create your proxy username, which you can then customise if you want in order to control your proxies. Here is a sample proxy username for proxies in the USA - as you can see it includes the zone name inside it:
brd-customer-<customer_id>-zone-<zone_name>-country-us
As mentioned, you cannot change the name of the zone after you have saved it - if you want to change the name, simply create a new with a new name.
Bright data can be easily integrated into many third-party tools. In general, simply configure an HTTP/HTTPS proxy using the credentials for your Bright Data product. If you are using residential proxies, web unlocker or SERP API don’t forget to use the SSL certificate to ensure and to end encrypted connections.
We have prepared detailed guides for the most commonly used tools. You can find links to the guides in the list below:
BrowserScan https://brightdata.com/integration/browserscan
XLogin https://brightdata.com/integration/xlogin
GeeLark https://brightdata.com/integration/geelark
Puppeteer https://brightdata.com/integration/puppeteer
Playwright https://brightdata.com/integration/playwright
Selenium https://brightdata.com/integration/selenium
AdsPower https://brightdata.com/integration/adspower
Dolphin Anty https://brightdata.com/integration/dolphin-anty
Incogniton https://brightdata.com/integration/incogniton
Marketerbrowser https://brightdata.com/integration/marketerbrowser
SMLOGIN https://brightdata.com/integration/smlogin
Hidemyacc https://brightdata.com/integration/hidemyacc
OpenBullet https://brightdata.com/integration/openbullet
Switchyomega https://brightdata.com/integration/switchyomega
PhantomBuster https://brightdata.com/integration/phantombuster
BitBrowser https://brightdata.com/integration/bitbrowser
Maskfog https://brightdata.com/integration/maskfog
Ghost Browser https://brightdata.com/integration/ghost-browser
Postman https://brightdata.com/integration/postman
NGINX https://brightdata.com/integration/nginx
StablerSOLO https://brightdata.com/integration/stablersolo
VMLogin https://brightdata.com/integration/vmlogin
GoLogin https://brightdata.com/integration/gologin
Windows https://brightdata.com/integration/windows
Scrapy https://brightdata.com/integration/scrapy
AEZAKMI https://brightdata.com/integration/aezakmi
Beautifulsoup https://brightdata.com/integration/beautifulsoup
WebHarvy https://brightdata.com/integration/webharvy
Ubuntu https://brightdata.com/integration/ubuntu
Lalicat https://brightdata.com/integration/lalicat
Multilogin https://brightdata.com/integration/multilogin
Undetectable https://brightdata.com/integration/undetectable
Apify https://brightdata.com/integration/apify
iPhone https://brightdata.com/integration/iphone
MuLogin https://brightdata.com/integration/mulogin
Changedetection https://brightdata.com/integration/changedetection
Morelogin https://brightdata.com/integration/morelogin
Proxifier https://brightdata.com/integration/proxifier
Texau https://brightdata.com/integration/texau
Android https://brightdata.com/integration/android
Kameleo https://brightdata.com/integration/kameleo
Screaming Frog https://brightdata.com/integration/screaming-frog
Foxy https://brightdata.com/integration/foxyproxy
SessionBox https://brightdata.com/integration/sessionbox
Insomniac https://brightdata.com/integration/insomniac
Helium Scraper https://brightdata.com/integration/heliumscraper
SaleFreaks https://brightdata.com/integration/salefreaks
Postern https://brightdata.com/integration/postern-proxy-integration
Antik https://brightdata.com/integration/antik
Easync https://brightdata.com/integration/easync
ParseHub https://brightdata.com/integration/parsehub
Sphere Browser https://brightdata.com/integration/sphere-browser
Octoparse https://brightdata.com/integration/octoparse
ixBrowser https://brightdata.com/integration/ixbrowser
Shadowrocket https://brightdata.com/integration/shadowrocket
Firefox https://brightdata.com/integration/firefox
Chrome https://brightdata.com/integration/chrome
MacOS https://brightdata.com/integration/macos
ClonBrowser https://brightdata.com/integration/clonbrowser
Octo Browser https://brightdata.com/integration/octobrowser
Genlogin https://brightdata.com/integration/genlogin
Web Scraper https://brightdata.com/integration/webscraper
If the tool you’re using does not appear on the list and you would like to get a guide for it, contact us!
There are two types of IPs that we offer within our Residential and Mobile Proxy:
IPs that are shared across multiple users. Connect to our entire network of 72+ million Residential IPs. Rotate between countries, cities, and ASNs using the Proxy Manager, Extension, or raw API commands.
Cost is calculated per GB consumed, according to your monthly plan.
Geolocation targeting
The IP allocated to your zone will be from the selected country, city, ASN, or zip code. Select the needed parameter from the drop-down menu.
Sending a request with Geolocation:
-
With a ‘Shared’ IP, you’ll be able to control your geolocation targeting when you send a request.
-
To target a specific country, add the
-country
flag to your request,
For example, if you’d like to send your request from the United States (‘us’), your request using a shared IP will look like
curl "http://target.site" --proxy brd.superproxy.io:22225 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-country-us: <zone_password>
Super-proxies are the gateway servers of Bright Data. Every request sent through Bright Data’s proxy platform passes through these servers, which select the best peer based on the request details and balance the load among peers.
In some cases, Bright Data may not be able to process your request via a peer, resulting in a super-proxy bypass. This means the request is sent directly from Bright Data’s gateway servers instead of a peer.
When a super-proxy bypass occurs, the response headers will include an info message explaining why it happened.
It is possible to block requests from being sent from the super-proxies when it happens, simply add the -route_err-block
flag to your proxy username:
curl "http://target.site" --proxy brd.superproxy.io:22225 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-route_err-block:<zone_password>
There are four types of IPs that we offer within our Datacenter Proxy Network:
-
Shared (Pay per usage)
-
Shared (Pay per IP)
-
Dedicated
-
Premium IPs (Note: no longer offered)
Shared IPs are shared across multiple users. We have 2 tiers for shared IPs within the ISP proxy network:
Connects you to a pool of 40,000 IPs that consist of a fixed allocated range of IPs from every country available. These IPs can be continuously rotated within the pool of 20k as needed (specific countries can be chosen in chrome, the Proxy manager, or through API)
Features
-
Geolocation targeting - With a ‘Shared’ IP (pay-per-usage), you’ll be able to control your geolocation targeting when you send a request, with the
-country
flag. A simple example of a request that targets the US would look likeExamplecurl --proxy brd.superproxy.io:22225 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-country-us:<zone_password> "http://lumtest.com/myip.json"
Learn more about targeting different country targeting options from our Proxy integration examples page
There are three types of IPs that we offer within our ISP Proxy Network:
-
Shared (pay per usage)
-
Shared unlimited (pay per proxy)
-
Dedicated unlimited (pay per proxy)
In all proxy types you can select the location of your proxies by zone configuration or using the -country parameter in your code/proxy user name.
Shared pay per usage
Rotating proxies from a pool of 10,000 proxies (IP addresses). Proxies are shared with others and charged by your usage: amount of GB you pass thru them.
Shared unlimited
Set of proxies, shared with others, paid by proxy. The more you buy, the less you pay per proxy. See our fair use policy for unlimited zones.
Dedicated unlimited
Set of proxies, exclusive to you, paid by proxy. The more you buy, the less you pay per proxy. See our fair use policy for unlimited zones.
You can find your IP list in the following pages:
-
Zone overview page: There are ‘Download’ and ‘View’ buttons below the code example.
-
Main zone table: There are ‘Download’ and ‘View’ icons under the ‘Allocated IPs’ column.
In both cases the buttons will do the same thing:
-
The view button will allow you to view, refresh and remove IPs from the pool.
-
The download button will download a csv file with the full list in
host:port:username:password
format.
Our proxy will automatically rotate thru the proxies assigned to this zone, and abide to location you specify. In case you want to have better control on driving a request thru a specific proxy (IP) you can use the -ip
or -gip
options. Read more about it here: Controlling your proxies. For advanced proxy rotation control you will need to install and route your requests using Bright Data proxy manager.
To integrate the proxies into your code, please visit the API examples page, which can be accessed via your zone’s settings:
On this page, you can choose integration examples for most modern coding languages, just choose the integration type, your proxy zone, coding language, etc. and the page will generate a code snippet you can use right away.
To test your proxy, use the terminal command that’s available in your Overview tab for the proxy.
Copy and paste it to your terminal. In Windows, click the ‘Start’ button and enter ‘cmd’. In Mac or Linux, run the ‘terminal’ application. Then, paste the code in the new window.
If your proxy is working well, you will see text on the console with details regarding your proxy.
If not, you will see an error code. You can always paste the error code in the AI agent integrated into the dashboard to get additional details.
Check out a few examples in the API examples page mentioned above (just choose “other software” in the “language” drop-down menu), or check our Integrations page, where we have specific guides to integrate our proxies within the most popular tools across the industry today.
Important note: If you are using Bright Data’s Web Unlocker, Residential Proxies or the SERP API you probably need to use our SSL certificate to enable end-to-end secure connections. See instructions here.
You can control the location of your country easily for every request that you send. You can choose proxies by country, state, city, zip code and ASN. In this answer, we will focus on choosing a specific country.
When sending your request, you can make you Proxy appear to be in a in a specific country by adding the -country
flag, after your zone’s name in the request, followed by the 2-letter ISO code for that country.
If you use a third-party tool or application, simply use the username that includes the country-xx
in the configuration. In other words, in the box where you need to put in the proxy username, enter the full string, including the country parameter, for example: brd-customer-<customer_id>-zone-<zone_name>-country-us
- don’t forget to use your own credentials that you can find in the “Overview” tab.
If you use your own code, see the example below: We added -country-us
to our request, so we will send a request originating from the United States (“us”).
curl "http://target.site" --proxy brd.superproxy.io:22225 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-country-us: <zone_password>
If you use a third-party application in conjunction with the proxies, you can enter the username, including the country to software’s configuration.
Of course, if you send requests to the proxy using code that you wrote, you can easily adjust the username to target specific countries as needed.
Remember, you can also choose proxies by country, state, city, zip code and ASN. See this guide for more info.
Some zones will give you access to a big fixed pool of IPs from all around the world. In these zones, you may select default countries to target without specifying the country in the request.
After selecting the default countries, the zone will target one of these countries in each request. You can still target other countries by adding the -country-xx
flag to your request and override the default country selection.
Bright data has Datacenter and ISP proxies in most countries around the world, but not all. We are constantly adding data centres in new countries as they become available.
If you cannot find the country you are looking for in Datacenter or ISP proxies, we recommend you check out Residential proxies. Since residential proxies are based on real people with real devices, we are able to offer residential proxies in every country in the world!
You can target the entire European Union region (member countries) in the same manner as “Country” above by adding “eu” after “country” in your request: -country-eu
Requests sent using -country-eu, will use IPs from a single, randomally selected country of the countries below which are included automatically within “eu”:
Member countries are: Austria, Belgium, Bulgaria, Croatia, Republic of Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden
When Allocating IPs to a Zone, in the country selection section in the configuration UI, you can select the ‘All except CN’ option, which allows you to allocate IPs from all around the world while excluding IPs from China.
Note this option is available only in the following zone types:
-
Data Center / ISP - Shared - Pay per proxy
-
Data Center / ISP - Dedicated
In order to target a specific city you will need to use a Shared (pay-per-IP) or Dedicated proxy network.
When configuring your proxy zone, after selecting your preferred country, you can add a city, to further specify your geotargeting
Once you click on ‘Add city’ you will be displayed the available cities to target, for the selected country.
Once configured, you’ll be able to send requests from the selected city, using the following syntax
curl "http://target.site" --proxy brd.superproxy.io:22225 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-country-<country>-city-<city>: <zone_password>
When selecting a two-word city it’ll be written as one word.
For example, when targeting Buenos Aires the syntax of the request will be:
-country-ar-city-buenosaires
To view more geotargeting examples, you may want to review our proxy integration examples page.
Bright Data offers proxies in every country in the world, except in the following countries:
-
Iran
-
Iraq
-
Syria
-
Lebanon
-
Palestine
-
North Korea
-
Cuba
-
Sudan
-
Crimea region of Russia
If you need proxies from there countries, unfortunately Bright Data will not be able to help you.
The event log will show you (at most) the last 200 requests you made with any zone in your account.
In your Bright Data control panel’s proxies page: https://brightdata.com/cp/zones
Go to the “Event Log” tab:
The presented data is:
-
Date: Time and date of the request
-
Zone: What Zone was used for the request
-
Source IP: What IP the request was made from
-
URL: The target site of the request
-
Result: Success or Fail of the request
The 100% uptime was built to prevent any ‘external’ events from affecting the user. The idea is simple and works the same way for both problems described above – if our system detects a problem, like a connectivity issue or an IP where the GEO isn’t exactly what you asked for when buying that IP, we will automatically route your requests through other IPs which are exactly the same as the original IPs.
At the same time, we wanted to make sure that customers that must use specific IPs will not be affected – so we made some exceptions:
-
If a specific IP is targeted in your request we will not assign a fallback IP to it
-
100% uptime will not interrupt a live connection. If the fallback is needed, it will play in once the next connection is established
The 100% uptime brings immediate value by providing 100% connectivity and continuous high-performance level, free of charge and without having to make any changes in your code or how you work.
100% uptime feature can be turned on or off, via API.
Navigate to your proxy configuration settings, and under Advanced settings enable ‘100% uptime’
- This can be done by adding the session flag to the proxy username:
brd.superproxy.io:22225 br-customer-<customer_id>-zone-<zone_name>-session-rand39484
Generate the random number on thread startup, and change it when you want to change the Proxy Peer assigned for the thread’s connection.
-
Session ID can be any random string/counter: requests with the same session string will use the same Proxy Peer (as long as possible); requests with different session strings will be assigned different Proxy Peers.
-
To force an IP change, just modify the session ID
-
If an assigned Proxy Peer(exit node IP) becomes unavailable, the Super Proxy will return an error “502 - No peers available” for the first request and then on the second request the super proxy will assign a new peer even if you do not change the session ID.
-
The Session IP is kept persistent for up to 1 minute of idle time. After a minute with no requests, the IP is released back to the pool.
To keep this Session/IP for longer, send a tiny keep-alive request every 30 seconds, to prevent this session from becoming idle for over a minute.
This request may be anything small, such as /favicon.ico, or even a request that returns 404 (as long as the web server does not disconnect the socket due to this request). -
If you have multiple Clients and would like to ignore your Clients source IP (which is used together with your session ID to create a session), then you want to use a global session then add
glob_
as a prefix to your session:
brd-customer-<customer_id>-zone-<zone_name>-session-glob_rand39484
Full request example:
brd-customer-CXXXXX-zone-ZONE_X-session-glob_rand39484
Generate the random number on thread startup, and change it when you want to change the Proxy Peer assigned for the thread’s connection.
You can find your proxy address and port in the “Overview” tab inside the proxies that you configured.
To do this, click on my zones, then click the line with the proxy you need.
On the overview tab you can also copy your proxy list, download it, refresh your proxies etc.
Whitelisting your IP is a good way to keep your account secure, as it prevents others from accessing your proxy even if they have your username and password.
When you whitelist an IP, only that IP will be allowed to send requests to your proxies. Note that any IP can connect to the configuration dashboard itself.
Remember, the fact that you whitelist an IP does not mean you can access your Bright data product from that IP without your username and password. You always need to use your username and password when accessing your Bright data product.
To add IPs to a zone’s blacklist/whitelist, there are 2 ways:
-
Via the control panel:
-
Go to any of your zone’s settings, and click the “Configuration” tab.
-
Scroll down to the “Security settings” section, as they are both responsible for white/blacklisting IPs and domains, respectively:
Add all the relevant IPs and domains you’ll allow access to with your proxy zone.
-
-
Via API endpoints:
-
Important tips re whitelist:
-
IPs that should be whitelisted are your machine’s IPs that you’ll be sending requests with, not the proxy IPs in your zones.
-
We strongly recommend whitelisting your IPs wherever possible, since when the whitelist is empty, you run the risk of getting your non-whitelisted IPs temporarily blocked in the case where our automatic security blocking system detects any irregular activity. See more info within this video.
-
There is no limit on how many IPs/domains you can add to the whitelist and we also support ranges of IPs.
Example for adding domains:
- Example for adding IPs:
-
HTTP and HTTPS Protocols
Protocols HTTP
& HTTPS
are supported by default.
SOCKS5 Protocol
Bright data supports SOCKS5
protocol, with a default port 22228 assigned for SOCKS5 communication.
SOCKS5 is supported for all Bright Data’s proxy networks: Datacenter, ISP, Residential, and Mobile.
See here for full SOCKS5 configuration instructions
Ports
Ports 80 and 443 are available by default in all zones, for HTTP and HTTPS protocols.
In zones of proxies of type Datacenter or ISP, all ports higher than 1024 are supported by default.
In zones of proxies of type Residential or Mobile, the following ports wills be available by default: 8080, 8443, 5678, 1962, 2000, 4443, 4433, 4430, 4444 and 1969.
Bright Data can support additional ports by request. Every request to support a new port will be followed by a dedicated and additional compliance process with the Bright Data compliance team.
Examples of ports that require Bright Data compliance review before activation:
To make a request to add a port permission to your zone:
-
Go to your zone’s settings (it will open on the “configuration” tab by default, if not, please click it)
-
Scroll down to “advanced options” and click it
-
Enable “ports”
-
Input the port numbers you would like to get approved
-
Fill out the form and wait for our compliance team to contact you and approve the request
Bright Data allows targeting the following Operating Systems:
If you’ve selected ‘Pay per IP’ shared, or a dedicated IP type in your zone configuration, you’ll be allocated a fixed IP address. From time to time, depending on your use case, you may need to refresh these IP addresses.
In order to refresh IPs allocated to your zone, navigate to your selected zone, under ‘Allocated IPs’ click on ‘Show allocated IPs’, check the box of the IP or gIP you wish to refresh and click ‘Refresh’
Alternatively, you can use API to refresh your dedicated Residential IPs or your Datacenter/ISP IPs
When using Bright Data’s Residential Proxy network, you may find the need to use a specific IP allocated to your zone.
- Send a test request with a ‘—verbose’ or ‘-v’ option added (this will turn on verbose logging)
curl "https://lumtest.com/myip.json" --verbose --proxy brd.superproxy.io:22225 --proxy-user brd-customer-<customer_id>-zone-<zone_name>:<zone_password>
- Locate the x-brd-ip response header and copy its value
-
Add the -ip- flag to your request, after your zone’s name and use the hashed IP value copied in the previous step
-
Send a test request, and review the response
curl "https://lumtest.com/myip.json" --proxy brd.superproxy.io:22225 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-ip-<hashed-ip>:<zone_password>
We recommend using https://lumtest.com/myip.json as the target domain for testing, and to review your IP credentials.
This feature can be enabled by adding the ASN parameter to your Zone configuration, under Geolocation Targeting.
Once the configuration is saved, the ASN flag can be added to the Zone’s credentials
and be integrated when using the Residential proxies. For example:
curl --proxy brd.superproxy.io:22225 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-asn-<asn-number>:<zone_password> "<target_site>"
Note: Values for ASN number can be found here.
Dedicated Residential IPs can be selected in the form of gIPs. They can be allocated under the zone’s configuration page by selecting a “Dedicated” IP type and choosing a number of gips. Also targeting a specific domain is required.
Once the configuration is saved, selecting “Show allocated Dedicated residential IPs” will provide
a list of hash values that represent group IPs.
These values can be used to target a specific gip. For example:
curl --proxy brd.superproxy.io:22225 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-gip-<gip_hash_value>:<zone_password> "<target_site>"
Long-Session Peers enables you to maintain a stable IP address for an extended period, ensuring seamless and consistent scraping operations. Unlike regular Residential IPs, where the IP can rotate at any moment if the peer unexpectedly disconnects, Long-Session Peers provide continuous sessions.
Enabling Long-Session Peers
Simply, go to an existing Residential zone or open a new one and toggle on the ‘Long-session peers’ feature. Now your future requests will only be routed through known long-lasting peers.
Note: This feature does add an extra cost for the zone when enabled
Where are These Peers From?
Our Residential peers’ pool consists of real users around the world who opt-in to use our product. As part of the opt-in, we utilize the IPs of these real users during their “down” periods as part of our ethical residential network. By enabling Long-Session Peers, you ensure that you will be routed through peers who are known to stay active for extended periods of time, and can maintain a single IP address for a much longer period of time in comparison to regular Residential peers.
Benefits of Long-Session Peers
Consistent scraping operations: This stability minimizes disruptions caused by IP changes and maximizes the efficiency of your scraping tasks. You can rely on a consistent IP address for an extended period, allowing for long-duration scraping operations. This feature is particularly useful for projects that require continuous data extraction or monitoring.
There are two ways to track usage:
Proxy Dashboard
Access the Main Proxy Dashboard here
-
Data Usage per Network: Shows total bandwidth and number of requests for each proxy product over the selected timeframe.
-
Usage Overview: Displays a graph where you can select timeframe, data point (bandwidth, requests, average bandwidth per request), and filter by zone, product, or target domain. You can also compare usage between timeframes using the “Compare to” option.
Zone Overview Page
- In the zone ‘Overview’ section you can view stats for a specific zone, with options to compare timeframes, choose data points (bandwidth, requests, bandwidth per request), and view additional metrics in a table below the graph.
When outside of China
Targeting Chinese Residential IP peers is enough:
curl --proxy brd.superproxy.io:22225 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-country-cn:<zone_password> "<target_site>"
When inside ChinaIn addition to Targeting Chinese Residentials IP peers, targeting our Chinese super-proxy is also required.
- When using API commands, the proxy DNS server isservercountry-cn.brd.superproxy.io
curl --proxy servercountry-cn.brd.superproxy.io:22225 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-country-cn:<zone_password> "<target_site>"
-
When using the Proxy Manager:
-
Select relevant Port settings by simply clicking on the port
-
Select Targeting and set China as the country
-
Select IP control: set Super Proxy location to “China”.
-
- You can choose to use a specific carrier from this list:
a1, aircel, airtel, att, celcom, chinamobile, claro, comcast, cox, digi,
dt, docomo, dtac, etisalat, idea, kyivstar, meo, megafon, mtn, mtnza, mts,
optus, orange, qwest, reliance_jio, robi, sprint, telefonica, telstra,
tmobile, tigo, tim, verizon, vimpelcom, vodacomza, vodafone, vivo, zain,
vivabo, telenormyanmar, kcelljsc, swisscom, singtel, asiacell, windit,
cellc, ooredoo, drei, umobile, cableone, proximus,tele2, mobitel, o2,
bouygues, free, sfr, digicel
- For Example
Geolocation databases (GeoDB) are used by internet websites to query information about the IP address used by the users. Bright Data monitors and maintains correct records of the 4 main GeoDBs used today: Maxmind, ip2location, db-ip and Google.
There are many other smaller GeoDBs, most of which are using outdated records or flawed testing methods, and so the information they present is not accurate or is presented to lure their viewers to buy VPN or proxy products from them. Therefore, we highly recommend using the GeoDBs mentioned above when testing your IPs.
In order to see our information about the proxy IP that you are using, browse one of the following:
For single-step scraping:
SERP API is the ideal product for targeting SERPs as it has a guaranteed success rate (pay only for success) with active unlocking, automatically chooses the best proxies, customizes headers, fingerprinting, solves CAPTCHAs, and more.
For multi-step scraping (playwright/puppeteer/selenium):
Scraping Browser is the ideal product as it is our fully cloud-hosted browser designed to help you easily focus on your multi-step data collection while we take care of the full proxy and unblocking infrastructure for you, including CAPTCHA solving.
Residential Proxy - No, SERP API is the ideal product for targeting SERPs as it has a guaranteed success rate (pay only for success) with active unlocking, automatically chooses the best proxies, customizes headers, fingerprinting, solves CAPTCHAs, and more. Targeting Google SERPs from the Residential network will result in Super-proxy bypass, which will casue the request to be sent from our servers instead of the peer.
Datacenter & ISP Proxies - No. When attempting to specifically target Google through either of these proxy networks, your request will be denied and you will receive the following error message in the response headers:
HTTP/1.1 403 Search engine host is not allowed
X-Luminati-Error: Forbidden: This target URL isn't supported on proxy networks, use the SERP API product for targeting this URL. You may contact your account manager or open a support ticket for assistance
When targeting search engines through the Residential or Mobile networks, the request will not go through the Residential or Mobile peer but instead will be sent directly through one of our super proxies. The reason for this behavior is to support the integration of the Residential and Mobile networks with a browser, which might need to load search engine resource endpoints from the target website.
In case you target a search engine domain using Residential or Mobile networks, the request will be passed directly through the super proxy, and the following response header will serve as an indication:
x-luminati-ip: superproxy bypass
No - requests directly to the host server, and not to the domain name are forbidden while using Bright data.
Requests using URL format such as 1.1.1.1:443 will be executed using the super proxy server, not the proxy peer IP.
Example of a request using the super proxy:
Since BrightData cares about our community and peers quality, SMTP requests which can be used for spamming are blocked. Please note that mail domains are also blocked from access, and requests to mail domains will be sent from the super proxy server, and not the peer IP.
Yes. Bright data supports SOCKS5
protocol, with a default port 22228 assigned for SOCKS5 communication.
See here for full SOCKS5 configuration instructions or visit the SOCKS5 proxies page.
- You can choose to perform the request from the super proxy directly instead of the IP of the peer. In that case the IP of the request will be the one of the Super proxy. You will need to add ‘-direct’ to your request authorization string.
brd-customer-<customer_id>-zone-<zone_name>-direct
If you want to use multiple countries and you don’t want to create a separate proxy port for each country it’s also possible to control the targeting dynamically.
Using the same IP for a long period of time makes it easier for the target website to mark the IP as proxy and can make your request get detectable by the target website. Refreshing your IPs will result in allocation of new IPs from Bright Data instead of your existing ones in your pool allowing you to gain control over your pool and reach higher success rates.
This section is relevant ONLY if you are in China. You must use our Chinese Super Proxy:
-
When using Proxy Manager: on Port settings > IP control: set Super Proxy location to “China”.
-
When using API commands, use servercountry-cn.brd.superproxy.io
Example:
curl "https://target-chinese-website.com" --proxy servercountry-cn.brd.superproxy.io:22225 --proxy-user brd-customer-<custormer_name>-zone-<zone_name>-country-cn:<zone_password>
A residential/mobile IP are real users’ devices’ IPs, and therefore can be used only when the user’s device is idle (i.e. the device is connected to the internet, has enough battery power, and the user is not currently using it).
If the IP becomes unavailable, our system will automatically assign you with another available residential IP of the same type and geo-location as you used.
HTTP error code 502 means “Bad Gateway”. This error occurs when you send a request to a URL, but the URL’s server receives an invalid response from another server that it depends on to fulfill the request.
Possible causes
-
We currently do have not matching peers for your zone’s configuration, as they might already be in use with a different client, or unavailable for other reasons.
-
The connection has been timed out due to inactivity.
-
The connection has been closed by the host (target URL) side
Sample error 502 response
Request URL: SAMPLE_URL Request method: GET ***Status code: 502***
Super proxy IP: IP
Username: brd-customer-customer_id-zone-zone_name-country
Password: zone_password
Sent from: 127.0.0.1
Response headers
connection: keep-alive
date: Tue, 14 Feb 2023 08:49:16 GMT
keep-alive: timeout=5
transfer-encoding: chunked
x-lpm-port: 24000
x-luminati-error: waiting for selector ".price-container" failed: timeout 90000ms exceeded
***x-luminati-error-code: expect_element***
x-request-id2: REQUEST_ID
Request headers
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36(KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36
host: SAMPLE_HOST
accept-encoding: gzip, deflate
Connection: keep-alive
HTTP 403 response code means you are forbidden from accessing a valid URL.
The server understands the request, but it can’t fulfill the request because of client-side issues.
Possible causes
-
Your machine IP is not whitelisted. In order to learn how to whitelist an IP, please review the following article.
-
The domain you try to access is blocked by Bright Data for compliance reasons.
-
The domain you try to access is not specified in your use case.
-
Failed captcha response (relevant for Web Unlocker and SERP API).
-
The domain you tried to access is blocked by the Target host settings of your zone (check your proxy zone’s credentials)
Sample error 403 response
< HTTP/1.1 403 response status was rejected
< x-debug-req-id: SAMPLE_ID
< X-Luminati-Error: response status was rejected: 403 Forbidden
< X-Luminati-Error-Code: http_status
< Date: Wed, 22 Feb 2023 11:11:23 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< Transfer-Encoding: chunked
The 403 X-luminati-error (Forbidden) indicates you are targeting a domain that is not allowed by Bright Data
This can be resolved by contacting the Bright data compliance team for approval.
As the Bright data compliance team reviews your use case and confirms - you will be informed that access is been added.
For better handling, it is advised to add your account manager to the ticket and inform him as well for your requirement
When error message:
You tried to target <host> but got blocked
Appears, contact the Bright data compliance team (compliance@brightdata.com) or from the ticketing system
Our compliance team will review your request to gain access to the particular domain
If you’ve tried using the residential or mobile proxy networks and have encounterred one of the following errors or ones similar to them:
-
net::ERR_SSL_PROTOCOL_ERROR
-
ERR_INVALID_CERT
-
Error: self-signed certificate in certificate chain
-
ERROR: No matching issuer found
-
NET::ERR_CERT_AUTHORITY_INVALID
-
SSL: CERTIFICATE_VERIFY_FAILED
It means that you are using the residential/mobile networks in ‘Immediate Access’ mode, and have not handled the SSL verification correctly
For more info on what ‘Immediate Access’ mode means: Immediate Access Explanation
How to resolve the SSL errors?
You have three options:
-
Complete the KYC process, which takes ~2-3 business days to process - after which you’ll be able to use the residential/mobile network with minimal restrictions, which also resolves the SSL errors.
To complete KYC process fill in the KYC form -
Install the BrightData SSL certificate, according to our SSL certificate installation guide (note that for some specific integration methods installing the ssl certificate may not be possible due to it not being supported)
-
If you are utilizing our proxies through code (NodeJS, Python, C#, etc.) you can ignore SSL verification entirely, which resolves all SSL verification errors. We provide examples on how to do this in the SSL certificate documentation page
When scraping large files or in order to record curl
output, you may want to redirect the output to a file. In order to do so add --output [filename]
to your curl
command options.
You can control access by allowing or blocking specific domains through simple configurations within your zones. Follow these steps to manage domain access:
-
Visit the Zones Page Go to the Zones Page in your dashboard.
-
Select Your Desired Zone Choose the zone you wish to configure for domain access.
-
Navigate to the ‘Configuration’ Tab Once in your selected zone, locate the Configuration tab, then scroll down to Security Options.
-
Allowing or Blocking Domains
-
To allow specific domains, add them to Allowed Target Hosts.
-
To block specific domains, add them to Blocked Target Hosts.
-
-
Domain Wildcard Options
-
Root Domain Inclusion: Adding a root domain (e.g., example.com) will automatically include all its subdomains (e.g., sub.example.com).
-
Wildcard Use: Utilize the * wildcard to cover all subdomains and suffix variations. For instance, adding example.* will include sub.example.* and various domain suffixes like example.com, example.co.uk, etc.
-
These configurations provide flexibility in managing domain-level access for your proxy zones, ensuring secure and controlled connectivity.
Bright Data policy allows targeting government website after passing the KYC process. Read more about KYC process.
Was this page helpful?