Dataceter Proxy FAQ

There are four types of IPs that we offer within our Datacenter Proxy Network:

1. Shared (Pay per usage)

2. Shared (Pay per IP)

3. Dedicated

• Shared
• Dedicated

There are three types of IPs that we offer within our ISP Proxy Network:

1. Shared (Pay per usage)

2. Shared (Pay per IP)

3. Dedicated

• Shared
• Dedicated

To integrate the proxies into your code, please visit the API examples page, which can be accessed via your zone’s settings, under “access parameters”:

On this page, you can choose integration examples for most modern coding languages, just choose the integration type, your proxy zone, coding language, etc. and the page will generate a code snippet you can use right away.

Check out a few examples in the API examples page mentioned above (just choose “other software” in the “language” drop-down menu), or check our Integrations page, where we have specific guides to integrate our proxies within the most popular tools across the industry today.

Important note: If you are using Bright Data’s Web Unlocker, Residential Proxies or the SERP API you probably need to use our SSL certificate to enable end-to-end secure connections. See instructions here.

When sending your request, add the -country flag, after your zone’s name in the request, followed by the 2-letter ISO code for that country.

In the example below: We added -country-us to our request, so we will send a request originating from the United States (“us”).

curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-country-us: <zone_password>

In order to target a specific city you will need to use a Shared (pay-per-IP) or Dedicated proxy network.

When configuring your proxy zone, after selecting your preferred country, you can add a city, to further specify your geotargeting

Once you click on ‘Add city’ you will be displayed the available cities to target, for the selected country. In order to check the number of available IPs for a specific geolocation.

Once configured, you’ll be able to send requests from the selected city, using the following syntax

curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-country-<country>-city-<city>: <zone_password>

To view more geotargeting examples, you may want to review our proxy integration examples page.

Bright Data offers proxies in every country in the world, except in the following countries:

• Iran

• Iraq

• Syria

• Lebanon

• Palestine

• North Korea

• Cuba

• Sudan

• Crimea region of Russia

If you need proxies from there countries, unfortunately Bright Data will not be able to help you.

The event log will show you (at most) the last 200 requests you made with any zone in your account.

In your Bright Data control panel’s proxies page: https://brightdata.com/cp/zones

Go to the “Event Log” tab:

The presented data is:

• Date: Time and date of the request

• Zone: What Zone was used for the request

• Source IP: What IP the request was made from

• URL: The target site of the request

• Result: Success or Fail of the request

The 100% uptime was built to prevent any ‘external’ events from affecting the user. The idea is simple and works the same way for both problems described above – if our system detects a problem, like a connectivity issue or an IP where the GEO isn’t exactly what you asked for when buying that IP, we will automatically route your requests through other IPs which are exactly the same as the original IPs.

At the same time, we wanted to make sure that customers that must use specific IPs will not be affected – so we made some exceptions:

• If a specific IP is targeted in your request we will not assign a fallback IP to it

• 100% uptime will not interrupt a live connection. If the fallback is needed, it will play in once the next connection is established

The 100% uptime brings immediate value by providing 100% connectivity and continuous high-performance level, free of charge and without having to make any changes in your code or how you work.

Navigate to your proxy configuration settings, and under Advanced settings enable ‘100% uptime’

• This can be done by adding the session flag to the proxy username:

brd.superproxy.io:33335 br-customer-<customer_id>-zone-<zone_name>-session-rand39484

Generate the random number on thread startup, and change it when you want to change the Proxy Peer assigned for the thread’s connection.

• Session ID can be any random string/counter: requests with the same session string will use the same Proxy Peer (as long as possible); requests with different session strings will be assigned different Proxy Peers.

• To force an IP change, just modify the session ID

• If an assigned Proxy Peer(exit node IP) becomes unavailable, the Super Proxy will return an error “502 - No peers available” for the first request and then on the second request the super proxy will assign a new peer even if you do not change the session ID.

• The Session IP is kept persistent for up to 1 minute of idle time. After a minute with no requests, the IP is released back to the pool.
  To keep this Session/IP for longer, send a tiny keep-alive request every 30 seconds, to prevent this session from becoming idle for over a minute.
  This request may be anything small, such as /favicon.ico, or even a request that returns 404 (as long as the web server does not disconnect the socket due to this request).

• If you have multiple Clients and would like to ignore your Clients source IP (which is used together with your session ID to create a session), then you want to use a global session then add glob_ as a prefix to your session:

brd-customer-<customer_id>-zone-<zone_name>-session-glob_rand39484

Full request example:

brd-customer-CXXXXX-zone-ZONE_X-session-glob_rand39484

Generate the random number on thread startup, and change it when you want to change the Proxy Peer assigned for the thread’s connection.

To add IPs to a zone’s blacklist/whitelist, there are 2 ways:

• Via the control panel:

  • Go to any of your zone’s settings, and click the “access parameters” tab.

  • Scroll down to the “IPs” and “target host” sections, as they are both responsible for white/blacklisting IPs and domains, respectively:

  Add all the relevant IPs and domains you’ll allow access to with your proxy zone.

• Via API endpoints:

  • Add IP to Zone whitelist

  • Remove IP from Zone whitelist

  • Add IP to Zone blacklist

  • Remove IP from Zone blacklist

  • Remove domain from Zone whitelist/blacklist

  • Add domain to Zone whitelist/blacklist

• Important tips re whitelist:

  • IPs that should be whitelisted are your machine’s IPs that you’ll be sending requests with, not the proxy IPs in your zones.

  • We strongly recommend whitelisting your IPs wherever possible, since when the whitelist is empty, you run the risk of getting your non-whitelisted IPs temporarily blocked in the case where our automatic security blocking system detects any irregular activity. See more info within this video.

  • There is no limit on how many IPs/domains you can add to the whitelist and we also support ranges of IPs.

  Example for adding domains:

  • Example for adding IPs:

Ports 80 and 443 are available by default in all zones, supporting HTTP and HTTPS protocols.

In zones of proxies of type Datacenter or ISP, all ports higher than 1024 are supported by default.

In zones of proxies of type Residential or Mobile, the following ports wills be available by default: 8080, 8443, 5678, 1962, 2000, 4443, 4433, 4430, 4444 and 1969.

Bright Data can support additional ports by request. Every request to support a new port will be followed by a dedicated and additional compliance process with the Bright Data compliance team.
Examples of ports that require Bright Data compliance review before activation:

To make a request to add a port permission to your zone:

• Go to your zone’s settings (it will open on the “configuration” tab by default, if not, please click it)

• Scroll down to “advanced options” and click it

• Enable “ports”

• Input the port numbers you would like to get approved

• Fill out the form and wait for our compliance team to contact you and approve the request

Bright Data allows targeting the following Operating Systems:

If you’ve selected ‘Pay per IP’ shared, or a dedicated IP type in your zone configuration, you’ll be allocated a fixed IP address. From time to time, depending on your use case, you may need to refresh these IP addresses.

In order to refresh IPs allocated to your zone, navigate to your selected zone, under ‘Allocated IPs’ click on ‘Show allocated IPs’, check the box of the IP or gIP you wish to refresh and click ‘Refresh’

Alternatively, you can use API to refresh your dedicated Residential IPs or your Datacenter/ISP IPs

When using Bright Data’s Residential Proxy network, you may find the need to use a specific IP allocated to your zone.

1. Send a test request with a ‘—verbose’ or ‘-v’ option added (this will turn on verbose logging)

curl "https://lumtest.com/myip.json" --verbose --proxy brd.superproxy.io:33335 --proxy-user brd-customer-<customer_id>-zone-<zone_name>:<zone_password>

2. Locate the x-brd-ip response header and copy its value

3. Add the -ip- flag to your request, after your zone’s name and use the hashed IP value copied in the previous step

4. Send a test request, and review the response

curl "https://lumtest.com/myip.json" --proxy brd.superproxy.io:33335 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-ip-<hashed-ip>:<zone_password>

We recommend using https://lumtest.com/myip.json as the target domain for testing, and to review your IP credentials.

This feature can be enabled by adding the ASN parameter to your Zone configuration, under Geolocation Targeting.

Once the configuration is saved, the ASN flag can be added to the Zone’s credentials
and be integrated when using the Residential proxies. For example:

curl --proxy brd.superproxy.io:33335 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-asn-<asn-number>:<zone_password> "<target_site>"

Note: Values for ASN number can be found here.

Dedicated Residential IPs can be selected in the form of gIPs. They can be allocated under the zone’s configuration page by selecting a “Dedicated” IP type and choosing a number of gips. Also targeting a specific domain is required.

Once the configuration is saved, selecting “Show allocated Dedicated residential IPs” will provide
a list of hash values that represent group IPs.

These values can be used to target a specific gip. For example:

curl --proxy brd.superproxy.io:33335 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-gip-<gip_hash_value>:<zone_password> "<target_site>"

Long-Session Peers enables you to maintain a stable IP address for an extended period, ensuring seamless and consistent scraping operations. Unlike regular Residential IPs, where the IP can rotate at any moment if the peer unexpectedly disconnects, Long-Session Peers provide continuous sessions.

Enabling Long-Session Peers

Simply, go to an existing Residential zone or open a new one and toggle on the ‘Long-session peers’ feature. Now your future requests will only be routed through known long-lasting peers.
Note: This feature does add an extra cost for the zone when enabled

Where are These Peers From?

Our Residential peers’ pool consists of real users around the world who opt-in to use our product. As part of the opt-in, we utilize the IPs of these real users during their “down” periods as part of our ethical residential network. By enabling Long-Session Peers, you ensure that you will be routed through peers who are known to stay active for extended periods of time, and can maintain a single IP address for a much longer period of time in comparison to regular Residential peers.

Benefits of Long-Session Peers

Consistent scraping operations: This stability minimizes disruptions caused by IP changes and maximizes the efficiency of your scraping tasks. You can rely on a consistent IP address for an extended period, allowing for long-duration scraping operations. This feature is particularly useful for projects that require continuous data extraction or monitoring.

When outside of China
Targeting Chinese Residential IP peers is enough:

curl --proxy brd.superproxy.io:33335 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-country-cn:<zone_password> "<target_site>"

When inside ChinaIn addition to Targeting Chinese Residentials IP peers, targeting our Chinese super-proxy is also required.

• When using API commands, the proxy DNS server isservercountry-cn.brd.superproxy.io

curl --proxy servercountry-cn.brd.superproxy.io:33335 --proxy-user brd-customer-<customer_id>-zone-<zone_name>-country-cn:<zone_password> "<target_site>"

• When using the Proxy Manager:

  • Select relevant Port settings by simply clicking on the port
    

  • Select Targeting and set China as the country
    

  • Select IP control: set Super Proxy location to “China”.

• You can choose to use a specific carrier from this list:

a1, aircel, airtel, att, celcom, chinamobile, claro, comcast, cox, digi, 
dt, docomo, dtac, etisalat, idea, kyivstar, meo, megafon, mtn, mtnza, mts,
optus, orange, qwest, reliance_jio, robi, sprint, telefonica, telstra, 
tmobile, tigo, tim, verizon, vimpelcom, vodacomza, vodafone, vivo, zain,
vivabo, telenormyanmar, kcelljsc, swisscom, singtel, asiacell, windit, 
cellc, ooredoo, drei, umobile, cableone, proximus,tele2, mobitel, o2, 
bouygues, free, sfr, digicel

• For Example

Geolocation databases (GeoDB) are used by internet websites to query information about the IP address used by the users. Bright Data monitors and maintains correct records of the 4 main GeoDBs used today: Maxmind, ip2location, db-ip and Google.

There are many other smaller GeoDBs, most of which are using outdated records or flawed testing methods, and so the information they present is not accurate or is presented to lure their viewers to buy VPN or proxy products from them. Therefore, we highly recommend using the GeoDBs mentioned above when testing your IPs.

In order to see our information about the proxy IP that you are using, browse one of the following:

• https://lumtest.com/echo.json

• https://www.iplocation.net

• https://www.maxmind.com/en/geoip-demo

For single-step scraping:

SERP API is the ideal product for targeting SERPs as it has a guaranteed success rate (pay only for success) with active unlocking, automatically chooses the best proxies, customizes headers, fingerprinting, solves CAPTCHAs, and more.

For multi-step scraping (playwright/puppeteer/selenium):

Scraping Browser is the ideal product as it is our fully cloud-hosted browser designed to help you easily focus on your multi-step data collection while we take care of the full proxy and unblocking infrastructure for you, including CAPTCHA solving.

Residential Proxy - No, SERP API is the ideal product for targeting SERPs as it has a guaranteed success rate (pay only for success) with active unlocking, automatically chooses the best proxies, customizes headers, fingerprinting, solves CAPTCHAs, and more. Targeting Google SERPs from the Residential network will result in Super-proxy bypass, which will casue the request to be sent from our servers instead of the peer.

Datacenter & ISP Proxies - No. When attempting to specifically target Google through either of these proxy networks, your request will be denied and you will receive the following error message in the response headers:

HTTP/1.1 403 Search engine host is not allowed
X-Luminati-Error: Forbidden: This target URL isn't supported on proxy networks, use the SERP API product for targeting this URL. You may contact your account manager or open a support ticket for assistance

No - requests directly to the host server, and not to the domain name are forbidden while using Bright data.
Requests using URL format such as 1.1.1.1:443 will be executed using the super proxy server, not the proxy peer IP.

Example of a request using the super proxy:

Since BrightData cares about our community and peers quality, SMTP requests which can be used for spamming are blocked. Please note that mail domains are also blocked from access, and requests to mail domains will be sent from the super proxy server, and not the peer IP.

Refreshing data-center IPs will have a different cost depending on the type of IP.

• Refreshing dedicated data-center IPs: $0.5/refresh/IP

• Refreshing dedicated domain data-center IPs: $0.04/refresh/IP

• Refreshing shared data-center IPs: $0.02/refresh/IP

Yes. Bright data supports SOCKS5 protocol, with a default port 22228 assigned for SOCKS5 communication.

See here for full SOCKS5 configuration instructions

• You can choose to perform the request from the super proxy directly instead of the IP of the peer. In that case the IP of the request will be the one of the Super proxy. You will need to add ‘-direct’ to your request authorization string. brd-customer-<customer_id>-zone-<zone_name>-direct

Using the same IP for a long period of time makes it easier for the target website to mark the IP as proxy and can make your request get detectable by the target website. Refreshing your IPs will result in allocation of new IPs from Bright Data instead of your existing ones in your pool allowing you to gain control over your pool and reach higher success rates.

• If you are in China
• If you are not in China

curl "https://target-chinese-website.com" --proxy servercountry-cn.brd.superproxy.io:33335 --proxy-user brd-customer-<custormer_name>-zone-<zone_name>-country-cn:<zone_password>

If the IP becomes unavailable, our system will automatically assign you with another available residential IP of the same type and geo-location as you used.

HTTP error code 502 means “Bad Gateway”. This error occurs when you send a request to a URL, but the URL’s server receives an invalid response from another server that it depends on to fulfill the request.

Possible causes

• We currently do have not matching peers for your zone’s configuration, as they might already be in use with a different client, or unavailable for other reasons.

• The connection has been timed out due to inactivity.

• The connection has been closed by the host (target URL) side

Sample error 502 response

Request URL: SAMPLE_URL Request method: GET ***Status code: 502***  
Super proxy IP: IP  
Username: brd-customer-customer_id-zone-zone_name-country  
Password: zone_password  
Sent from: 127.0.0.1  
Response headers  
connection: keep-alive  
date: Tue, 14 Feb 2023 08:49:16 GMT  
keep-alive: timeout=5  
transfer-encoding: chunked  
x-lpm-port: 24000  
x-luminati-error: waiting for selector ".price-container" failed: timeout 90000ms exceeded  
***x-luminati-error-code: expect_element***  
x-request-id2: REQUEST_ID  
Request headers  
user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36(KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36  
host: SAMPLE_HOST  
accept-encoding: gzip, deflate  
Connection: keep-alive

HTTP 403 response code means you are forbidden from accessing a valid URL.

The server understands the request, but it can’t fulfill the request because of client-side issues.

Possible causes

• Your machine IP is not whitelisted. In order to learn how to whitelist an IP, please review the following article.

• The domain you try to access is blocked by Bright Data, usually for compliance reasons.

• The domain you try to access is not specified in your use case.

• Failed captcha response (relevant for Web Unlocker and SERP API).

• The domain you tried to access is blocked by the Target host settings of your zone (check your proxy zone’s Access Parameters)

Sample error 403 response

< HTTP/1.1 403 response status was rejected  
< x-debug-req-id: SAMPLE_ID  
< X-Luminati-Error: response status was rejected: 403 Forbidden  
< X-Luminati-Error-Code: http_status  
< Date: Wed, 22 Feb 2023 11:11:23 GMT  
< Connection: keep-alive  
< Keep-Alive: timeout=5  
< Transfer-Encoding: chunked

The 403 X-luminati-error (Forbidden) indicates you are targeting a domain that is not allowed by Bright Data 
This can be resolved by contacting the Bright data compliance team for approval.
As the Bright data compliance team reviews your use case and confirms - you will be informed that access is been added.
For better handling, it is advised to add your account manager to the ticket and inform him as well for your requirement 

When error message:

You tried to target <host> but got blocked

Appears, contact the Bright data compliance team (compliance@brightdata.com) or from the ticketing system 

Our compliance team will review your request to gain access to the particular domain