# Account Management Source: https://docs.brightdata.com/account-management-quickstart # Python SDK: Launch Requests From Your IDE Source: https://docs.brightdata.com/api-reference/SDK Quickly integrate the Python SDK to send web unlocking and data extraction requests directly from your IDE with minimal setup. ### Install the package Open the terminal and run: ```python pip install brightdata-sdk ``` Ensure `pip` is installed on your machine (for macOS first open a virtual env) **In your code file**, import the package: ```python from brightdata import bdclient ``` In your IDE, hover over the `brightdata` package or **any of its functions** to view available methods, parameters, and usage examples. Hover Over1 Jp ### Start making requests Try these examples to use Bright Data's SDK functions from your IDE ```python search(...) from brightdata import bdclient client = bdclient(api_token="your-api-token") # Can also be taken from .env file results = client.search(query=["pizza"]) # Try adding parameters like: search_engine="bing"/"yandex", country="gb" print (results) # save this code as Search.py in the same folder, # then run it by pasting "python Search.py" in your terminal. ``` ```python scrape(...) from brightdata import bdclient client = bdclient(api_token="your-api-token") # can also be taken from .env file URL = ["https://docs.brightdata.com/api-reference/SDK"] # Add more to scrape multiple websites concurrently results = client.scrape(url=URL) # Hover over the function to discover additional parameters bd.download_content(results, filename="scrape_results.json") # save this code as Scrape.py in the same folder, # then run it by pasting "python Scrape.py" in your terminal. ``` When working with multiple queries or URLs, requests are handled concurrently for optimal performance. ### Use LinkedIn Scrapers to get structured response Search LinkedIn to get all the relevant data for your search, response is structured data ```python search_linkedin(...) expandable # Add the following function after package import and setting the client # Search LinkedIn profiles by name first_names = ["James", "Idan"] last_names = ["Smith", "Vilenski"] result = client.search_linkedin.profiles(first_names, last_names) # Search jobs by URL job_urls = [ "https://www.linkedin.com/jobs/search?keywords=Software&location=Tel%20Aviv-Yafo", "https://www.linkedin.com/jobs/reddit-inc.-jobs-worldwide?f_C=150573" ] result = client.search_linkedin.jobs(url=job_urls) # Search jobs by keyword and location result = client.search_linkedin.jobs( location="Paris", keyword="product manager", country="FR", time_range="Past month", job_type="Full-time" ) # Search posts by profile URL with date range result = client.search_linkedin.posts( profile_url="https://www.linkedin.com/in/bettywliu", start_date="2018-04-25T00:00:00.000Z", end_date="2021-05-25T00:00:00.000Z" ) # Search posts by company URL result = client.search_linkedin.posts( company_url="https://www.linkedin.com/company/bright-data" ) # Returns snapshot ID that can be used to download the content later using download_snapshot function ``` ```python scrape_linkedin(...) # Add the following function after package import and setting the client # LinkedIn Profile URLs profile_url = "https://www.linkedin.com/in/elad-moshe-05a90413/" # LinkedIn Company URLs company_urls = [ "https://il.linkedin.com/company/ibm", "https://www.linkedin.com/company/bright-data", "https://www.linkedin.com/company/stalkit" ] # LinkedIn Job URLs job_urls = [ "https://www.linkedin.com/jobs/view/remote-typist-%E2%80%93-data-entry-specialist-work-from-home-at-cwa-group-4181034038?trk=public_jobs_topcard-title", "https://www.linkedin.com/jobs/view/arrt-r-at-shared-imaging-llc-4180989163?trk=public_jobs_topcard-title" ] # LinkedIn Post URLs post_urls = [ "https://www.linkedin.com/posts/orlenchner_scrapecon-activity-7180537307521769472-oSYN?trk=public_profile", "https://www.linkedin.com/pulse/getting-value-out-sunburst-guillaume-de-b%C3%A9naz%C3%A9?trk=public_profile_article_view" ] # The scrape function results = client.scrape_linkedin.posts(post_urls) # can also be changed to async print(results) ``` ### Use ChatGPT to collect any number of responses ```python expandable # Add the following function after package import and setting the client result = client.search_chatGPT( prompt="what day is it today?" # prompt=["What are the top 3 programming languages in 2024?", "Best hotels in New York", "Explain quantum computing"], # additional_prompt=["Can you explain why?", "Are you sure?", ""] ) client.download_content(result) # In case of timeout error, your snapshot_id is presented and you will downloaded it using download_snapshot() ``` ### Parameters ```python client = bdclient( api_token="your_token", auto_create_zones=False, # Else it creates Zone automatically web_unlocker_zone="custom_zone", serp_zone="custom_serp_zone" ) ``` Search the web using the SERP API ```python - `query`: Search query string or list of queries - `search_engine`: "google", "bing", or "yandex" - `zone`: Zone identifier (auto-configured if None) - `format`: "json" or "raw" - `method`: HTTP method - `country`: Two-letter country code - `data_format`: "markdown", "screenshot", etc. - `async_request`: Enable async processing - `max_workers`: Max parallel workers (default: 10) - `timeout`: Request timeout in seconds (default: 30) ``` Scrape a single URL or list of URLs using the Web Unlocker API. ```python - `url`: Single URL string or list of URLs - `zone`: Zone identifier (auto-configured if None) - `format`: "json" or "raw" - `method`: HTTP method - `country`: Two-letter country code - `data_format`: "markdown", "screenshot", etc. - `async_request`: Enable async processing - `max_workers`: Max parallel workers (default: 10) - `timeout`: Request timeout in seconds (default: 30) ``` ### Error handling Enable `VERBOSE` in the Client for advenced logging (see Client parameters) Use the `list_zones()` func to check available zones Create a [Bright Data](https://brightdata.com/) account and copy your API key Go to [account settings](https://brightdata.com/cp/setting/users), and make sure that your API key have **admin permissions** ### Resources Visit the Bright Data SDK GitHub repository # Add Static IP(s) Source: https://docs.brightdata.com/api-reference/account-management-api/Add_Static_Datacenter_ISP_IPs POST /zone/ips Add Static IPs for Datacenter and ISP **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Add a Zone Source: https://docs.brightdata.com/api-reference/account-management-api/Add_a_Zone POST /zone Add a new zone **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Change Account Status Source: https://docs.brightdata.com/api-reference/account-management-api/Change_account_status PUT /status **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Get Available Countries Source: https://docs.brightdata.com/api-reference/account-management-api/Get_Available_Countries GET /countrieslist List all available countries per zone type Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Zone Statistics Source: https://docs.brightdata.com/api-reference/account-management-api/Get_Zone_Static_Datacenter_ISP_IPs GET /zone/ips Get Zone Static (Datacenter/ISP) IPs Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) ```json Response when "ip_per_country=true" { "gb":198, "de":282, "br":418, "au":115, "jp":292, "nl":421, "uz":333, "il":517, "kg":566, "az":498, "lv":484, "tw":372, "sg":184 } ``` # Zone info Source: https://docs.brightdata.com/api-reference/account-management-api/Get_Zone_info GET /zone Get Zone status Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Zone Password Source: https://docs.brightdata.com/api-reference/account-management-api/Get_Zone_passwords GET /zone/passwords Get Zone passwords Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Zone Permission Source: https://docs.brightdata.com/api-reference/account-management-api/Get_Zone_permissions GET /zone/permissions Get Zone permissions Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Zone status Source: https://docs.brightdata.com/api-reference/account-management-api/Get_Zone_status GET /zone/status Get Zone status Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Account status Source: https://docs.brightdata.com/api-reference/account-management-api/Get_account_status GET /status Get account status Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Active Zones Source: https://docs.brightdata.com/api-reference/account-management-api/Get_active_Zones GET /zone/get_active_zones Return all active zones Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Get available dedicated IPs per Zone Source: https://docs.brightdata.com/api-reference/account-management-api/Get_all_available_dedicated_residential_IPs_per_Zone GET /zone/route_vips Get all available residential dedicated IPs per Zone Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) Only works with residential dedicated zone # Amount of available Datacenter and ISP IPs Source: https://docs.brightdata.com/api-reference/account-management-api/Get_amount_of_available_Datacenter_and_ISP_IPs GET /zone/count_available_ips Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) * Available IPs for current Zone plan: ```sh curl "https://api.brightdata.com/count_available_ips?zone=ZONE" -H "Authorization: Bearer API_KEY" ``` * Abstract plan, dedicated IPs: ```sh dedicated IPs curl "https://api.brightdata.com/count_available_ips?plan=\{\"ips_type\":\"dedicated\"\}" -H "Authorization: Bearer API_KEY" ``` * Abstract plan, shared IPs located in United States: ```sh curl "https://api.brightdata.com/count_available_ips?plan=\{\"country\":\"us\",\"ips_type\":\"shared\"\}" -H "Authorization: Bearer API_KEY" ``` * Abstract plan, dedicated ips located in United States: ```sh curl "https://api.brightdata.com/count_available_ips?plan=\{\"country\":\"us\",\"ips_type\":\"dedicated\"\}" -H "Authorization: Bearer API_KEY" ``` * Abstract plan, shared IPs located in United States, Denver: ```sh curl "https://api.brightdata.com/count_available_ips?plan=\{\"ips_type\":\"shared\",\"country_city\":\"us-denver\",\"city\":true\}" -H "Authorization: Bearer API_KEY" ``` * Abstract plan, shared IPs located in US, exclusive for domains: amazon.com, fb.com: ```sh curl "https://api.brightdata.com/count_available_ips?plan=\{\"ips_type\":\"selective\",\"country\":\"us\",\"domain_whitelist\":\"amazon.com%20fb.com\"\}" -H "Authorization: Bearer API_KEY" ``` * Abstract plan, shared IPs located in US, geo IP databases: should persist in both maxmind and dbip: ```sh curl "https://api.brightdata.com/count_available_ips?plan=\{\"ips_type\":\"shared\",\"country\":\"us\",\"geo_db\":\{\"maxmind\":true,\"dbip\":true\}\}" -H "Authorization: Bearer API_KEY" ``` # Get Country Cities Source: https://docs.brightdata.com/api-reference/account-management-api/Get_country_cities GET /cities Get a list of cities from a Country Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Get current service status Source: https://docs.brightdata.com/api-reference/account-management-api/Get_current_service_status GET /network_status/{NETWORK_TYPE} Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # List all available country's cities (static network) Source: https://docs.brightdata.com/api-reference/account-management-api/Get_list_of_available_cities_of_static_network_per_country GET /zone/static/cities Get list of available cities of static network per country Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Live status of Static Zone Source: https://docs.brightdata.com/api-reference/account-management-api/Get_live_status_of_Static_Datacenter_ISP_Zone_and_IPs_with_connectivity_problem GET /zone/ips/unavailable Get live status of Static (Datacenter/ISP) Zone and IPs with connectivity problem Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Recent IPs attempting to use your Zone Source: https://docs.brightdata.com/api-reference/account-management-api/Get_recent_IPs_attempting_to_use_your_Zone GET /zone/recent_ips Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Available Datacenter & ISP IPs/Zone Source: https://docs.brightdata.com/api-reference/account-management-api/Get_the_available_Data_center_ISP_IPs_per_Zone GET /zone/route_ips Get the available Data center/ISP IPs per Zone Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Bandwidth stats for a zone Source: https://docs.brightdata.com/api-reference/account-management-api/Get_the_bandwidth_stats_for_a_Zone GET /zone/bw Get the bandwidth stats for a Zone Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Bandwidth stats for all your zones Source: https://docs.brightdata.com/api-reference/account-management-api/Get_the_bandwidth_stats_for_all_your_Zones GET /customer/bw Get the bandwidth stats for all your Zones Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) Sample response redacted # Total cost & bandwidth for a zone Source: https://docs.brightdata.com/api-reference/account-management-api/Get_the_total_cost_and_bandwidth_stats_for_a_Zone GET /zone/cost Get the total cost and bandwidth stats for a Zone Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Total Balance Source: https://docs.brightdata.com/api-reference/account-management-api/Get_total_balance_through_API GET /customer/balance Get total balance through API Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Migrate Static IPs between zones Source: https://docs.brightdata.com/api-reference/account-management-api/Migrate_Static_Datacenter_ISP_IPs_between_zones POST /zone/ips/migrate Migrate Static IPs between zones (Datacenter/ISP) **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Refresh Static IPs (Datacenter/ISP) Source: https://docs.brightdata.com/api-reference/account-management-api/Refresh_Static_Datacenter_ISP_IPs POST /zone/ips/refresh **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) ```json Sample Response for Static IPs { "ips":[ "1.1.1.1", "1.1.1.2", "1.1.1.3" ], "new_ips":[ "1.1.1.1", "1.1.1.3" ] } ``` # Refresh dedicated residential IPs Source: https://docs.brightdata.com/api-reference/account-management-api/Refresh_dedicated_residential_IPs POST /zone/ips/refresh **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) Each dedicated residential IP refreshed costs \$0.02/refresh/IP # Remove Static IPs (Datacenter/ISP) Source: https://docs.brightdata.com/api-reference/account-management-api/Remove_Static_Datacenter_ISP_IPs DELETE /zone/ips **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Remove a Zone Source: https://docs.brightdata.com/api-reference/account-management-api/Remove_Zone DELETE /zone **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Remove dedicated residential IPs Source: https://docs.brightdata.com/api-reference/account-management-api/Remove_dedicated_residential_IPs DELETE /zone/vips **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Switch Automatic Failover Source: https://docs.brightdata.com/api-reference/account-management-api/Switch_100_uptime_ON_OFF_in_a_Static_zone POST /zone/switch_100uptime Switch automatic failover (100% uptime) ON/OFF in a Static zone **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Turn on/off Zone Source: https://docs.brightdata.com/api-reference/account-management-api/Turn_on_off_Zone POST /zone/change_disable **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Allowlist IP(s) Source: https://docs.brightdata.com/api-reference/account-management-api/allowlist-ip POST /zone/whitelist Add IP to zone's allowlist **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Allowlist/Denylist Domain(s) Source: https://docs.brightdata.com/api-reference/account-management-api/allowlist-or-denylist-domains POST /zone/domain_perm Add domain to Zone allowlist or denylist **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Denylist IP(s) Source: https://docs.brightdata.com/api-reference/account-management-api/denylist-ip POST /zone/blacklist Add IP to zone's denylist **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Get Proxies Pending Replacement Source: https://docs.brightdata.com/api-reference/account-management-api/get-proxies-pending-replacement GET /zone/proxies_pending_replacement Get list of all proxies in the provided zone, which are pending replacement # Remove domain from Zone allowlist/denylist Source: https://docs.brightdata.com/api-reference/account-management-api/remove-domain-from-zone-allowlist-or-denylist DELETE /zone/domain_perm **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Remove IP from Zone allowlist Source: https://docs.brightdata.com/api-reference/account-management-api/remove-ip-from-zone-allowlist DELETE /zone/whitelist **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Remove IP from Zone denylist Source: https://docs.brightdata.com/api-reference/account-management-api/remove-ip-from-zone-denylist DELETE /zone/blacklist **Warning:** This API can modify your account settings, damage your operations or incur charges. Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # List of allowlisted IPs for a zone Source: https://docs.brightdata.com/api-reference/account-management-api/zone-ip-allowlist GET /zone/whitelist Get a list of allowlisted IPs for a specific Zone Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # List of denylisted IPs for a zone Source: https://docs.brightdata.com/api-reference/account-management-api/zone-ip-denylist GET /zone/blacklist Get a list of denylisted IPs for a specific Zone Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) # Authentication Source: https://docs.brightdata.com/api-reference/authentication Learn how to authenticate with Bright Data using API Access or Native Access methods. Bright Data ensures secure access to its services. Bright Data provides two primary methods for authentication: 1. **API Access** - Authenticate and interact with the Bright Data platform using an API key/token. 2. **Native Access** - Authenticate and interact with the Bright Data platform using proxy protocol `username` and `password`. Both API and Native access methods are billed at the same rate. Usage is charged according to your pricing plan for each Bright Data product. ## How do I authenticate with API Key? An API key is a secure hashed token used to authenticate with Bright Data API. We create a default key for you upon account creation automatically. You can create additional API keys for different users and with different permissions. Manage your API Keys anytime in the [Account settings](https://brightdata.com/cp/setting/users). ### How do I relay the API key within a request? Here’s an example request: ```sh Sample Request curl --request POST \ --url https://api.brightdata.com/request \ --header 'Authorization: Bearer ' \ --header 'Content-Type: application/json' \ --data '{ "zone": "my_unlocker_zone", "url": "https://example.com/page", "format": "json", "method": "GET", "country": "us", "data_format": "markdown" }' ``` The important part of this request is the `Authorization` header: ```sh 'Authorization: Bearer ' ``` This `` is your **API key**, which is required for every API request. ### How do I generate a new API key? To generate a new API key, follow these steps: Sign in to Bright Data's control panel and go to your [Account settings](https://brightdata.com/cp/setting/users). Click the **Add API key** button in the top right of the API key section. If you don’t see the 'API key' section, switch to your **admin account**. Only admins can generate API keys. Set the User, Permissions, and Expiration date (or choose 'Unlimited'), then click **Save**. Once generated, your API key will only be shown once. Be sure to **save it in a secure location**. ### What are API Key options and configuration? When creating an API key, you can assign one of five permission levels to control access and ensure security based on user roles or use cases. [Learn more about user roles here](/api-reference/authentication#understanding-api-key-permissions). #### What permissions level does an API key have? There are 5 types of permissions to choose from to control API key access: * **Admin:** Grants full access to the account, including all Billing, Financial, product settings and configurations. * **Finance:** Allows access to Billing and Financial pages only. * **Ops:** Allows access to zone/product configurations but restricts Billing access. * **Limit:** Permits management of zone passwords and IP allowlists/denylists. * **User:** Enables API usage on the zone/product level without access to Billing or product configuration pages. #### Does the API Key have an expiration date? When creating an API key, you can setup its expiration date. Requests authenticating with this key past the expiration date will be denied. You can setup the expiration date as `unlimited` - although this is possible, we strongly recommend to setup an expiration date. Consult your organization's information or data security officer for instructions. #### How can I view the API Key settings and configuration? You can view and manage your API Keys in [Account settings](https://brightdata.com/cp/setting/users). #### Why can't I see a plain text version of my API Key, or copy it? For your security, we do not allow plain text view of API keys with `admin` permissions. After creation you will be offered to copy and save the key. #### Can I refresh the API key? Yes. Once you click "Refresh" we will generate a new API key. After the refresh, any request you will send with the old key will fail to authenticate. #### Where should I save the API key with admin permissions? API keys are like passwords and should be dealt and saved with outmost care and under controlled access. Consult your IT manager or security administrator on how and where to save the keys per your organizations' processes and regulation. ## How do I authenticate with native proxy protocol? ### Which product zones can I access with proxy protocol? Proxy protocol access is supported by: * All proxy networks (Datacenter, ISP, residential & mobile) * Web Unlocker * Scraping browser ### How do I autheticate with native proxy username and password? Here’s an example of a Native Access request: ```sh Sample Request curl "https://www.example.com" \ -i --proxy brd.superproxy.io:33335 \ --proxy-user brd-customer-[ACCOUNT_ID]-zone-[ZONE_NAME]:[ZONE_PASSWORD] ``` The key part of this request is the `--proxy-user` parameter which holds the authentication credentials: ```sh proxy-user brd-customer-[ACCOUNT_ID]-zone-[ZONE_NAME]:[ZONE_PASSWORD] ``` This parameter is made up of three elements combined into a single string: 1. `ACCOUNT_ID` - your unique customer identifier. 2. `ZONE_NAME` - the name of the zone you created. 3. `ZONE_PASSWORD` - the password associated with your zone. Together, they form the complete proxy-user value. ### Username/password format * All the parameters are case sensitive. * There must be a consecutive string without spaces. * `-` (dash/minus sign) separates between the username substrings, and `:` (colon) between the username string and the password. #### Account ID Your Account ID is a unique identifier automatically generated when your Bright Data account is created. It is used for authentication and account-related operations. To find your Account ID, follow these steps: Click on Account Settings in the left-hand menu.} /> Account ID Your Account ID is always a text string that begins with `hl_###`. #### Zone name Zone is a bright data collection of configuration, for specific service. Read more about what is a zone [here](https://docs.brightdata.com/api-reference/terminology#what-is-a-zone). The zone name is set \*\*once \*\*by you when zone is created, and it cannot be changed later. You can review all of your zone names by visiting [My zones](https://brightdata.com/cp/zones). #### Zone password Each **zone** is assigned a unique password that is required for authenticating both API and Native Access requests. To view or update your zone password, follow these steps: Open My zones in your dashboard.} /> ## Comparing API Access vs. Native Access | Feature | API Access | Native Access | | :--------------------------------------------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Used for** | Seamless integration with scripts, automation tools, or third-party APIs | Direct proxy connections in browsers, crawlers, or proxy-compatible tools | | **Recommended products** | - [Unlocker API](https://docs.brightdata.com/scraping-automation/web-unlocker)
- [SERP](https://docs.brightdata.com/scraping-automation/serp-api)
- [Browsers](https://docs.brightdata.com/scraping-automation/scraping-browser)
- [Scrapers](https://docs.brightdata.com/datasets/scrapers/overview)
- [Functions](https://docs.brightdata.com/datasets/functions/introduction)
- [Marketplace](https://docs.brightdata.com/datasets/marketplace) | [Proxies](https://docs.brightdata.com/proxy-networks) | | [**SSL certificate**](https://docs.brightdata.com/general/account/ssl-certificate) | Not required | Required to access without KYC the residential network & unlocker. More info [here.](https://docs.brightdata.com/proxy-networks/residential/network-access) | | **Connection via** | API endpoint | Proxy endpoint | | **Authentication** | [API key](https://docs.brightdata.com/general/account/api-token) | `username:password` | | **Output options** | HTML or JSON | HTML | # Deliver Snapshot Source: https://docs.brightdata.com/api-reference/marketplace-dataset-api/deliver-snapshot dca-api POST /datasets/snapshots/{id}/deliver Deliver the dataset snapshot # Snapshot Content Source: https://docs.brightdata.com/api-reference/marketplace-dataset-api/download-the-file-by-snapshot_id dca-api GET /datasets/snapshots/{id}/download Get dataset snapshot content # Filter Dataset (BETA) Source: https://docs.brightdata.com/api-reference/marketplace-dataset-api/filter-dataset dca-api POST /datasets/filter Create a dataset snapshot based on a provided filter Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token) ## General Description * A call to this endpoint starts the async job of filtering the dataset and creating a snapshot with filtered data in your account. * The maximum amount of time for the job to finish is 5 minutes. If the job doesn't finish in this timeframe it will be cancelled. * Creating the dataset snapshot is subject to charges based on the snapshot size and record price. * The maximum depth of nesting the filter groups is 3. ## Modes of Use ### 1. JSON Mode (No File Uploads) Use this when you are not uploading any files. * All parameters (`dataset_id`, `records_limit`, and `filter`) are sent in the JSON request body. * `Content-Type` must be `application/json`. * No query parameters are used. ```bash Example curl --request POST \ --url https://api.brightdata.com/datasets/filter \ --header 'Authorization: Bearer ' \ --header 'Content-Type: application/json' \ --data '{ "dataset_id": "gd_l1viktl72bvl7bjuj0", "records_limit": 100, "filter": { "name": "name", "operator": "=", "value": "John" } }' ``` *** ### 2. Multipart/Form-Data Mode (File Uploads) Use this when uploading CSV or JSON files containing filter values. * `dataset_id` and `records_limit` must be sent as **query parameters** in the URL. * The `filter` and any uploaded files are included in the **form-data body**. * `Content-Type` must be `multipart/form-data`. ```bash Example curl --request POST \ --url "https://api.brightdata.com/datasets/filter?dataset_id=gd_l1vijqt9jfj7olije&records_limit=100" \ --header 'Authorization: Bearer ' \ --header 'Content-Type: multipart/form-data' \ --form 'filter={"operator":"and","filters":[{"name":"industries:value","operator":"includes","value":"industries.csv"}]}' \ --form 'files[]=@/path/to/industries.csv' ``` *** ## Filter Syntax ### Operators The following table shows operators that can be used in the field filter. | Operator | Field Types | Description | | -------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | = | Any | Equal to | | != | Any | Not equal to | | \< | Number, Date | Lower than | | \<= | Number, Date | Lower than or equal | | > | Number, Date | Greater than | | >= | Number, Date | Greater than or equal | | in | Any | Tests if field value is equal to any of the values provided in filter's value | | not\_in | Any | Tests if field value is not equal to all of the values provided in filter's value | | includes | Array, Text | Tests if the field value contains the filter value. If the filter value is a single string, it matches records where the field value contains that string. If the filter value is an array of strings, it matches records where the field value contains a least one string from the array. | | not\_includes | Array, Text | Tests if the field value does not contain the filter value. If the filter value is a single string, it matches records where the field value does not contain that string. If the filter value is an array of strings, it matches records where the field value does not contain any of the strings from the array. | | array\_includes | Array | Tests if filter value is in field value (exact match) | | not\_array\_includes | Array | Tests if filter value is not in field value (exact match) | | is\_null | Any | Tests if the field value is equal to NULL. Operator does not accept any value. | | is\_not\_null | Any | Tests if the field value is not equal to NULL. Operator does not accept any value. | ### Combining Multiple Filters Multiple field filters can be combined into the filter group using 2 logical operators: 'and', 'or'. API supports filters with a maximum nesting depth of 3. Example of filter group: ```json { // operator can be one of ["and", "or"] "operator": "and", // an array of field filters "filters": [ { "name": "reviews_count", "opeartor": ">", "value": "200" }, { "name": "rating", "operator": ">", "value": "4.5" } ] } ``` # Filter Dataset (JSON or File Uploads) Source: https://docs.brightdata.com/api-reference/marketplace-dataset-api/filter-dataset-with-csv-json-files filter-csv-json POST /datasets/filter Create a dataset snapshot based on a provided filter Paste your API key into the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token). ## General Description * This endpoint filters a dataset and creates a snapshot of the filtered data in your account. * The job runs asynchronously and can take up to 5 minutes to complete. If it exceeds this time, it will be cancelled. * Charges apply based on the size of the snapshot and per-record pricing. * Filters can have up to 3 levels of nested filter groups. * You can upload CSV or JSON files for efficient filtering when handling large sets of values. *** ## File Format Requirements * First line must be a header matching the field name in your filter. * Each following line contains a single value. ```csv Example: industries.csv industries:value Accounting Ad Network Advertising ``` * Must be an array of objects, each with a key matching the field name in your filter. ```json Example industries.json [ {"industries:value": "Accounting"}, {"industries:value": "Ad Network"}, {"industries:value": "Advertising"} ] ``` *** ## Filter Syntax with File References When using file uploads, reference the filename in the filter's `value` field. ```json Example { "operator": "and", "filters": [ { "name": "industries:value", "operator": "includes", "value": "industries.csv" } ] } ``` ### **Supported Operators for File References** | Operator | Field Types | Description | | :------------------- | :---------- | :--------------------------------------------- | | `in` | Any | Field value equals any value in file | | `not_in` | Any | Field value does not equal any value in file | | `includes` | Array, Text | Field value contains any value in file | | `not_includes` | Array, Text | Field value does not contain any value in file | | `array_includes` | Array | Any value in file exists in field value | | `not_array_includes` | Array | No values in file exist in field value | *** ## Example: Filtering with Multiple Files ```bash curl \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: multipart/form-data" \ -F "files[]=@/path/to/industries.csv" \ -F "files[]=@/path/to/regions.csv" \ -F "filter={\"operator\":\"and\",\"filters\":[{\"name\":\"industries:value\",\"operator\":\"includes\",\"value\":\"industries.csv\"},{\"name\":\"region\",\"operator\":\"in\",\"value\":\"regions.csv\"}]}" \ "api.brightdata.com/datasets/filter?dataset_id=gd_l1vijqt9jfj7olije" ``` ## Troubleshooting | Issue | Possible Solution | | :------------------------ | :----------------------------------------------------------------------------------------------------------------------- | | **"File not found"** | Make sure the filename in your filter exactly matches the uploaded file name. | | **"Invalid file format"** | Check CSV header matches the filter field name, or ensure JSON is an array of objects. | | **"Field not found"** | Verify field exists in dataset. Use [Get Dataset Metadata](/api-reference/marketplace-dataset-api/get-dataset-metadata). | ## Related Documentation * [Get Dataset List](/api-reference/marketplace-dataset-api/get-dataset-list) * [Get Dataset Metadata](/api-reference/marketplace-dataset-api/get-dataset-metadata) * [Dataset Download API](/datasets/custom-datasets/custom-dataset-api) # Dataset list Source: https://docs.brightdata.com/api-reference/marketplace-dataset-api/get-dataset-list dca-api GET /datasets/list List all Dataset IDs of all Scraper APIs, you can use this API endpoint to retrieve a list of available datasets. # Get Dataset Metadata Source: https://docs.brightdata.com/api-reference/marketplace-dataset-api/get-dataset-metadata dca-api GET /datasets/{dataset_id}/metadata Paste your API key to the authorization field. To get an API key, [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground) and learn how to [generate an API key](/api-reference/unlocker/api_token). ## General Description * This endpoint retrieves detailed metadata for a specific dataset. * Metadata includes available fields, data types, and descriptions. * Use this endpoint to understand the structure of a dataset before querying or downloading it. ## Request ### **Endpoint** ``` GET http://api.brightdata.com/datasets/{dataset_id}/metadata ``` ### **Path Parameters** | Parameter | Type | Description | | ------------ | -------- | ------------------------------------ | | `dataset_id` | `string` | The unique identifier of the dataset | ### **Headers** | Header | Type | Description | | --------------- | ------ | ------------------------------- | | `Authorization` | string | Your API key for authentication | ## Response ### **Response Example** ```json { "id": "gd_l1vijqt9jfj7olije", "fields": { "name": { "type": "text", "active": true, "description": "The name of the company" }, "url": { "type": "url", "required": true, "description": "The URL or web address associated with the company" }, "cb_rank": { "type": "number", "description": "Crunchbase rank assigned to the company" } } } ``` ### **Response Fields** | Field | Type | Description | | -------- | -------- | ------------------------------------------------- | | `id` | `string` | Unique identifier for the dataset | | `fields` | `object` | Contains metadata about each field in the dataset | ### **Field Metadata** Each field in the `fields` object contains the following attributes: | Attribute | Type | Description | | ------------- | --------- | ------------------------------------------------------ | | `type` | `string` | Data type of the field (e.g., `text`, `number`, `url`) | | `active` | `boolean` | Indicates if the field is currently active | | `required` | `boolean` | Indicates if the field is mandatory (if applicable) | | `description` | `string` | Brief description of the field | ## Example Use Case ### **Fetching Dataset Metadata** To retrieve metadata for the "Crunchbase companies information" dataset: #### **Request** ``` GET http://api.brightdata.com/datasets/gd_l1vijqt9jfj7olije/metadata ``` #### **Response** ```json { "id": "gd_l1vijqt9jfj7olije", "fields": { "name": { "type": "text", "active": true, "description": "The name of the company" }, "url": { "type": "url", "required": true, "description": "The URL or web address associated with the company" }, "cb_rank": { "type": "number", "description": "Crunchbase rank assigned to the company" } } } ``` ## Troubleshooting & FAQs ### **Issue: "Unauthorized" response** **Solution**: Ensure you have included a valid API key in the request header. ### **Issue: "Dataset not found"** **Solution**: Verify that the `dataset_id` is correct and exists in the dataset list. ### **Issue: "Field missing in metadata"** **Solution**: Some fields may be inactive or unavailable for certain datasets. ## Related Documentation * [Get Dataset List](./get-dataset-list.md) * [Dataset Filtering API](./filter-dataset.md) * [Dataset Download API](./download-dataset.md) # Get Snapshot Metadata Source: https://docs.brightdata.com/api-reference/marketplace-dataset-api/get-snapshot-meta dca-api GET /datasets/snapshots/{id} Get dataset snapshot metadata # Get Snapshot Parts Source: https://docs.brightdata.com/api-reference/marketplace-dataset-api/get-snapshot-parts dca-api GET /datasets/snapshots/{id}/parts Get dataset snapshot delivery parts. All query parameters used here need to match those used when downloading the snapshot to get accurate parts # Troubleshooting Source: https://docs.brightdata.com/api-reference/marketplace-dataset-api/troubleshooting This section covers errors encountered when interacting directly with Bright Data's Unlocker API, SERP API, Web Scraper API, Marketplace Dataset API, and Web Scraper IDE. ## Proxy networks troubleshooting Error codes and description for proxy networks can be found [here](https://docs.brightdata.com/proxy-networks/errorCatalog) ## Datasets API errors ### 200 ok (with error/status messages) While technically a successful HTTP status, these responses carry messages indicating an ongoing process or a specific failure within the operation, requiring developer attention. | Message | Associated endpoints | Cause | Suggested action | | :--------------------------------------------------------------------------- | :--------------------------------------- | :-------------------------------------------------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `{status: "STATUS", message: "Snapshot is not ready yet, try again in 10s"}` | `GET /datasets/v3/snapshot/:snapshot_id` | The snapshot is still being built or processed. | This is an interim status. Poll the endpoint again after the suggested delay (e.g., `10s`). | | `{status: "building", message: "Snapshot is building, try again in 10s"}` | `GET /datasets/v3/snapshot/:snapshot_id` | The snapshot is actively being built. | Poll the endpoint again after the suggested delay. | | `{status: "failed", error_message: "ERROR_MESSAGE"}` | `GET /datasets/v3/progress/:snapshot_id` | A collection or snapshot operation has failed. | Check the specific `ERROR_MESSAGE` for details. Review your collection setup and inputs. | | `Something went wrong. Our team is looking into it.` | `GET /datasets/v3/progress/:snapshot_id` | An internal system error occurred during monitoring. | This indicates a problem on Bright Data's end. No immediate action is required from your side, but you can contact support if it persists. | | `Account is suspended` | `GET /datasets/v3/progress/:snapshot_id` | Your Bright Data account has been suspended, often due to a negative balance. | Top up your account balance. If suspension extends beyond 24 hours, previously allocated static IPs may be released. Go to your Bright Data Zones page for updated IPs. | | `Account is new, please activate it in account settings. URL` | `GET /datasets/v3/progress/:snapshot_id` | Your newly created account requires activation. | Log in to your Bright Data account settings and complete the activation process. | | `No data found in discovery` | `GET /datasets/v3/progress/:snapshot_id` | The discovery phase of your collection did not yield any data. | Review your discovery configuration and target settings. | | `Snapshot is empty` | `GET /datasets/v3/progress/:snapshot_id` | The completed snapshot contains no data. | Check your collection process and dataset configuration to ensure data is being collected. | | `Failed to deliver snapshot` | `GET /datasets/v3/progress/:snapshot_id` | An error occurred during the process of delivering the collected data to storage. | Review your delivery options and try again. If the issue persists, contact support. | | `Failed to download response` | `GET /datasets/v3/progress/:snapshot_id` | An error occurred while attempting to download the response. | This could be a transient network issue or a problem on Bright Data's side. Retry the request. | | `Failed to trigger collector` | `GET /datasets/v3/progress/:snapshot_id` | An internal error prevented the collection from being triggered. | Retry the request. If the issue persists, contact support. | | `Internal server error` | `GET /datasets/v3/progress/:snapshot_id` | A general internal server error. | Retry the request. If the issue persists, contact support. | | `Input validation failed: DETAILS` | `GET /datasets/v3/progress/:snapshot_id` | There was an internal validation error with the input. | Review your input against the API documentation. If it appears correct, contact support. | ### 202 accepted **Cause:** The request has been accepted for processing, but the operation is not yet complete. This is typically an interim status indicating that a resource is being prepared or built. | Message | Associated Endpoint(s) | Cause | Suggested Action | | :--------------------------------------------------------------------------- | :--------------------------------------- | :---------------------------------------------- | :------------------------------------------------------------------------------------------ | | `{status: "STATUS", message: "Snapshot is not ready yet, try again in 10s"}` | `GET /datasets/v3/snapshot/:snapshot_id` | The snapshot is still being built or processed. | This is an interim status. Poll the endpoint again after the suggested delay (e.g., `10s`). | | `{status: "building", message: "Snapshot is building, try again in 10s"}` | `GET /datasets/v3/snapshot/:snapshot_id` | The snapshot is actively being built. | Poll the endpoint again after the suggested delay. | ### 400 bad request **Cause:** Your request was invalid, malformed, or contained incorrect parameters. This is a common client-side error, indicating an issue with how the request was constructed. | Message | Associated Endpoint(s) | Cause | Suggested Action | | :------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------ | | `{validation_errors: [ERRORS]}` | Many Scrapers Library endpoints, `GET /datasets/v3/snapshots`, Marketplace Dataset API | General validation failure for request inputs; the ERRORS array will provide specific details. | Check the ERRORS array for specific validation issues and correct your request payload. | | `dataset missing` | `POST /datasets/v3/trigger` | The required dataset ID or name is missing from the request. | Ensure the dataset identifier is included in your request. | | `Invalid attachments` | `POST /datasets/v3/trigger` | The attachments provided are not valid. | Verify the format and content of your attachments. | | `This dataset is not allowed for API` | `POST /datasets/v3/trigger` | The specified dataset cannot be triggered via API. | Confirm that the dataset is configured for API access. | | `This dataset is not ready yet` | `POST /datasets/v3/trigger` | The dataset is still being prepared or is not in an active state. | Wait for the dataset to become ready before attempting to trigger a collection. | | `No data to trigger` | `POST /datasets/v3/trigger` | The dataset has no inputs or configuration to initiate a collection. | Ensure your dataset has valid inputs defined. | | `Should be at least LIMIT inputs` | `POST /datasets/v3/trigger` | The request does not meet the minimum number of required inputs. | Provide at least the specified `LIMIT` number of inputs. | | `Snapshot is expired` | `GET /datasets/v3/snapshot/:snapshot_id`, `POST /datasets/v3/deliver/:snapshot_id` | The snapshot you are trying to access has passed its validity period. | Trigger a new collection to generate a fresh snapshot. | | `Snapshot is empty` | `GET /datasets/v3/snapshot/:snapshot_id`, `POST /datasets/v3/deliver/:snapshot_id` | The collected snapshot contains no data. | Check your collection process and dataset configuration to ensure data is being collected. | | `Snapshot is not ready` | `GET /datasets/v3/snapshot/:snapshot_id/parts`, `POST /datasets/v3/deliver/:snapshot_id` | The snapshot is still being processed or is not in a state to be downloaded or delivered. | Wait for the snapshot to complete processing. You can monitor its status via the "Monitor progress" endpoint. | | `Snapshot input does not exist` | `GET /datasets/v3/snapshot/:snapshot_id/input` | The input file associated with the snapshot could not be found. | Verify the `snapshot_id` and ensure the input file was successfully generated. | | `Snapshot is not running` | `POST /datasets/v3/snapshot/:snapshot_id/cancel` | You attempted to cancel a collection that is not currently active. | Check the status of the collection before attempting to cancel it. | | `Deliver options are missing` | `POST /datasets/v3/deliver/:snapshot_id` | Required delivery configuration (e.g., destination details) was not provided. | Ensure all necessary delivery options are included in your request. | | `Snapshot is too big for single file delivery` | `POST /datasets/v3/deliver/:snapshot_id` | The collected data exceeds the limit for a single file delivery. | Consider delivering the snapshot in multiple parts or adjusting your collection scope. | | `Batch size should be at least MIN_BATCH_SIZE` | `POST /datasets/v3/deliver/:snapshot_id` | The specified batch size for delivery is below the minimum allowed. | Increase the `batch_size` to at least `MIN_BATCH_SIZE`. | | `Type \{init\_types\.compr\_update is no longer supported\. Use '{init_types.discover_all}' instead.` | Marketplace Dataset API | The requested operation type is deprecated. | Update your request to use the recommended operation type, `${init_types.discover_all}`. | | `Type ${init_types.update_existing} is no longer supported.` | Marketplace Dataset API | The requested operation type is no longer supported. | Use a supported operation type. | | `Type ${init_types.discover_new} is no longer supported.` | Marketplace Dataset API | The requested operation type is no longer supported. | Use a supported operation type. | | `Initiation reason is required.` | Marketplace Dataset API | The request is missing the required initiation reason. | Include the `initiation reason` in your request. | | `This feature is not available.` | Marketplace Dataset API | The requested feature is not supported for the dataset. | Check the dataset's capabilities and adjust your request accordingly. | | `This dataset was rejected.` | Marketplace Dataset API | The dataset was rejected and cannot be processed. | This dataset cannot be used. Contact Bright Data support for more information. | | `This dataset is not ready.` | Marketplace Dataset API | The dataset is not in a state where it can be processed. | Wait for the dataset to become ready. | | `This dataset does not support discovery. Supported types: ['${init_types.url_collection}']` | Marketplace Dataset API | The dataset does not support the requested discovery type. | Use a supported discovery type, such as `url_collection`. | | `Incorrect discovery collector id.` | Marketplace Dataset API | The discovery collector ID provided in the request is invalid. | Verify the discovery collector ID. | | `View not found.` | Marketplace Dataset API | The requested view is not available. | Check for correct view names and available views for the dataset. | | `This dataset does not support collection.` | Marketplace Dataset API | The dataset does not support the requested collection operation. | Use an operation supported by the dataset. | | `Batch size must be at least 1000.` | Marketplace Dataset API | The batch size specified in the request is below the minimum allowed value. | Increase the `batch_size` to at least 1000. | | `{error: 'Snapshot failed'}` | Marketplace Dataset API | The snapshot operation failed. | Review the collection process for underlying issues. | | `{error: 'Invalid snapshot type'}` | Marketplace Dataset API | The snapshot type provided in the request is invalid. | Use a valid snapshot type. | ### 401 unauthorized **Cause:** Your request lacks valid authentication credentials for the API. | Message | Associated Endpoint(s) | Cause | Suggested Action | | :------------- | :---------------------- | :------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------- | | `Unauthorized` | Marketplace Dataset API | The API key or authentication credentials provided are invalid or missing. | Ensure your API key and authentication credentials are correct and properly included in the request. | ### 402 payment required **Cause:** Your account balance is insufficient to process the requested API operation. | Message | Associated Endpoint(s) | Cause | Suggested Action | | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :---------------------- | :----------------------------------------------------------------- | :------------------------------------------------------------------------------------------ | | `{error: 'Your current balance is insufficient to process this data collection request. Please add funds to your account or adjust your request to continue. ($220 is missing)'}` | Marketplace Dataset API | The user's account balance is insufficient to process the request. | Add funds to your Bright Data account or adjust your request parameters to reduce its cost. | ### 403 forbidden **Cause:** You do not have permission to access the requested API resource, or your request is being blocked by a Bright Data policy. | Message | Associated Endpoint(s) | Cause | Suggested Action | | :--------------------- | :---------------------- | :----------------------------------------------------------------------- | :----------------------------------------------------------------------------------- | | `Access denied.` | Marketplace Dataset API | The user does not have the necessary permissions to access the resource. | Check your account permissions and ensure you have access to the requested resource. | | `Cannot skip billing.` | Marketplace Dataset API | The user attempted to skip billing, which is not allowed. | Billing | ### 404 not found **Cause:** The specific API resource you're trying to access does not exist in the system. You may encounter this error when trying to access a dataset, snapshot, delivery, or a general request that isn't found. | Message | Associated Endpoint(s) | Cause | Suggested Action | | :------------------------ | :-------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------ | :------------------------------------------------------------------------- | | `dataset does not exist` | POST /datasets/v3/trigger, Marketplace Dataset API | The specified dataset ID or name could not be found. | Double-check the dataset identifier in your request. | | `Snapshot does not exist` | Many scrapers endpoints, Marketplace Dataset API | The snapshot\_id provided does not correspond to an existing snapshot. | Verify the snapshot\_id. It might be incorrect, expired, or never existed. | | `Delivery does not exist` | GET /datasets/v3/delivery/:delivery\_id | The delivery\_id provided could not be found. | Confirm the delivery\_id is correct. | | `Request not found` | Web Scraper IDE API, Marketplace Dataset API | The specified request ID or other request details were not found in our system. | Verify your request details and ensure they are valid. | | `Page not found` | Invalid URL, which suggests the URL might be broken or dead (specific to Unlocker API). | Verify the URL is correct and active. | | ### 422 unprocessable entity **Cause:** The API request was well-formed but could not be processed due to semantic errors in the data provided. | Message | Associated Endpoint(s) | Cause | Suggested Action | | :----------------------------------------------------- | :---------------------- | :--------------------------------------------------------------------------- | :-------------------------------------------------------------------- | | `{error: 'Provided filter did not match any records'}` | Marketplace Dataset API | The filter provided in the request did not match any records in the dataset. | Adjust your filter criteria to match existing records in the dataset. | ### 429 too many requests **Cause:** You have exceeded a rate limit or the maximum number of parallel jobs allowed for your account or dataset when using the API. | Message | Associated Endpoint(s) | Cause | Suggested Action | | :------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------- | | `You have too many running jobs for this dataset.` | `POST /datasets/v3/trigger` | You’ve hit the concurrency limit for collection jobs on this dataset. | Wait for some active jobs to complete. For large workloads, consider combining multiple inputs into a single collection request to reduce concurrency. | | `{error: 'Maximum limit of ${max_parallel_jobs} jobs per dataset has been exceeded'}` | Marketplace Dataset API | You have exceeded the maximum allowed number of parallel jobs for the dataset. | Reduce the number of concurrent jobs, or wait for existing jobs to complete before initiating new ones. | | | This error code implies a rate limit (rare) and auto-throttling by Bright Data (specific to Unlocker API). | Open a ticket or email [support@brightdata.com](mailto:support@brightdata.com) for assistance. | Confirm the delivery\_id is correct. | ### 500 internal server error **Cause:** An unexpected error occurred on the Bright Data API server. These are server-side issues that are typically outside of your direct control. | Message | Associated Endpoint(s) | Cause | Suggested Action | | :---------------------- | :------------------------------------------------------------------------------------- | :----------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `Internal server error` | Many Scrapers Library endpoints, `GET /datasets/v3/snapshots`, Marketplace Dataset API | A general, unhandled server-side error occurred. | This is typically a temporary issue. Retry your request after a short delay. If the problem persists, contact Bright Data support with the request details and any error messages. | | `Internal error.` | Marketplace Dataset API | An unexpected error occurred on the server. | This is a server-side issue. Retry the request. If the problem persists, contact Bright Data support. | ### 502 bad gateway **Cause:** The Bright Data API server received an invalid response from an upstream server. | Message | Associated Endpoint(s) | Cause | Suggested Action | | :------------------------------------------------------------------------------------------- | :---------------------- | :----------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------- | | `Unexpected error. The server encountered an unexpected error while processing the request.` | Marketplace Dataset API | The server encountered an unexpected error while processing the request. | This is typically a temporary server-side issue. Retry the request. If it persists, contact support. | ### 503 service unavailable | Message | Associated Endpoint(s) | Cause | Suggested Action | | :-------------------- | :----------------------- | :------------------------------------------------------ | :---------------------------------------------------------------------------------------------------------------------------------- | | `Service Unavailable` | Specific to Unlocker API | Browser check failed or browser check wasn't completed. | This indicates a temporary service unavailability or a browser rendering issue. Retry the request. If it persists, contact support. | ## Dataset API errors ### 400 - Bad Request Indicates that the request was invalid or cannot be processed. | Error Message | Cause | | :-------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------- | | `{validation_errors: ["filter.name is required"]}` | The request is missing a required filter parameter (e.g., `filter.name`). | | `{validation_errors: ["Invalid input provided"]}` | The input provided in the request is not valid. | | `Type ${init_types.compr_update} is no longer supported. Use '${init_types.discover_all}' instead.` | The requested operation type is deprecated. | | `Type ${init_types.update_existing} is no longer supported.` | The requested operation type is no longer supported. | | `Type ${init_types.discover_new} is no longer supported.` | The requested operation type is no longer supported. | | `Initiation reason is required.` | The request is missing the required initiation reason. | | `This feature is not available.` | The requested feature is not supported for the dataset. | | `This dataset was rejected.` | The dataset was rejected and cannot be processed. | | `This dataset is not ready.` | The dataset is not in a state where it can be processed. | | `This dataset does not support discovery. Supported types: ['${init_types.url_collection}']` | The dataset does not support the requested discovery type. | | `Incorrect discovery collector id.` | The discovery collector ID provided in the request is invalid. | | `View not found.` | The requested view is not available. | | `This dataset does not support collection.` | The dataset does not support the requested collection operation. | | `Batch size must be at least 1000.` | The batch size specified in the request is below the minimum allowed value. | | `{error: 'Snapshot failed'}` | The snapshot operation failed. | | `{error: 'Snapshot not ready'}` | The snapshot is not ready for processing. | | `{error: 'Invalid snapshot type'}` | The snapshot type provided in the request is invalid. | | `{validation_errors: [e.message]}` | A validation error occurred, and the specific message is included in the response. | *** ### 401 - Unauthorized Indicates that the user is not authorized to access the requested resource. | Error Message | Cause | | :------------- | :------------------------------------------------------------------------- | | `Unauthorized` | The API key or authentication credentials provided are invalid or missing. | *** ### 402 - Payment Required Indicates that the user does not have sufficient funds to process the request. | Error Message | Cause | | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------- | | `{error: 'Your current balance is insufficient to process this data collection request. Please add funds to your account or adjust your request to continue. ($220 is missing)'}` | The user's account balance is insufficient to process the request. | *** ### 403 - Forbidden Indicates that the user does not have permission to access the requested resource. | Error Message | Cause | | :--------------------- | :----------------------------------------------------------------------- | | `Access denied.` | The user does not have the necessary permissions to access the resource. | | `Cannot skip billing.` | The user attempted to skip billing, which is not allowed. | *** ### 404 - Not Found Indicates that the requested resource could not be found. | Error Message | Cause | | :------------------------------ | :---------------------------------------------------- | | `{error: 'Dataset not found'}` | The specified dataset ID does not exist. | | `{error: 'Snapshot not found'}` | The specified snapshot ID does not exist. | | `Dataset does not exist.` | The dataset referenced in the request does not exist. | | `Request not found.` | The specified request ID does not exist. | *** ### 422 - Unprocessable Entity Indicates that the request was well-formed but could not be processed due to semantic errors. | Error Message | Cause | | :----------------------------------------------------- | :--------------------------------------------------------------------------- | | `{error: 'Provided filter did not match any records'}` | The filter provided in the request did not match any records in the dataset. | *** ### 429 - Too Many Requests Indicates that the user has exceeded the rate limit or the maximum number of parallel jobs. | Error Message | Cause | | :------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------------- | | `{error: 'Maximum limit of ${max_parallel_jobs} jobs per dataset has been exceeded'}` | The user has exceeded the maximum allowed number of parallel jobs for the dataset. | *** ### 500 - Internal Server Error Indicates that an unexpected error occurred on the server. | Error Message | Cause | | :---------------- | :------------------------------------------ | | `Internal error.` | An unexpected error occurred on the server. | *** ### 502 - Bad Gateway Indicates that the server received an invalid response from an upstream server. | Error Message | Cause | | :---------------- | :----------------------------------------------------------------------- | | Unexpected error. | The server encountered an unexpected error while processing the request. | *** ### Appendix **Sources:** * [http://cvs.brightdata.com/cvs/zon/pkg/www/dca/pub/util.js?rev=1.197](http://cvs.brightdata.com/cvs/zon/pkg/www/dca/pub/util.js?rev=1.197) * pkg/www/dca/dataset\_ext\_api.js > Some error codes (e.g., 400, 404) may have multiple causes, as outlined above.\ > Error messages are designed to provide clear feedback to users about the issue and how to resolve it.\ > For errors with dynamic content (e.g., balance or job limits), placeholders like `${max_parallel_jobs}` or `${init_types}` are replaced with actual values at runtime. # Bright Data API Postman collection Source: https://docs.brightdata.com/api-reference/postman-collection Quickly test and explore Bright Data's APIs using our Postman collection. View, run, and fork API requests without setting up your own backend environment.
## Getting Started Get up and running with Bright Data API Postman collection in just 3 simple steps: 1. **Get your Bright Data API key**: Generate your [Bright Data API key from your account dashboard](/general/account/api-token) 2. **Update Postman collection environment variable**: Set your `BRIGHT_DATA_API_KEY` in the Postman collection environment variables under Production environment. postman-collection-environment-varaible.jpg 3. **Start scraping**: Select the `Production` environment from the dropdown menu in the top-right corner, then customize the request body and parameters. Once configured, simply click the **Send** button to execute your request and retrieve the results. This Postman collection includes few popular endpoints and will expand based on user needs. For complete documentation, see the official [Bright Data API reference](/api-reference). # Add a User Source: https://docs.brightdata.com/api-reference/proxy-manager/add_a_user **API endpoint:** `POST` `/api/lpm_user` ## `POST` body User email address to add {/* TODO: Missing Sample Response */} ```sh Shell curl "http://127.0.0.1:22999/api/lpm_user" -H "Content-Type: application/json" -d '{"email":"test@example.com"}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/lpm_user', json: {'email':'test@example.com'} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"email\":\"test@example.com\"}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/lpm_user") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/lpm_user"), Content = new StringContent(JsonConvert.SerializeObject(new { email = "test@example.com" }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'email':'test@example.com'} r = requests.post('http://127.0.0.1:22999/api/lpm_user', data=json.dumps(data)) print(r.content) ``` # Allowlist IPs Source: https://docs.brightdata.com/api-reference/proxy-manager/allowlist-ips **API endpoint:** `PUT` `/api/wip` API key API key should be generated using [Generate API key](/api-reference/proxy-manager/generate_token_for_token_based_authentication) ## `PUT` body IP to allowlist. e.g, `ip="1.2.1.2"` ```sh Shell curl -X PUT "http://127.0.0.1:22999/api/wip" -H "Authorization: API key" -H "Content-Type: application/json" -d '{"ip":"1.2.1.2"}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'PUT', url: 'http://127.0.0.1:22999/api/wip', json: {'ip':'1.2.1.2'}, headers: {'Authorization': 'API key'}, }).then(function(data){ console.log(data);}, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ip\":\"1.2.1.2\"}"; String res =Executor.newInstance() .addHeader("Authorization", "API key") .execute(Request.Put("http://127.0.0.1:22999/api/wip") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent() .asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Put, RequestUri = new Uri("http://127.0.0.1:22999/api/wip"), Headers = {{"Authorization", "API key"}}, Content = new StringContent(JsonConvert.SerializeObject(new { ip = "1.2.1.2" }), Encoding.UTF8, "application/json") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import json import requests data = {'ip':'1.2.1.2'} headers = {'Authorization': 'API key'} r = requests.put( 'http://127.0.0.1:22999/api/wip', data=json.dumps(data), headers=headers ) print(r.content) ``` ```json 200 Successful response ``` ```json 400 Bad request. No IP was passed ``` ```json 403 Forbidden. No authentication provided ``` ```json 422 Unprocessable entity. Invalid IP was passed ``` # Ban an IP (All Ports) Source: https://docs.brightdata.com/api-reference/proxy-manager/ban_an_ip_for_all_ports API endpoint: `POST` `/api/banip` ## `POST` body IP to ban. e.g. `1.2.1.2` Ban the IP for sending requests to the specified domain Ban the IP for specified milliseconds ```sh Shell curl "http://127.0.0.1:22999/api/banip" -H "Content-Type: application/json" -d '{"ip":"1.2.1.2","domain":"example.com","ms":60000}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/banip', json: {'ip':'1.2.1.2','domain':'example.com','ms':60000} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ip\":\"1.2.1.2\",\"domain\":\"example.com\",\"ms\":60000}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/banip") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/banip"), Content = new StringContent(JsonConvert.SerializeObject(new { ip = "1.2.1.2", domain = "example.com", ms = 60000 }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'ip':'1.2.1.2','domain':'example.com','ms':60000} r = requests.post('http://127.0.0.1:22999/api/banip', data=json.dumps(data)) print(r.content) ``` # Ban an IP Source: https://docs.brightdata.com/api-reference/proxy-manager/ban_an_ip_single_ **API endpoint:** `POST` `/api/proxies/{PORT}/banip` ## Path Parameter Existing proxy port number ## `POST` body IP to ban. e.g. `1.2.1.2` Ban the IP for sending requests to the specified domain Ban the IP for specified milliseconds ```sh Shell curl "http://127.0.0.1:22999/api/proxies/{PORT}/banip" -H "Content-Type: application/json" -d '{"ip":"1.2.1.2","domain":"example.com","ms":60000}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/proxies/{PORT}/banip', json: {'ip':'1.2.1.2','domain':'example.com','ms':60000} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ip\":\"1.2.1.2\",\"domain\":\"example.com\",\"ms\":60000}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/proxies/{PORT}/banip") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}/banip"), Content = new StringContent(JsonConvert.SerializeObject(new { ip = "1.2.1.2", domain = "example.com", ms = 60000 }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'ip':'1.2.1.2','domain':'example.com','ms':60000} r = requests.post('http://127.0.0.1:22999/api/proxies/{PORT}/banip', data=json.dumps(data)) print(r.content) ``` # Ban multiple IPs (array) Source: https://docs.brightdata.com/api-reference/proxy-manager/ban_ips_array_ **API endpoint:** `POST` `/api/proxies/{PORT}/banips` ## Path Parameter Existing proxy port number ## `POST` body IPs to ban. e.g. \["10.0.0.1","20.0.0.1"] Ban the IP for sending requests to the specified domain Ban the IP for specified milliseconds ```sh Shell curl "http://127.0.0.1:22999/api/proxies/{PORT}/banips" -H "Content-Type: application/json" -d '{"ips":["10.0.0.1","20.0.0.1"],"domain":"example.com","ms":60000}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/proxies/{PORT}/banips', json: {'ips':['10.0.0.1','20.0.0.1'],'domain':'example.com','ms':60000} }).then(function(data){ console.log(data);}, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ips\":[\"10.0.0.1\",\"20.0.0.1\"],\"domain\":\"example.com\",\"ms\":60000}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/proxies/{PORT}/banips") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}/banips"), Content = new StringContent(JsonConvert.SerializeObject(new { ips = [ "10.0.0.1","20.0.0.1" ], domain = "example.com", ms = 60000 }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'ips':['10.0.0.1','20.0.0.1'],'domain':'example.com','ms':60000} r = requests.post('http://127.0.0.1:22999/api/proxies/{PORT}/banips', data=json.dumps(data)) print(r.content) ``` # Create new Proxy Port Source: https://docs.brightdata.com/api-reference/proxy-manager/create_a_new_proxy_port **API endpoint:** `POST` `/api/proxies` ## `POST` body Port for the HTTP proxy Set to `persist` to save proxy into the configuration file. Multiply the port definition given number of times List of `user`s `[string]`. This option has to be used along with `multiply_users` Enable SSL analyzing Choose the SSL library | value | description | | ---------- | ---------------- | | `open_ssl` | Open SSL Library | | `flex_tls` | Flex TLS Library | Interface or IP to listen on Customer name Zone name Zone password Hostname or IP of super proxy Super proxy port Determines what kind of connection will be used between Proxy Manager and Super Proxy | | | | ------- | - | | `http` | | | `https` | | | `socks` | | Automatically retry on super proxy failure Enable SSL connection/analyzing to insecure hosts Country State City ASN Data Center IP gIP A list of proxies from external vendors. Format: \[username:password@]ip\[:port] * proxy\[string] Default username for external vendor ips Default password for external vendor ips Default port for external vendor ips DNS resolving | | | | -------- | - | | `local` | | | `remote` | | Process reverse lookup via DNS Process reverse lookup via file Process reverse lookup via value Session for all proxy requests Use session per requesting host to maintain IP per host Session pool size Throttle requests above given number Proxy request rules Block or allow requests to be automatically sent through super proxy on error Operating System of the Peer IP Request headers * name\[string] * value\[string] Request debug info | | | | ------ | - | | `full` | | | `none` | | x-lpm-authorization header Unblocker Mobile UA Timezone ID to be used by the browser Browser screen size WebRTC plugin behavior in the browser BW limit params * days \[integer] * bytes \[integer] * renewable\[boolean] - Renew limit of bytes each period or use single period and stop usage once last day of period is reached. Default is true {/* TODO: Add description */} ```JSON Sample Response { "port":24000, "zone":"zone_name", "proxy_type":"persist", "customer":"customer_id", "password":"password", "whitelist_ips":[] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/proxies" -H "Content-Type: application/json" -d '{"proxy":{"port":24000,"zone":"ZONE","proxy_type":"persist","customer":"CUSTOMER","password":"password","whitelist_ips":[]}}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', method: 'POST', method: 'POST', url: 'http://127.0.0.1:22999/api/proxies', json: {'proxy':{'port':24000,'zone': 'ZONE','proxy_type':'persist','customer':'CUSTOMER','password':'password','whitelist_ips':[]}} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"proxy\":{\"port\":24000,\"zone\":\"ZONE\",\"proxy_type\":\"persist\",\"customer\":\"CUSTOMER\",\"password\":\"password\",\"whitelist_ips\":[]}}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/proxies") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` # Delete a Proxy Port Source: https://docs.brightdata.com/api-reference/proxy-manager/delete_a_proxy_port **API endpoint:** `DELETE` `/api/proxies/{PORT}` ## Path Parameter Existing proxy port number ```sh Shell curl -X DELETE "http://127.0.0.1:22999/api/proxies/{PORT}" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'DELETE', url: 'http://127.0.0.1:22999/api/proxies/{PORT}' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Delete("http://127.0.0.1:22999/api/proxies/{PORT}")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Delete, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.delete('http://127.0.0.1:22999/api/proxies/{PORT}') print(r.content) ``` # Delete Proxy Ports (multiple) Source: https://docs.brightdata.com/api-reference/proxy-manager/delete_proxy_ports **API endpoint:** `POST /api/proxies/delete` ## `POST` body Array of ports ```sh Shell curl "http://127.0.0.1:22999/api/proxies/delete" -H "Content-Type: application/json" -d '{"ports":[24000,24001]}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/proxies/delete', json: {'ports':[24000,24001]} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ports\":[24000,24001]}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/proxies/delete") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/delete"), Content = new StringContent(JsonConvert.SerializeObject(new { ports = [ 24000, 24001 ] }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'ports':[24000,24001]} r = requests.post('http://127.0.0.1:22999/api/proxies/delete', data=json.dumps(data)) print(r.content) ``` # Enable SSL Analyzing (All Ports) Source: https://docs.brightdata.com/api-reference/proxy-manager/enable_ssl_analyzing_on_all_proxy_ports **API endpoint:** `POST` `/api/enable_ssl` ```sh curl -X POST "http://127.0.0.1:22999/api/enable_ssl" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/enable_ssl' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/enable_ssl")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/enable_ssl") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.post('http://127.0.0.1:22999/api/enable_ssl') print(r.content) ``` # Generate API key Source: https://docs.brightdata.com/api-reference/proxy-manager/generate_token_for_token_based_authentication Generate API key for Token based Authentication **API endpoint:** `GET` `/api/gen_token` ```json Sample Response { "token": "RZD9vaQQaL6En7" } ``` ```sh Shell curl "http://127.0.0.1:22999/api/gen_token" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/gen_token' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/gen_token")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/gen_token") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/gen_token') print(r.content) ``` # Get all users Source: https://docs.brightdata.com/api-reference/proxy-manager/get_all_users **API endpoint:** `GET` `/api/lpm_users` ```JSON Sample Response [ { "email": "test@example.com", "password":"password" } ] ``` ```sh Shell curl "http://127.0.0.1:22999/api/lpm_users" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/lpm_users' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance().execute( Request.Get("http://127.0.0.1:22999/api/lpm_users") ).returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/lpm_users") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/lpm_users') print(r.content) ``` # Get allocated gIPs in a zone Source: https://docs.brightdata.com/api-reference/proxy-manager/get_allocated_gips_in_zone **API endpoint:** `GET` `/api/allocated_vips` ## Query Parameter Residential/Mobile exclusive zone name ```json Sample Response { "ips": [ "gIP_1", "gIP_2" ] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/allocated_vips?zone=ZONE" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/allocated_vips?zone=ZONE' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/allocated_vips?zone=ZONE")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/allocated_vips?zone=ZONE") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/allocated_vips?zone=ZONE') print(r.content) ``` # Get allocated IPs in a zone Source: https://docs.brightdata.com/api-reference/proxy-manager/get_allocated_ips_in_zone **API endpoint:** `GET` `/api/allocated_ips` ## Query Parameter Static (Datacenter/ISP) zone name ```JSON Sample Response { "ips": [ "10.0.0.1", "20.0.0.1" ] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/allocated_ips?zone=ZONE" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/allocated_ips?zone=ZONE' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/allocated_ips?zone=ZONE")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/allocated_ips?zone=ZONE") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/allocated_ips?zone=ZONE') print(r.content) ``` # Get Banned IPs Source: https://docs.brightdata.com/api-reference/proxy-manager/get_banned_ips **API endpoint:** `GET` `/api/banlist/{PORT}` `full=true` parameter is optional and can be used to provide additional details about each entry in the ban-list ```JSON Sample Response { "ips": [ "10.20.30.40", "50.60.70.80|example.com" ] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/banlist/{PORT}?full=true" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/banlist/{PORT}?full=true' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/banlist/{PORT}?full=true")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/banlist/{PORT}?full=true") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/banlist/{PORT}?full=true') print(r.content) ``` # Get current NodeJS version Source: https://docs.brightdata.com/api-reference/proxy-manager/get_currently_running_nodejs_version **API endpoint:** `GET` `/api/node_version` ```JSON Sample Response { "current": { "options": { "loose": false, "includePrerelease":false }, "loose": false, "raw":"v12.16.1\n", "major":12, "minor":16, "patch":1, "prerelease":[], "build":[], "version":"12.16.1" }, "satisfied": true, "recommended": ">=10.0" } ``` {/* TODO: Missing Code Examples */} # Get effective configuration for all running proxies Source: https://docs.brightdata.com/api-reference/proxy-manager/get_effective_configuration_for_all_running_proxies **API endpoint:** `GET` `/api/proxies_running` ```JSON Sample Response [ { "port": 24000, "zone": "ZONE", "proxy_type": "persist", "customer": "CUSTOMER", "password": "password", "whitelist_ips":[] } ] ``` ```sh Shell curl "http://127.0.0.1:22999/api/proxies_running" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/proxies_running' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/proxies_running")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies_running") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/proxies_running') print(r.content) ``` # Get enabled Zone's configuration Source: https://docs.brightdata.com/api-reference/proxy-manager/get_enabled_zones_configuration **API endpoint:** `GET` `/api/zones` ```JSON Sample Response { "name": "ZONE", "perm": "country ip route_all route_dedicated", "plan": {}, "password":"password" } ``` ```sh Shell curl "http://127.0.0.1:22999/api/zones" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/zones' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/zones")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/zones") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import requests r = requests.get('http://127.0.0.1:22999/api/zones') print(r.content) ``` # Get explicit configuration of specified Proxies Source: https://docs.brightdata.com/api-reference/proxy-manager/get_explicit_configuration_of_all_or_specified_proxies **API endpoint:** `GET` `/api/proxies/{PORT}` PORT parameter is optional. You can skip it to fetch all the proxies\* ```JSON Sample Response [ {"port": 24000, "zone": "ZONE", "proxy_type": "persist", "customer": "CUSTOMER", "password": "password", "whitelist_ips": [] }, { "port": 33335, "zone": "ZONE", "listen_port": 33335, "customer": "CUSTOMER", "password":"password" } ] ``` ```sh Shell curl "http://127.0.0.1:22999/api/proxies/{PORT}" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/proxies/{PORT}' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/proxies/{PORT}")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/proxies/{PORT}') print(r.content) ``` # Get general settings Source: https://docs.brightdata.com/api-reference/proxy-manager/get_general_settings **API endpoint:** `GET` `/api/settings` ```JSON Sample Response { "customer": "CUSTOMER", "zone": "ZONE", "password": "password", "www_whitelist_ips": [], "whitelist_ips": [], "fixed_whitelist_ips": [], "read_only": false, "config": "/home/user/proxy_manager/.luminati.json", "test_url":"http://lumtest.com/myip.json", "logs":1000, "log":"notice", "har_limit": 1024, "request_stats": true, "dropin": true, "pending_ips": [], "pending_www_ips": [], "zagent": false, "sync_config": true } ``` ```sh Shell curl "http://127.0.0.1:22999/api/settings" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/settings' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/settings")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/settings") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/settings') print(r.content) ``` # Get HAR Logs Source: https://docs.brightdata.com/api-reference/proxy-manager/get_har_logs API endpoint: `GET` `/api/logs` Number of logs to get from tail Number of logs to get from tail Maximum number of requests to be fetched regex search query for the URL lower bound for port number upper bound for port number filter requests by status code parameter to be sorted by is descending sorting direction {/* TODO: Missing Sample Response */} ```sh Shell curl "http://127.0.0.1:22999/api/logs" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/logs' }).then(function(data){ console.log(data);}, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/logs")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/logs") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/logs') print(r.content) ``` # Get Proxy Manager version Source: https://docs.brightdata.com/api-reference/proxy-manager/get_proxy_manager_version **API endpoint:** `GET` `/api/version` ```json Sample Response { "version":"1.280.385", "argv":"" } ``` ```sh Shell curl "http://127.0.0.1:22999/api/version" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/version' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/version")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/version") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/version') print(r.content) ``` # Get Proxy Port Status Source: https://docs.brightdata.com/api-reference/proxy-manager/get_proxy_port_status **API endpoint:** `GET` `/api/proxy_status/{PORT}` Existing proxy port number ```JSON Sample Response { "status":"ok", "status_details":[] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/proxy_status/{PORT}" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/proxy_status/{PORT}' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance().execute( Request.Get("http://127.0.0.1:22999/api/proxy_status/{PORT}") ).returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/proxy_status/{PORT}") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/proxy_status/{PORT}') print(r.content) ``` # Get Recent Status Source: https://docs.brightdata.com/api-reference/proxy-manager/get_recent_stats **API endpoint:** `GET /api/recent_stats` ```JSON Sample Response { "ports": {}, "status_code": [], "hostname": [], "protocol": [], "total": 0, "success": 0, "ssl_enable":true} ``` ```sh Shell curl "http://127.0.0.1:22999/api/recent_stats" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/recent_stats' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance().execute( Request.Get("http://127.0.0.1:22999/api/recent_stats") ).returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/recent_stats") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/recent_stats') print(r.content) ``` # Get tail from the Log file Source: https://docs.brightdata.com/api-reference/proxy-manager/get_tail_from_the_log_file **API endpoint:** `GET` `/api/general_logs` Number of logs to get from tail {/* TODO: Missing Sample Response */} ```sh Shell curl "http://127.0.0.1:22999/api/general_logs?limit=5" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/general_logs?limit=5' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/general_logs?limit=5")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/general_logs?limit=5") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/general_logs?limit=5') print(r.content) ``` # Get the latest Proxy Manager (PM) version Source: https://docs.brightdata.com/api-reference/proxy-manager/get_the_latest_proxy_manager_versions **API endpoint:** `GET` `/api/last_version` ```json Sample Response { "version": "1.280.385", "newer": false, "versions": [ { "ver": "1.280.385", "type": "stable", "changes": [ { "type": "star", "text": "Add render option for unblocker and serp zones" } ] } ] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/last_version" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/last_version' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/last_version")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/last_version") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/last_version') print(r.content) ``` # Refresh IP/gIP in a zone Source: https://docs.brightdata.com/api-reference/proxy-manager/refresh_ips_or_gips_in_zone **API endpoint:** `POST` `/api/refresh_ips` ## `POST` body Zone name Static IPs. e.g. `ips=["ip1","ip2"]` gIPs. e.g. `vips=["gip1","gip2"] [array]` ```json Sample Response { "ips": [ { "ip":"10.0.0.1", "maxmind":"us" }, { "ip":"20.0.0.1", "maxmind":"us" } ] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/refresh_ips" -H "Content-Type: application/json" -d '{"zone":"ZONE","ips":["10.0.0.1"]}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/refresh_ips', json: {'zone':'ZONE','ips':['10.0.0.1']} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"zone\":\"ZONE\",\"ips\":[\"10.0.0.1\"]}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/refresh_ips") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/refresh_ips"), Content = new StringContent(JsonConvert.SerializeObject(new { zone = "ZONE", ips = ["10.0.0.1"] }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import requests data = {'zone':'ZONE','ips':['10.0.0.1']} r = requests.post('http://127.0.0.1:22999/api/refresh_ips', data=json.dumps(data)) print(r.content) ``` # Refresh Proxy Manager Port Session Source: https://docs.brightdata.com/api-reference/proxy-manager/refresh_proxy_manager_port_sessions **API endpoint:** `GET` `/api/refresh_sessions/{PORT}` ## Path Parameter Existing proxy port number ```sh Shell curl "http://127.0.0.1:22999/api/refresh_sessions/{PORT}" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/refresh_sessions/{PORT}' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/refresh_sessions/{PORT}")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/refresh_sessions/{PORT}") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import requests r = requests.get('http://127.0.0.1:22999/api/refresh_sessions/{PORT}') print(r.content) ``` # Restart Proxy Manager Source: https://docs.brightdata.com/api-reference/proxy-manager/restart_proxy_manager **API endpoint:** `POST` `/api/restart` ```sh Shell curl -X POST "http://127.0.0.1:22999/api/restart" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/restart' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/restart")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/restart") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.post('http://127.0.0.1:22999/api/restart') print(r.content) ``` # Shutdown Proxy Manager Source: https://docs.brightdata.com/api-reference/proxy-manager/shutdown_proxy_manager **API endpoint:** `POST` `/api/shutdown` ```sh Shell curl -X POST "http://127.0.0.1:22999/api/shutdown" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/shutdown' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/shutdown")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/shutdown") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import requests r = requests.post('http://127.0.0.1:22999/api/shutdown') print(r.content) ``` # Allowlist IPs to Access UI Source: https://docs.brightdata.com/api-reference/proxy-manager/ui-allowlist-ips **API endpoint:** `PUT` `/api/add_whitelist_ip` API key API Key should be generated using [Generate API Key](/api-reference/proxy-manager/generate_token_for_token_based_authentication) ## `PUT` body IP to allowlist e.g. `ip="1.2.1.2"` ```sh Shell curl -X PUT "http://127.0.0.1:22999/api/whitelist_ip" -H "Authorization: API key" -H "Content-Type: application/json" -d '{"ip":"1.2.1.2"}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'PUT', url: 'http://127.0.0.1:22999/api/add_whitelist_ip', json: {'ip':'1.2.1.2'}, headers: {'Authorization': 'API key'}, }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ip\":\"1.2.1.2\"}"; String res = Executor.newInstance() .addHeader("Authorization", "API key") .execute(Request.Put("http://127.0.0.1:22999/api/add_whitelist_ip") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent() .asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Put, RequestUri = new Uri("http://127.0.0.1:22999/api/add_whitelist_ip"), Headers = {{"Authorization", "API key"}}, Content = new StringContent(JsonConvert.SerializeObject(new { ip = "1.2.1.2" }), Encoding.UTF8, "application/json") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import json import requests data = {'ip':'1.2.1.2'} headers = {'Authorization': 'API key'} r = requests.put( 'http://127.0.0.1:22999/api/add_whitelist_ip', data=json.dumps(data) ) print(r.content) ``` # Unban an IP Source: https://docs.brightdata.com/api-reference/proxy-manager/unban_an_ip **API endpoint:** `POST` `/api/proxies/{PORT}/unbanip` ## Path Parameter Existing proxy port number ## `POST` body IP to unban. e.g. `ip="1.2.1.2"` Unban the IP for sending requests to the specified domain. ```sh Shell curl "http://127.0.0.1:22999/api/proxies/{PORT}/unbanip" -H "Content-Type: application/json" -d '{"ip":"1.2.1.2","domain":"example.com"}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/proxies/{PORT}/unbanip', json: { 'ip': '1.2.1.2', 'domain': 'example.com' } }).then(function(data){ console.log(data);}, function(err){ console.error(err);}); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ip\":\"1.2.1.2\",\"domain\":\"example.com\"}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/proxies/{PORT}/unbanip") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}/unbanip"), Content = new StringContent(JsonConvert.SerializeObject(new { ip = "1.2.1.2", domain = "example.com" }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'ip':'1.2.1.2','domain':'example.com'} r = requests.post('http://127.0.0.1:22999/api/proxies/{PORT}/unbanip', data=json.dumps(data)) print(r.content) ``` # Unban IPs (multiple) Source: https://docs.brightdata.com/api-reference/proxy-manager/unban_ips **API endpoint:** `POST` `/api/proxies/{PORT}/unbanips` ## Path Parameter Existing proxy port number ```sh Shell curl "http://127.0.0.1:22999/api/proxies/{PORT}/unbanips" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/proxies/{PORT}/unbanips' }).then(function(data){ console.log(data);}, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance().execute( Request.Post("http://127.0.0.1:22999/api/proxies/{PORT}/unbanips" )).returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}/unbanips") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.post('http://127.0.0.1:22999/api/proxies/{PORT}/unbanips') print(r.content) ``` # Update a Proxy Port Source: https://docs.brightdata.com/api-reference/proxy-manager/update_a_proxy_port **API endpoint:** `PUT` `/api/proxies/{PORT}` ## Path Parameter Existing proxy port number ## `PUT` body Port for the HTTP proxy Set to `persist` to save proxy into the configuration file. Multiply the port definition given number of times List of `user`s `[string]`. This option has to be used along with `multiply_users` Enable SSL analyzing Choose the SSL library | value | description | | ---------- | ---------------- | | `open_ssl` | Open SSL Library | | `flex_tls` | Flex TLS Library | Interface or IP to listen on Customer name Zone name Zone password Hostname or IP of super proxy Super proxy port Determines what kind of connection will be used between Proxy Manager and Super Proxy | | | | ------- | - | | `http` | | | `https` | | | `socks` | | Automatically retry on super proxy failure Enable SSL connection/analyzing to insecure hosts Country State City ASN Data Center IP gIP A list of proxies from external vendors. Format: \[username:password@]ip\[:port] * proxy\[string] Default username for external vendor ips Default password for external vendor ips Default port for external vendor ips DNS resolving | | | | -------- | - | | `local` | | | `remote` | | Process reverse lookup via DNS Process reverse lookup via file Process reverse lookup via value Session for all proxy requests Use session per requesting host to maintain IP per host Session pool size Throttle requests above given number Proxy request rules Block or allow requests to be automatically sent through super proxy on error Operating System of the Peer IP Request headers * name\[string] * value\[string] Request debug info | | | | ------ | - | | `full` | | | `none` | | x-lpm-authorization header Unblocker Mobile UA Timezone ID to be used by the browser Browser screen size WebRTC plugin behavior in the browser BW limit params * days \[integer] * bytes \[integer] * renewable\[boolean] - Renew limit of bytes each period or use single period and stop usage once last day of period is reached. Default is true ```JSON Sample Response { "port":24000, "zone":"ZONE", "proxy_type":"persist", "customer":"CUSTOMER", "password":"password", "whitelist_ips":[] } ``` ```sh Shell curl -X PUT "http://127.0.0.1:22999/api/proxies/{PORT}" -H "Content-Type: application/json" -d '{"proxy":{"port":24000,"zone":"ZONE","proxy_type":"persist","customer":"CUSTOMER","password":"password","whitelist_ips":[]}}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'PUT', url: 'http://127.0.0.1:22999/api/proxies/{PORT}', json: {'proxy':{'port':24000,'zone': 'ZONE','proxy_type':'persist','customer':'CUSTOMER','password':'password','whitelist_ips':[]}} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"proxy\":{\"port\":24000,\"zone\":\"ZONE\",\"proxy_type\":\"persist\",\"customer\":\"CUSTOMER\",\"password\":\"password\",\"whitelist_ips\":[]}}"; String res = Executor.newInstance() .execute(Request.Put("http://127.0.0.1:22999/api/proxies/{PORT}")) .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Put, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}"), Content = new StringContent(JsonConvert.SerializeObject(new { proxy = new { port = 24000, zone = "ZONE", proxy_type = "persist", customer = "CUSTOMER", password = "password", whitelist_ips = [] } }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import requests data = {'proxy':{'port':24000,'zone': 'ZONE','proxy_type':'persist','customer':'CUSTOMER','password':'password','whitelist_ips':[]}} r = requests.put('http://127.0.0.1:22999/api/proxies/{PORT}', data=json.dumps(data)) print(r.content) ``` # Update General Setting Source: https://docs.brightdata.com/api-reference/proxy-manager/update_general_settings **API Endpoint:** `PUT` `/api/settings` ## `PUT` Body: Default zone Allowlist ip list for granting access to browser admin UI Default for all proxies allowlist ip list for granting access to them Number of request logs to store Enable requests statistics ```JSON Sample Response { "customer":"CUSTOMER", "zone":"ZONE", "password":"password", "www_whitelist_ips":[], "whitelist_ips":[], "fixed_whitelist_ips":[], "read_only":false, "config":"/home/user/proxy_manager/.luminati.json", "test_url":"http://lumtest.com/myip.json", "logs":1000, "log":"notice", "har_limit":1024, "request_stats":true, "dropin":true, "pending_ips":[], "pending_www_ips":[], "zagent":false, "sync_config":true } ``` ```sh Shell curl -X PUT "http://127.0.0.1:22999/api/settings" -H "Content-Type: application/json" -d '{"zone":"ZONE","www_whitelist_ips":[],"whitelist_ips":[],"logs":1000,"request_stats":true}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'PUT', url: 'http://127.0.0.1:22999/api/settings', json: {'zone': 'ZONE','www_whitelist_ips':'[]','whitelist_ips':'[]','logs':1000,'request_stats':true} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"zone\":\"ZONE\",\"www_whitelist_ips\":[],\"whitelist_ips\":[],\"logs\":1000,\"request_stats\":true}"; String res = Executor.newInstance() .execute(Request.Put("http://127.0.0.1:22999/api/settings")) .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Put, RequestUri = new Uri("http://127.0.0.1:22999/api/settings"), Content = new StringContent(JsonConvert.SerializeObject(new { zone = "ZONE", www_whitelist_ips = [], whitelist_ips = [], logs = 1000, request_stats = true }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'zone': 'ZONE','www_whitelist_ips':[],'whitelist_ips':[],'logs':1000,'request_stats':true} r = requests.put('http://127.0.0.1:22999/api/settings', data=json.dumps(data)) print(r.content) ``` # Upgrade Proxy Manager Source: https://docs.brightdata.com/api-reference/proxy-manager/upgrade_proxy_manager **API endpoint:** `POST` `/api/upgrade` ```sh Shell curl -X POST "http://127.0.0.1:22999/api/upgrade" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/upgrade' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/upgrade")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/upgrade") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.post('http://127.0.0.1:22999/api/upgrade') print(r.content) ``` # Request and Response C-Tag Source: https://docs.brightdata.com/api-reference/proxy/c-tag When you're dealing with a massive volume of requests, keeping track of individual responses can be challenging. Enter C-Tag, a simple yet powerful solution that improves request tracking. With C-Tag, users include a unique `c_tag` flag in their requests. In response, businesses echo back the same tag in the header. This seamless exchange ensures that each response is bound to its corresponding request, eliminating confusion and streamlining data management. Add any C-Tag text in your request and get the identical sting in your response header: ```sh Sample Request curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--c_tag-: ``` ```js Output headers x-brd-c_tag: ``` # Carrier-specific Proxy peer IP (Mobile only) Source: https://docs.brightdata.com/api-reference/proxy/carrier_specific_proxy_peer_ip_mobile_only_ * You can choose to use a specific carrier from this list: a1, \[aircel]\(%20), airtel, att, celcom, chinamobile, claro, comcast, cox, digi, dt, docomo, dtac, etisalat, idea, kyivstar, meo, megafon, mtn, mtnza, mts, optus, orange, qwest, reliance\_jio, robi, sprint, telefonica, telstra, tmobile, tigo, tim, verizon, vimpelcom, vodacomza, vodafone, vivo, zain, vivabo, telenormyanmar, kcelljsc, swisscom, singtel, asiacell, windit, cellc, ooredoo, drei, umobile, cableone, proximus, tele2, mobitel, o2, bouygues, free, sfr, digicel * Example for carrier Deutsche Telekom: ```sh Carrier: Deutsche Telekom brd-customer--zone--carrier-dt ``` ```sh Carrier: Sprint brd-customer--zone--carrier-sprint ``` # Configuring DNS Resolution Source: https://docs.brightdata.com/api-reference/proxy/configuring_dns_resolution It is possible to direct Bright Data proxy network to use either remote or local DNS resolution. This is useful if your target has more than two IPs (depending on where the DNS query is made) and would like to target a specific IP of the target site. All requests to Bright Data start with a connection to a Super Proxy (brd.superproxy.io). The Super Proxy then send the request to local servers in different countries, who can send the request if needed to Residental proxies ("peers"). When using `local` DNS resolution, domain names are resolved and cached by Bright Data Super Proxy network server, the resolved IP is used the real request by the proxy in the actual country. When requests are made using `remote` they initially received by a Super Proxy (brd.superproxy.io). The Super Proxy does a preliminary DNS resolution locally, only to check if domain is valid. If the domain is valid, the request is sent to a Bright Data server that is closest to the country where the proxy is located. * If the Bright Data server is in the same country as the proxy, DNS is performed on the server, the resolved IP is used the real request. This DNS resolution is usually faster than DNS resolution by the proxy. * If the Bright Data server is in a different country, DNS is performed on actual proxy. ```sh DNS (Local) curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--dns-local: ``` ```sh DNS (Remote) curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--dns-remote: ``` When using remote DNS resolution you may see some performance degradation due to local DNS resolution lag. # Geolocation Targeting Source: https://docs.brightdata.com/api-reference/proxy/geolocation-targeting Geolocation targeting allows you to target specific locations based on `Country`, `City`, `State`, `ASN`, or `ZIP code` ## Country Targeting You can target a specific country by adding a country code to your username, like `username-country-country_code`. Use only two-letter country codes in lowercase. ```sh Shell curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--country-us: ``` ### Default countries' selection In all Bright Data shared pool proxies: Datacenter, ISP, Residential and Mobile share pools (also known as "Rotating proxies") we offer a default country selection during zone configuration. When a country(s) is selected, Bright Data will attempt to allocate a peer from the selected default countries. The allocation sequence is as follows: If there are default countries configured: 1. Select a random country from the **default countries** list. 2. Attempt to allocate a peer in that country 3. If peer found: route request to the peer 4. If peer not found: fail the request and reply with error In case there are **no default countries selected,** Bright Data will allocate a random proxy peer from the pool and route the request through it. ### EU Countries targeting Bright Data proxy networks as well as our Unlocker API allow random peer selection specifically from **EU-member countries** via the use of the `-country-eu` flag. When needing to target EU peers, this will improve success rates over targeting from just one EU country, while still keeping the peer within the EU. ## City Targeting City targeting works best in Residential & Mobile proxies. City targeting in Datacenter & ISP proxies has deprecated and we do not recommend using it. You can target a specific city within a country by adding a city name to the username, like `username-country-country_code-city-city_code`. The city\_codemust be in lowercase only and have no spaces, for example `-city-sanfrancisco` ```sh Shell curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--country-us-city-sanfrancisco: ``` ## State Targeting Residential proxies only For relevant counties (e.g., USA and Australia), you can target a state within a country by adding to the `username-country-country_code-state-state_code`. Use only two-letter state codes in lowercase. ```sh Country: USA curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--country-us-state-ny: ``` ```sh Country: Australia curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--country-au-state-wa: ``` ## ASN Targeting Residential proxies only You can target a specific ASN from the [list of ASNs](http://bgp.potaroo.net/cidr/autnums.html). Choosing an ASN country is done as following: -asn-ASN\_NUMBER ```sh Shell curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--asn-56386: ``` ## ZIP Code Targeting Residential proxies only You can target proxy peers by a ZIP code of a specific area. Here is an example of targeting an IP from Memphis, US and zip code 12345: ```sh Shell curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--city-memphis-zip-12345: ``` # Disable peer rotation Source: https://docs.brightdata.com/api-reference/proxy/keep_same_peer_in_session You can avoid assigning a new peer to the session in case the peer used in the session went offline. When using this option parameter, you will get a `502` error code for your request when that peer used for your session went offline, with error message: ```sh Error Message 502 Proxy Error: server_error Failed to establish connection with peer ``` This workflow can be useful for Residential and Mobile proxy zones, which consist of real PCs and mobile devices It's used by adding `-const` after your session name, for example `-session-mystring12345-const` ```sh Shell curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--session-mystring12345-const: ``` # OS Targeting Source: https://docs.brightdata.com/api-reference/proxy/os_targeting Bright Data allows targeting the following **Operating Systems**: ```sh Windows curl "https://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--os-windows: ``` ```sh macOS curl "https://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--os-osx: ``` ```sh Android curl "https://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--os-android: ``` # Request Error Handling Source: https://docs.brightdata.com/api-reference/proxy/request_error_handling | value | description | | ---------- | --------------------------------------------------------------------------------- | | `pass_dyn` | if a request can't pass via proxy peer, automatically pass it via the Super Proxy | | `block` | if a request can't pass via proxy peer, block it and don't send via Super Proxy | ```sh pass_dyn curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--route_err-pass_dyn: ``` ```sh block curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--route_err-block: ``` # Proxy and IP Rotation control Source: https://docs.brightdata.com/api-reference/proxy/rotate_ips How does proxy IP rotation work, and how can you configure or control it. ## Default proxy IP rotation in Shared Pool Shared pool proxies, also known as "Rotating Proxies" are available in all Bright Data proxy networks: Datacenter, ISP, Residential & Mobile. By default, Bright Data assigns a random proxy from the available proxies in the zone, according to your configuration. For example: if you chose to target a specific country, for example Canada, using the `-country=ca` parameter, we will randomly assign a proxy in Canada out of all the proxies located in Canada that we have in the pool. The next request, will be directed to a different proxy, and so on, selected randomly from our pool. Executing requests in high rate or by multiple calling agents may result in reuse of the same proxy IPs. ## Default proxy IP rotation in dedicated proxy zone When you configure a set of dedicated proxies dedicated proxies are assigned to your exclusive use in the following manner: 1. In Datacenter and ISP network: proxies are exclusively yours for all domains. 2. In Residential and Mobile networks: are exclusively yours towards **specific domains**. In dedicated proxy zones we will rotate to next available proxy to serve the next request. ### What are "Static residential" proxies? "Static residential proxies" are datacenter hosted proxies registered as owned by Internet Service Providers (ISP) and enable have a fixed, static IP address. In Bright Data we name these proxies: "ISP Proxies" and not "Static residential". ### Rotation in dedicated residential proxies Since residential proxies vary in their availability, we group together a number of proxies to a group which we call `gIP`. This `gIP` gets a unique identifier, and peers assigned to it (members of the group) are selected according to configuration you specify. For example: if you require 3 dedicated residential ips towards target [http://example.com](http://example.com) ; we will provide 3 `gIPs` which we rotate over to serve your requests. ## Using explicit proxy for multiple requests: `ip` To explicitly target a request to a specific proxy, use the `-ip` parameter in your proxy zone username request. This is when you issue a set of requests on a different connection (like a sequence of `curl` requests). ## Using the same proxy for multiple requests: `session` To keep the same proxy IP address for multiple requests, use the `-session` parameter with a session identifier you create and control. Each request, carrying the same `-session` value will be forwarded to the **same proxy IP**. The `-session` parameter value is generated by you and controlled by you, yet its format should include alphanumeric characters only. Using special characters like `-` or `*` will result in errors. If you work with native proxy access (within code libraries or 3rd party tools which take proxy ip:port and credentials) then you embed the session parameter in the user name parameter as shown below for session identified `mystring12345`. If you are using our proxy REST API, then the `session` parameter in request `body` shall carry the session identified value. ```sh Shell curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--session-mystring12345: ``` ## Disable proxy rotation in case of unavailable peer Our default behavior is to direct your request to another peer in case the peer is unavailable to allow fluent operations. In case you would like to prevent this, and only use a single peer for a session, and fail the request in case the peer is unavailable, add the `-const `option to the `session `parameter. When using the `-const` option, in case peer is unavailable you will get a `HTTP 502` Error. ```sh Shell curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--session-mystring12345-const: ``` ## Losing session context The following describes scenarios where session context can be lost, and consequent requests will be executed by **different** proxies. ### Session Idle Time Session idle time is 5 minutes. In case there are more than 5 minutes idle time between two consequent requests, the second request, although carrying the same `session` parameter as the first, will use a randomly selected proxy from the pool. ### Session origin must be from same global region In order to keep the same session, you must ensure your calling process issuing the request is not switching between global regions. We recognize 3 global regions: 1. AMER: North, central and south America 2. EMEA: Europe, Middle East and Africa 3. APAC: Asia and Pacific Example: If you issue 2 consequent requests, one from **your company** server in Canada (in AMER region) and one from **your company server** in France, even if both requests will carry the same `session` parameter, they will be executed by different proxy peers. ### Other parameters must be identical In case 2 consequent requests carry different country, city or any other parameter: session context will be lost , even if they carry the same `session` parameter. ### Error on lost session By default, is session is lost or reset, Bright Data will randomly assign a proxy to your request. If you wish to avoid this default assignment, and error instead, use the `-const` parameter in your request. If the same peer is not available from any reason to 2 consequent requests, the second request will result in `HTTP Error 502` ```sh Shell curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--session-mystring12345-const: ``` ## Keeping track of requests and their responses (C-Tag) We've implemented an optional feature that can greatly help tracking and linking requests with their corresponding responses, see the following article for more information: [more info on C-Tag](https://docs.brightdata.com/api-reference/proxy/c-tag) # Select a specific gIP Source: https://docs.brightdata.com/api-reference/proxy/select_a_specific_gip Option available only for Residential or Mobile zones with multiple IPs allocated. To target a specific gIP allocated to your zone, use -gip-gip\_name request parameter. ```sh curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--gip-us_7922_fl_hollywood_0: ``` # Select a specific IP Source: https://docs.brightdata.com/api-reference/proxy/select_a_specific_ip Option available only for zones with multiple IPs allocated. To target a specific IP allocated to your zone, use -ip-ip\_address request parameter. ```sh Shell curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--ip-1.1.1.1.1: ``` # null Source: https://docs.brightdata.com/api-reference/proxy/select_super_proxy_in_specific_country {/*tags:nosophie*/} Important note: This article is relevant ONLY for choosing a 'super proxy', which is rarely needed. In order to select a specific standard proxy (for example, Residential Proxy, Datacenter proxy or ISP proxy) * In **Shared proxy zones**, you can add the '-country' flag (and also '-city') to your proxy username like this: username-country-country\_code-city-city\_code. Country codes are standard 2-character codes. The city\_code must be in lowercase only and have no spaces, for example -city-sanfrancisco. * In **Dedicated proxy zones**, you can select the countries and cities in your zone configuration page by following these steps: 1. Add a new proxy zone and set the IP type to 'Dedicated' 2. Choose the domains you would like to target with this zone. 3. Select the number of dedicated proxies you want to allocate to your zone 4. Select the countries you want to target 5. Optional - After selecting the countries you want, click 'Add city' next to each country you want to target in city resolution 6. Add the chosen cities for each country and save the zone. 7. Your new Dedicated proxy zone will allocate the number of grproxies you requested evenly between the countries and cities you selected. If you actualy require our load balancer to provide a super proxy in a specific country: If no specific server country is selected, Bright Data will automatically choose the optimal load-balancing server for speed. If speed is your primary concern, use this feature with caution, as it may negatively impact performance over time. Choosing the super proxy country can be done like this: `servercountry-{country}`.brd.superproxy.io. E.g. `servercountry-gb.brd.superproxy.io` ```sh Shell curl "https://example.com" --proxy servercountry-gb.brd.superproxy.io:33335 --proxy-user brd-customer--zone-: ``` # Send requests directly from Super Proxy Source: https://docs.brightdata.com/api-reference/proxy/send_requests_directly_from_super_proxy You can choose to perform the request from the super proxy directly instead of the IP of the peer. In that case, the IP of the request will be the one of the Super Proxy by adding `-direct` to your request authorization string. ```sh Shell curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--direct: ``` # Crawl API Source: https://docs.brightdata.com/api-reference/rest-api/scraper/crawl-api api-reference/rest-api/scraper/crawl-rest-api.json post /datasets/v3/trigger Bright Data Web Scraper API enables you to trigger a collection for automated web data extraction. Related guide: [Crawl API Introduction](/scraping-automation/crawl-api/overview) # Trigger collection Source: https://docs.brightdata.com/api-reference/rest-api/scraper/trigger-collection api-reference/rest-api/scraper/scraper-rest-api.json post /datasets/v3/trigger Bright Data Web Scraper API enables you to trigger a collection for automated web data extraction. Related guide: [Web Scraper API Introduction](/scraping-automation/web-scraper-api/introduction) # Scrape SERP Source: https://docs.brightdata.com/api-reference/rest-api/serp/scrape-serp serp-rest-api POST /request Extract search engine results using Bright Data SERP API. Extract structured data from major search engines including Google, Bing, Yandex, DuckDuckGo, and more. Get organic results, paid ads, local listings, shopping results, and other SERP features. Related guide: [SERP API Introduction](/scraping-automation/scraping-automation/serp-api/introduction) For an easy start using our tools check out our new Python SDK. # Unlock website Source: https://docs.brightdata.com/api-reference/rest-api/unlocker/unlock-website unlocker-rest-api POST /request Bright Data Unlocker API enables you to bypass anti-bot measures, manages proxies, and solves CAPTCHAs automatically for easier web data collection. Related guide: [Unlocker API Introduction](/scraping-automation/web-unlocker/introduction) For an easy start using our tools check out our new Python SDK. # All Classification Data Source: https://docs.brightdata.com/api-reference/scraping-shield-api/all-classification-data scraping-shield-rest-api GET /shield/class Get all classification data # Domains by Classification Source: https://docs.brightdata.com/api-reference/scraping-shield-api/domains-by-classification-data scraping-shield-rest-api GET /shield/domains_by_class Returns usage data per domain for the chosen classification # Samples by Classification Source: https://docs.brightdata.com/api-reference/scraping-shield-api/samples-by-classification-data scraping-shield-rest-api GET /shield/samples Returns sample requests data for the chosen classification # Zone by Classification Source: https://docs.brightdata.com/api-reference/scraping-shield-api/zones-by-classification-data scraping-shield-rest-api GET /shield/zones_by_class Returns usage data per zone for the chosen classification # Terminology Source: https://docs.brightdata.com/api-reference/terminology ## Zones Zones are the foundation for how you configure and manage Bright Data products. Each zone represents a specific product or setup, such as Residential, Mobile, Datacenter, or Unlocker API. Access your zones [here](https://brightdata.com/cp/zones). bright-data-zones.png ### What is a zone? A zone is a logical container that defines how a Bright Data product behaves. When you enable a product, a corresponding zone is created. Each zone includes its own configuration settings, such as: * Targeting rules (e.g., country, city, ASN) * Output format * Headers and request behavior * Access permissions ### Using zones in your workflow You can create multiple zones for the same product to support different use cases, targets, or operational needs. This gives you more control and flexibility when managing traffic. Example use cases: * `zone_1` for targeting example.com * `zone_2` for targeting anotherexample.com By separating traffic into different zones, you can: * Apply unique settings per target or workflow * Monitor usage and performance independently * Simplify debugging and optimization * Maintain cleaner, more organized configurations You can rename your zones at any time to better reflect their purpose or target. ## Snapshot Snapshot in Bright Data represent saved state of data collection at the time of trigger. It is used to track, manage, and retrieve the exact data that was collected during a specific run of a dataset. ### What is Snapshot? A snapshot is a record of a single data collection event. Each time a dataset is triggered—whether manually or via API, a snapshot is created to store the results and the inputs used for that specific run. Snapshots are especially useful when: * You need to retrieve historical data from previous runs * You want to verify what inputs were used to trigger a collection * You are delivering data in batches and need to track how many parts were generated ### Example Scenarios * You’ve triggered a dataset to collect product listings from an e-commerce site. A snapshot is automatically created, capturing the results and the inputs (e.g., URL, filters, headers). * You’ve requested the data to be delivered in batches. You can use the snapshot to check how many parts were generated and ensure the delivery settings match your request. ### Snapshot ID A Snapshot ID is a unique identifier assigned to a specific data [snapshot](#Snapshot). Example: `s_manlyn268d2p9hdmx` ## Dataset A [dataset](https://brightdata.com/products/datasets) in the context of Bright Data refers to a structured collection of data that has been gathered and organized for specific use cases, such as eCommerce, social media, real estate, etc. These datasets are available in the Dataset Marketplace. ### Dataset ID A Dataset ID is a unique identifier assigned to each [dataset](#dataset) in the Dataset Marketplace. It helps in identifying and accessing specific datasets for data retrieval and management. You can find all Dataset IDs of all Scraper APIs, you can use the [dataset lists API endpoint](/api-reference/marketplace-dataset-api/get-dataset-list) to retrieve a list of available datasets. # Send your first Request Source: https://docs.brightdata.com/api-reference/unlocker/sending_first_request Here's an example of the most simple Unlocker API request using cURL that returns a JSON: ```sh curl "https://geo.brdtest.com/welcome.txt" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone-: ``` You can find your API credentials including Username (`Customer_ID`), Zone name, and Password, within your proxy zone’s ‘Overview’ tab. | | | | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `brd.superproxy.io` | Address of our load balancer that will find the fastest Super Proxy for your request | | [`33335`](https://docs.brightdata.com/general/faqs#which-port-shall-i-use-22225-or-33335) | Infrastructure port of our Super Proxies that is used to receive your requests | | `-user brd-customer--zone-`  | Username authentication. In its most basic form, it defines your username and what zone you will use for your request.  | | ``  | Zone password. All zones have passwords that are used for authentication | | [https://geo.brdtest.com/welcome.txt](https://geo.brdtest.com/welcome.txt) | Replace with your target domain. This is just a placeholder that goes to our server for testing purposes. | For an in-depth interactive display of all the API use cases, integrations, and preferences, please see our [API examples](https://brightdata.com/cp/zones/proxy_examples) page. You'll need to [sign up](https://brightdata.com/) (for free), and **login** to the Bright Data control panel in order to access this API tool. If you add your payment method, you’ll even receive a \$5 credit to get you started! # null Source: https://docs.brightdata.com/api-reference/web-scraper-api/delivery-apis/deliver-snapshot dca-api POST /datasets/v3/deliver/{snapshot_id} Deliver the snapshot content to the specified storage # null Source: https://docs.brightdata.com/api-reference/web-scraper-api/delivery-apis/download-snapshot dca-api GET /datasets/v3/snapshot/{snapshot_id} Download the snapshot content If needed, the result can be divided into several parts: 1. Download the first part by specifying a `batch_size` and `part=1` 2. Check how many part were created using this endpoint 3. Download the rest of the parts by changing the `part` parameter. (`batch_size`, `format` and `compress` must stay the same for each call) # null Source: https://docs.brightdata.com/api-reference/web-scraper-api/management-apis/cancel-snapshot post /datasets/v3/snapshot/{snapshot_id}/cancel Cancel the running collection # null Source: https://docs.brightdata.com/api-reference/web-scraper-api/management-apis/get-snapshot-delivery-parts get /datasets/v3/snapshot/{snapshot_id}/parts When requesting a delivery in batches (see available delivery APIs) use this endpoint to check how many parts were created. format, compress and batch_size should exactly match what was sent to the delivery/download API calls. # null Source: https://docs.brightdata.com/api-reference/web-scraper-api/management-apis/get-snapshot-input get /datasets/v3/snapshot/{snapshot_id}/input Get the inputs that were used to trigger the collection # null Source: https://docs.brightdata.com/api-reference/web-scraper-api/management-apis/get-snapshots get /datasets/v3/snapshots Get a list of triggered collections, the list contains only snapshots created for a specific dataset # null Source: https://docs.brightdata.com/api-reference/web-scraper-api/management-apis/monitor-delivery get /datasets/v3/delivery/{delivery_id} The call returns the delivery status # null Source: https://docs.brightdata.com/api-reference/web-scraper-api/management-apis/monitor-progress get /datasets/v3/progress/{snapshot_id} Get the status of a specific collection # Snapshot Data Source: https://docs.brightdata.com/api-reference/web-scraper-api/management-apis/snapshot-data get /datasets/v3/log{snapshot_id} Retrieve the logs for a specific dataset snapshot # Facebook API Scrapers Source: https://docs.brightdata.com/api-reference/web-scraper-api/social-media-apis/facebook ## Overview The Facebook API Suite offers multiple types of APIs, each designed for specific data collection needs from Facebook. Below is an overview of how these APIs connect and interact, based on the available features: This API allows users to collect multiple posts based on a single input URL (such as a Facebook page, group, or profile URL).
*   **Discovery functionality**: *   Requires the direct URL of the Facebook page, group, or profile.
*   **Interesting Columns**: *   `post_url`, `content`, `hashtags`, `likes`
This API allows users to collect multiple comments from a post using its URL.
*   **Discovery functionality**: *   `N/A`
*   **Interesting Columns**: *   `commenter_url`, `comment_text`, `num_likes`, `num_replies`
There is no discovery capability across Facebook datasets since the data is locked behind login requirements. The suite of APIs is designed to offer flexibility for targeted data collection, where users can input specific URLs to gather detailed post and comment data, either in bulk or with precise filtering options. ## Posts API Posts API allows users to collect detailed post data from Facebook profiles, groups, and pages. The API provides comprehensive data points about posts, including post details, page/profile details, and attachments and media. ### Collect by Profile URL This API enables users to collect detailed data from Facebook posts by providing a profile URL. **Input Parameters** The Facebook profile URL. The number of recent posts to collect. If omitted, there is no limit. Array of post IDs to exclude from the results. Start date for filtering posts in MM-DD-YYYY format (should be earlier than end\_date). End date for filtering posts in MM-DD-YYYY format (should be later than start\_date). **Output Structure** Includes comprehensive data points: * **Post Details**: `post_id`, `content`, `hashtags`, `date_posted`, `num_comments`, `num_likes`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lkaxegm826bjpoo9m5?id=all\&tab=overview). * **Page/Profile Details**: `page_name`, `page_category`, `page_followers`, `profile_handle`. > We provide a limited set of data points about the profile. * **Attachments and Media**: `attachments` and `post_image` (link only, not the file itself), `video_view_count`. ### Collect by Group URL This API enables users to collect detailed posts from Facebook groups by providing a group URL. **Input Parameters** The Facebook group URL. The number of posts to collect. If omitted, there is no limit. Array of post IDs to exclude from the results. Start date for filtering posts in MM-DD-YYYY format (should be earlier than end\_date). End date for filtering posts in MM-DD-YYYY format (should be later than start\_date). **Output Structure** Includes comprehensive data points: * **Post Details**: `post_id`, `user_url`, `user_username`, `content`, `date_posted`, `hashtags`, `num_comments`, `num_shares`, `num_likes_type`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lz11l67o2cb3r0lkj3?id=all\&tab=overview). * **Group Details**: `group_name`, `group_id`, `group_url`, `group_intro`, `group_category`, and more. * **User Details**: `user_is_verified`, `profile_handle` and more. * **Attachments and External Links**: `Attachments` (link only, not the file itself), `original_post_url`, `other_posts_url`, `post_external_link` and more. ### Collect by Post URL This API enables users to collect detailed data from specific Facebook posts using post URLs. **Input Parameters** The Facebook profile URL. **Output Structure** Includes comprehensive data points: * **Post Details**: `post_id`, `content`, `hashtags`, `date_posted`, `num_comments`, `num_likes`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lyclm1571iy3mv57zw?id=all\&tab=overview). * **Page/Profile Details**: `page_name`, `page_category`, `page_followers`, `profile_handle`. > We provide a limited set of data points about the profile. * **Attachments and Media**: `attachments` and `post_image` (link only, not the file itself), `video_view_count`. ## Comments API Comments API allows users to collect detailed comment data from Facebook posts. The API provides comprehensive data points about comments, including comment details, user details, post metadata, and attachments and media. ### Collect by Post URL This API enables users to collect detailed comment data from Facebook posts by providing a post URL. **Input Parameters** The Facebook post URL. The number of comments to collect. If omitted, there is no limit. Array of comment IDs to exclude. Start date for filtering comments in MM-DD-YYYY format (should be earlier than end\_date). End date for filtering comments in MM-DD-YYYY format (should be later than start\_date). **Output Structure** Includes comprehensive data points: * **Comment Details**: `comment_id`, `comment_text`, `num_likes`, replies, `num_replies`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lkay758p1eanlolqw8?id=all\&tab=overview). * **User Details**: `user_name`, `user_id`, `user_url` > We provide a limited set of data points about the profile. * **Post Metadata**: `post_id`, `post_url`. * **Attachments and Media**: `attached_files` (link only, not the file itself) , `video_length`. ## Reels API Reels API allows users to collect detailed data about Facebook reels from public profiles. The API provides comprehensive data points about reels, including post details, page/profile details, and attachments and media. ### Collect by Profile URL This API allows users to collect detailed data about Facebook reels from public profiles by providing the profile URL. **Input Parameters** The Facebook profile URL. Number of reels to collect (default: up to 1600). Array of reel IDs to exclude. Start of the date range for filtering reels. End of the date range for filtering reels. **Output Structure** Includes comprehensive data points: * **Reel Details**: `post_id`, `content`, `hashtags`, `date_posted`, `num_comments`, `num_likes`, `audio` (soundtrack details), `video_view_count` and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lyclm3ey2q6rww027t?id=all\&tab=overview). * **Page/Profile Details**: `page_name`, `page_category`, `page_followers`, `profile_handle`. > We provide a limited set of data points about the profile. * **Attachments and Media**: `external_link` (link only, not the file itself) # Instagram API Scrapers Source: https://docs.brightdata.com/api-reference/web-scraper-api/social-media-apis/instagram ## Overview The Instagram API Suite offers multiple types of APIs, each designed for specific data collection needs from Instagram. Below is an overview of how these APIs connect and interact, based on the available features: This API allows users to collect profile details based on a single input: profile URL.
*   **Discovery functionality**: *   `N/A`
*   **Interesting Columns**: *   `followers`, `post_count`, `post_hashtags`, `profile_name`.
This API allows users to collect multiple posts based on a single input URL (such as an Instagram reels URL, search URL or profile URL).
*   **Discovery functionality**: *   - Direct URL of the Instagram reel *   - Direct URL of the search *   - Direct URL of the profile
*   **Interesting Columns**: *   `url`, `followers`, `hashtags`, `engagement_score_view`.
This API allows users to collect multiple comments from a post using its URL.
*   **Discovery functionality**: *   `N/A`
*   **Interesting Columns**: *   `comment_user`, `comment`, `likes_number`, `replies_number`.
The suite of APIs is designed to offer flexibility for targeted data collection, where users can input specific URLs to gather detailed post and comment data, either in bulk or with precise filtering options. ## Profiles API ### Collect by URL This API allows users to collect detailed data about an Instagram profile by providing the profile URL. It provides a comprehensive overview of an Instagram profile, including business and engagement information, posts, and user details. **Input Parameters** The Instagram profile URL. **Output Structure** Includes comprehensive data points: * **Page/Profile Details**: `account`, `id`, `followers`, `posts_count`, `is_business_account`, `is_professional_account`, `is_verified`, `avg_engagement`, `profile_name`, `profile_url`, `profile_image_link`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_l1vikfch901nx3by4?id=all\&tab=overview). ## Posts API ### Collect by URL This API enables users to collect detailed data from Instagram posts by providing a post URL. **Input Parameters** The Instagram post URL. **Output Structure** Includes comprehensive data points: * **Post Details**: `post_id`, `description`, `hashtags`, `date_posted`, `num_comments`, `likes`, `content_type`, `video_view_count`, `video_play_count`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk5ns7kz21pck8jpis?id=all\&tab=overview). * **Page/Profile Details**: `user_posted`, `followers`, `posts_count`, `profile_image_link`, `is_verified`, `profile_url`. > We provide a limited set of data points about the profile. * **Attachments and Media**: `photos`, `videos`, `thumbnail`, `display_url` (link only, not the file itself), audio. ### Discover by URL This API allows users to discover recent Instagram posts from a public profile by providing the profile URL and specifying additional parameters. **Input Parameters** The Instagram profile URL. The number of recent posts to collect. If omitted, there is no limit. Array of post IDs to exclude from the results. Start date for filtering posts in MM-DD-YYYY format (should be earlier than end\_date). End date for filtering posts in MM-DD-YYYY format (should be later than start\_date). Specify the type of posts to collect (e.g., post, reel). **Output Structure** Includes comprehensive data points: * Post Details: `post_id`, `description`, `hashtags`, `date_posted`, `num_comments`, `likes`, `video_view_count`, `video_play_count`,and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk5ns7kz21pck8jpis/url?id=all\&tab=overview). * Page/Profile Details: `user_posted`, `followers`, `posts_count`, `profile_image_link`, `is_verified`, `profile_url`, `is_paid_partnership`, `partnership_details`, `user_posted_id` * Attachments and Media: `photos`, `videos`, `thumbnail`, `audio`, `display_url`, `content_type`, `product_type`, `coauthor_producers`, `tagged_users`. This API is designed to allow for filtering, exclusion of specific posts, and collecting posts by type (regular post or reel) within a defined time frame. It provides detailed post and profile information, making it ideal for data collection and analytics. ## Comments API ### Collect by URL This API allows users to collect the latest comments from a specific Instagram post by providing the post URL. This API retrieves the most recent 10 comments along with associated metadata. **Input Parameters** The Instagram post URL. **Output Structure** Includes comprehensive data points: * **Comment Details**: `comment_id`, `comment_user`, `comment_user_url`, `comment_date`, `comment`, `likes_number`, `replies_number`, `replies`, `hashtag_comment`, `tagged_users_in_comment`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_ltppn085pokosxh13?id=all\&tab=overview). * **User Details**: `user_name`, `user_id`, `user_url` > We provide a limited set of data points about the profile. * **Post Metadata**: `post_url`, `post_user`, `post_id`. ## Reels API ### Collect by URL This API allows users to collect detailed data about Instagram reels from public profiles by providing the reel URL. **Input Parameters** The Instagram reel URL. **Output Structure** Includes comprehensive data points: * **Reel Details**: `post_id`, `description`, `hashtags`, `date_posted`, `tagged_users`, `num_comments`, `likes`, `views`, `video_play_count`, `length`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lyclm20il4r5helnj?id=all\&tab=overview). * **Page/Profile Details**: `user_posted`, `followers`, `posts_count`, `profile_image_link`, `is_verified`, `profile_url`. > We provide a limited set of data points about the profile. * **Attachments and Media**: `video_url`, `thumbnail`, `audio_url`. ### Discover by URL This API allows users to discover Instagram Reels videos from a profile URL or direct search URL. **Input Parameters** The Instagram profile or direct search URL. The number of recent reels to collect. If omitted, there is no limit. Array of post IDs to exclude from the results. Start date for filtering reels in MM-DD-YYYY format. End date for filtering reels in MM-DD-YYYY format (should be later than `start_date`). **Output Structure** Includes comprehensive data points: * **Reel Details**: `post_id`, `description`, `hashtags`, `date_posted`, `num_comments`, `likes`, `views`, `video_play_count`, `top_comments`, `length`, `video_url`, `audio_url`, `content_id`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lyclm20il4r5helnj/url?id=all\&tab=overview). * **Profile Details**: `user_posted`, `followers`, `posts_count`, `following`. * **Attachments and Media**: `video_url`, `thumbnail`, `audio_url` (link only, not the file itself). This API provides detailed information about Instagram Reels, with filtering options by date range, exclusion of specific posts, and a limit on the number of reels collected. # LinkedIn API Scrapers Source: https://docs.brightdata.com/api-reference/web-scraper-api/social-media-apis/linkedin ## Overview [The LinkedIn API](https://brightdata.com/products/web-scraper/linkedin) Suite offers multiple types of APIs, each designed for specific data collection needs from LinkedIn. Below is an overview of how these APIs connect and interact, based on the available features: The [LinkedIn Profiles API](https://brightdata.com/products/web-scraper/linkedin/profiles) allows users to collect profile details based on a single input: profile URL.
*   **Discovery functionality**: *   Discover LinkedIn profiles by names
*   **Interesting Columns**: *   `name`, `country_code`, `current_company`, `about`.
The [LinkedIn Posts API](https://brightdata.com/products/web-scraper/linkedin/post) allows users to collect multiple posts based on a single input URL.
*   **Discovery functionality**: *   - Direct URL of the LinkedIn profile *   - Direct URL user’s articles *   - Direct URL of the company
*   **Interesting Columns**: *   `url`, `title`, `hashtags`, `num_likes`.
The [LinkedIn Jobs API](https://brightdata.com/products/web-scraper/linkedin/jobs) allows users to collect job details based on a single input URL.
*   **Discovery functionality**: *   - Direct URL of the search *   - Discover by keywords
*   **Interesting Columns**: *   `job_title`, `company_name`, `job_location`, `job_seniority_level`.
The [LinkedIn Companies API](https://brightdata.com/products/web-scraper/linkedin/company) allows users to collect company information by using its URL.
*   **Discovery functionality**: N/A
*   **Interesting Columns**: *   `name`, `country_code`, `followers`, `employees_in_linkedin`.
The suite of APIs is designed to offer flexibility for targeted data collection, where users can input specific URLs to gather detailed post, profile, job, and company data, either in bulk or with precise filtering options. ## Profiles API ### Collect by URL This API allows users to collect detailed information about a LinkedIn profile by providing the profile URL. **Input Parameters**: The LinkedIn profile URL. **Output Structure**:\ Includes comprehensive data points: * **Profile Details**: `linkedin_id`, `name`, `country_code`, `city`, `position`, `current_company`, `current_company_name`, `current_company_company_id`, `about`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_l1viktl72bvl7bjuj0?id=all\&tab=overview). * **Professional Information**: `experience`, `education`, `educations_details`, `certifications`, `languages`, `recommendations_count`, `recommendations`, `volunteer_experience`, `courses`, `publications`, `patents`, `projects`, `organizations`, and `honors_and_awards`. * **Social Activity**: `posts`, `activity`, `followers`, `connections`, `people_also_viewed`. * **Media and Identification**: `avatar`, `banner_image`, `url`, `input_url`, `linkedin_num_id`, `id`, `timestamp`. This API provides a rich set of data points for analyzing LinkedIn profiles, including professional experience, education, recommendations, and social activity, as well as media and unique identification details. ### Discover by Name This API allows users to discover LinkedIn profiles based on a given first and last name. **Input Parameters**: The first name of the individual. The last name of the individual. **Output Structure**:\ Includes comprehensive data points: * **Profile Details**: `linkedin_id`, `name`, `country_code`, `city`, `position`, `current_company`, `current_company_name`, `current_company_company_id`, `about`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_l1viktl72bvl7bjuj0/name?id=all\&tab=overview). * **Professional Information**: `experience`, `education`, `educations_details`, `certifications`, `languages`, `recommendations_count`, `recommendations`, `volunteer_experience`, `courses`, `publications`, `patents`, `projects`, `organizations`, and `honors_and_awards`. * **Social Activity**: `posts`, `activity`, `followers`, `connections`, `people_also_viewed`. * **Media and Identification**: `avatar`, `banner_image`, `url`, `input_url`, `linkedin_num_id`, `id`, `timestamp`. This API provides an efficient way to retrieve LinkedIn profile details based on a person's name, delivering rich professional and social insights along with associated media assets. ## Posts API ### Collect by URL This API enables users to collect detailed data from Instagram posts by providing a post URL. **Input Parameters**: The LinkedIn post URL. **Output Structure**:\ Includes comprehensive data points: * **Post Details**: `id`, `title`, `headline`, `post_text`, `post_text_html`, `date_posted`, `hashtags`, `embedded_links`, `images`, `videos`, `post_type`, `account_type`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lyy3tktm25m4avu764?id=all\&tab=overview). * **Engagement Metrics**: `num_likes`, `num_comments`, `top_visible_comments`. * **User Details**: `user_id`, `use_url`, `user_followers`, `user_posts`, `user_articles`. * **Related Content**: `more_articles_by_user`, `more_relevant_posts`. This API provides detailed information about a LinkedIn post, including text, media attachments, engagement metrics, and additional user-related and related content for comprehensive post analysis. ### Discover by URL This API allows users to discover articles authored by a LinkedIn user by providing their profile URL and optionally limiting the number of results. **Input Parameters**: The LinkedIn profile URL. The maximum number of articles to retrieve. If omitted, all available articles will be collected. **Output Structure**:\ Includes comprehensive data points: * **Article Details**: `id`, `title`, `headline`, `post_text`, `post_text_html`, `date_posted`, `hashtags`, `embedded_links`, `images`, `videos`, `post_type`, `account_type`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lyy3tktm25m4avu764/url?id=all\&tab=overview). * **Engagement Metrics**: `num_likes`, `num_comments`, `top_visible_comments`. * **User Details**: `user_id`, `use_url`, `user_followers`, `user_posts`, `user_articles`. * **Related Content**: `more_articles_by_user`, `more_relevant_posts`. This API delivers detailed information about articles written by a LinkedIn user, including text, media, and engagement insights, along with associated user and related content data for enhanced analysis. ### Discover by Profile URL This API allows users to discover LinkedIn posts from a profile by providing the profile URL and applying optional date filters. **Input Parameters**: The LinkedIn profile URL. Start date for filtering posts by their publish date (YYYY-MM-DD format). Posts published before this date will be excluded. End date for filtering posts by their publish date (YYYY-MM-DD format). Posts published after this date will be excluded. **Output Structure**:\ Includes comprehensive data points: * **Post Details**: `id`, `title`, `headline`, `post_text`, `post_text_html`, `date_posted`, `hashtags`, `embedded_links`, `images`, `videos`, `post_type`, `account_type`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lyy3tktm25m4avu764/company_url?id=all\&tab=overview). * **Engagement Metrics**: `num_likes`, `num_comments`, `top_visible_comments`. * **User Details**: `user_id`, `use_url`, `user_followers`, `user_posts`, `user_articles`. * **Related Content**: `more_articles_by_user`, `more_relevant_posts`. This API enables precise discovery of LinkedIn posts based on specific profiles and time ranges, providing detailed post data, engagement metrics, and associated user information for in-depth analysis. ### Discover by Company URL This API allows users to discover recent LinkedIn posts published by a company, filtered by optional date ranges, using the company URL. **Input Parameters**: The LinkedIn company profile URL. Start date for filtering posts by their publish date (YYYY-MM-DD format). Only posts published on or after this date will be collected. End date for filtering posts by their publish date (YYYY-MM-DD format). Only posts published on or before this date will be collected. **Output Structure**:\ Includes comprehensive data points: * **Post Details**: `post_id`, `title`, `post_text`, `date_posted`, `hashtags`, `embedded_links`, `images`, `videos`, `post_type`, `account_type`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lyy3tktm25m4avu764/company_url?id=all\&tab=overview). * **Engagement Metrics**: `num_likes`, `num_comments`, `top_visible_comments`. * **Company Details**: `company_id`, `company_name`, `company_url`, `followers`. * **Related Content**: `more_relevant_posts`, `additional_media_links`. This API provides detailed insights into company posts, allowing businesses and analysts to track updates, engagement, and content trends effectively. ## Job Listings Information API ### Collect by URL This API allows users to collect detailed information about a LinkedIn job listing by providing the job URL. **Input Parameters**: The LinkedIn job listing URL. **Output Structure**:\ Includes comprehensive data points: * **Job Details**: `job_posting_id`, `job_title`, `job_summary`, `job_seniority_level`, `job_function`, `job_employment_type`, `job_industries`, `job_base_pay_range`, `base_salary`, `job_description_formatted`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lpfll7v5hcqtkxl6l?id=all\&tab=overview). * **Company Details**: `company_name`, `company_id`, `company_url`, `company_logo`. * **Job Metadata**: `job_location`, `job_posted_time`, `job_posted_date`, `job_num_applicants`, `discovery_input`, `apply_link`, `country_code`, `title_id`. * **Job Poster Information**: `job_poster`. This API provides a comprehensive overview of LinkedIn job listings, including job details, company information, applicant statistics, and metadata for enhanced job analysis and application tracking. ### Discover by Keyword This API allows users to discover LinkedIn job postings based on keyword search, location, and other filters like job type, experience level, and more. **Input Parameters**: The specific location where the jobs are located. A keyword or job title for searching jobs (e.g., "Product Manager"). Use quotation marks for exact matches. The country code in 2-letter format (e.g., `US`, `FR`). The time range of the job postings (e.g., `past_week`, `past_month`). The type of job (e.g., "full-time", "part-time"). The required experience level for the job (e.g., "entry-level", "mid-senior"). Whether the job is remote (e.g., "yes" or "no"). The company name to filter jobs by. When set to true, excludes job titles that do not contain the specified keywords. **Output Structure**:\ Includes comprehensive data points: * **Job Details**: `job_posting_id`, `job_title`, `job_summary`, `job_seniority_level`, `job_function`, `job_employment_type`, `job_industries`, `job_base_pay_range`, `job_description_formatted`, `base_salary`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lpfll7v5hcqtkxl6l/keyword?id=all\&tab=overview). * **Company Details**: `company_name`, `company_id`, `company_url`, `company_logo`. * **Job Metadata**: `job_location`, `job_posted_time`, `job_posted_date`, `job_num_applicants`, `job_poster`, `apply_link`, `country_code`, `title_id`. * **Application Details**: `job_poster`, `application_availability`, `discovery_input`. This API allows users to discover and filter jobs on LinkedIn based on specific keywords, location, job type, and other criteria, offering detailed job listings and company information for further analysis. ### Discover by URL This API allows users to discover job listings from a direct LinkedIn job search URL, providing job details, company information, and job metadata based on the specific URL query. **Input Parameters**: A direct LinkedIn search URL for jobs (e.g., by company or job title). When set to `true`, the search will exclude job titles that do not match the specified keywords or criteria. **Output Structure**:\ Includes comprehensive data points: * **Job Details**: `job_posting_id`, `job_title`, `job_summary`, `job_seniority_level`, `job_function`, `job_employment_type`, `job_industries`, `job_base_pay_range`, `job_description_formatted`, `base_salary`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lpfll7v5hcqtkxl6l/url?id=all\&tab=overview). * **Company Details**: `company_name`, `company_id`, `company_url`, `company_logo`. * **Job Metadata**: `job_location`, `job_posted_time`, `job_posted_date`, `job_num_applicants`, `job_poster`, `apply_link`, `country_code`, `title_id`. * **Application Details**: `job_poster`, `application_availability`, `discovery_input`. This API allows users to retrieve detailed job listings from a LinkedIn search URL, offering insights into job opportunities, company details, and more, enabling efficient job discovery and application tracking. ## Companies Information API ### Collect by URL This API allows users to collect detailed information about a LinkedIn company profile by providing the company URL. **Input Parameters**: The LinkedIn company profile URL. **Output Structure**:\ Includes comprehensive data points: * **Company Details**: `id`, `name`, `about`, `slogan`, `description`, `specialties`, `organization_type`, `company_size`, `industries`, `founded`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_l1vikfnt1wgvvqz95w?id=all\&tab=overview). * **Business Information**: `country_code`, `country_codes_array`, `locations`, `formatted_locations`, `headquarters`, `stock_info`, `funding`, `investors`, `crunchbase_url`, `get_directions_url`. * **Engagement and Network**: `followers`, `employees`, `employees_in_linkedin`, `alumni`, `alumni_information`, `affiliated`, `similar`. * **Media and Metadata**: `logo`, `image`, `url`, `updates`, `timestamp`. This API provides a comprehensive overview of LinkedIn company profiles, including business details, locations, alumni network, funding information, and media assets. # Social Media Scrapers APIs Overview Source: https://docs.brightdata.com/api-reference/web-scraper-api/social-media-apis/overview The Social Media API Suite provides extensive tools for data collection across multiple platforms. The Social Media API Suite is designed to simplify data collection while providing deep insights into user engagement and content reach across multiple platforms. Users can refer to the detailed guide provided to understand more about leveraging the APIs together for comprehensive results. ## Supported Platforms The Social Media API Suite supports a wide range of social media platforms, including: This unified suite is designed to provide insights into user profiles, posts, and interactions such as comments and likes. Below is an overview based on the sales guide for all social media APIs. ## API Types and Data Connections * **Profiles API**: Collects data on user or channel profiles, including metrics such as followers, subscribers, and content statistics. * **Posts API**: Provides data on individual posts or videos, detailing engagement metrics like likes, views, and hashtags. * **Comments API**: Offers insights into user interactions with posts, such as replies and likes on comments. Each platform, including Facebook, YouTube, Instagram, TikTok, and Pinterest, supports these APIs, enabling a consistent and comprehensive approach to social data collection. ## Key Features: 1. **Comprehensive Coverage**: The APIs cover a broad range of social platforms, from video content sites like YouTube and TikTok to social networks like Instagram and Facebook, providing versatile data collection solutions. 2. **Fresh Data**: Each API offers on-demand data collection, ensuring the data provided is always up-to-date for the user. 3. **Customizable Solutions**: The suite of APIs is flexible, enabling users to customize data collection and delivery to meet specific needs. Users can combine different APIs to gain broader insights tailored to their use cases. 4. **Ease of Integration**: Simple API calls are supported for seamless integration into existing workflows, enhancing usability and reducing the learning curve for developers. 5. **Interconnectivity**: These APIs are interconnected, meaning data from one API can be used as an input for another, allowing users to perform cross-platform analysis and derive comprehensive insights. > **For example**, on YouTube: > > * Use the **Profiles API** to gather data on a specific profile. > * Utilize the **Posts API** to fetch data on videos linked to that profile. > * Leverage the **Comments API** to obtain data on comments related to those posts. This interconnected approach applies to all supported platforms, giving users the ability to conduct comprehensive analyses that span profiles, posts, and their associated interactions. # Pinterest API Scrapers Source: https://docs.brightdata.com/api-reference/web-scraper-api/social-media-apis/pinterest ## Overview The Pinterest API Suite offers multiple types of APIs, each designed for specific data collection needs from Pinterest. Below is an overview of how these APIs connect and interact, based on the available features: This API allows users to collect profile details based on a single input: profile URL.
*   **Discovery functionality**: *   Discover by Keywords.
*   **Interesting Columns**: *   `name`, `following_count`, `website`, `follower_count`.
This API allows users to collect multiple posts based on a single input.
*   **Discovery functionality**: *   - Discover by profile URL. *   - Discover by Keywords.
*   **Interesting Columns**: *   `title`, `content`, `user_name`, `likes`.
## Profiles API ### Collect by URL This API allows users to retrieve detailed Pinterest profile information using the provided profile URL. **Input Parameters**: The Pinterest profile URL. **Output Structure**:\ Includes comprehensive data points: * **Profile Details**:\ `url`, `profile_picture`, `name`, `nickname`, `website`, `bio`, `country_code`, `profile_id`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk0zv93c2m9qdph46z?id=all\&tab=overview). * **Engagement & Metrics**:\ `following_count`, `follower_count`, `boards_num`, `saved`. * **Additional Information**:\ `last_updated`, `posts_page_url`, `discovery_input`. This API allows users to collect detailed insights into a Pinterest profile, including user statistics, engagement metrics, and profile information. ### Discover by Keywords This API allows users to discover Pinterest profiles based on a specified keyword. **Input Parameters**: The keyword to search for profiles. **Output Structure**:\ Includes comprehensive data points: * **Profile Details**:\ `url`, `profile_picture`, `name`, `nickname`, `website`, `bio`, `country_code`, `profile_id`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk0zv93c2m9qdph46z/keyword?id=all\&tab=overview). * **Engagement & Metric**s:\ `following_count`, `follower_count`, `boards_num`, saved. * **Additional Information**:\ `last_updated`, `posts_page_url`, `discovery_input`. This API enables users to find Pinterest profiles related to a specific keyword, offering insights into user statistics, engagement, and profile details. ## Posts API ### Collect by URL This API allows users to collect detailed post data from a specific Pinterest post using the provided post URL. **Input Parameters**: The Pinterest post URL. **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `url`, `post_id`, `title`, `content`, `date_posted`, `post_type`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk0sjs4d21kdr7cnlv?id=all\&tab=overview). * **User Details**:\ `user_name`, `user_url`, `user_id`, `followers`. * **Post Metrics**:\ `likes`, `comments_num`, `comments`, `categories`. * **Media & Attachments**:\ `image_video_url`, `video_length`, `attached_files`. * **Hashtags & Discovery**:\ `hashtags`, `discovery_input`. This API allows users to retrieve detailed insights into a specific Pinterest post, including user engagement, post content, media, and other related information. ### Discover by Profile URL This API allows users to retrieve posts from a specific Pinterest profile based on the provided profile URL. **Input Parameters**: The Pinterest profile URL from which to collect posts. The number of posts to collect. If omitted, there is no limit. An array of post IDs to exclude from the results. Start date for filtering posts in `MM-DD-YYYY` format (should be earlier than `end_date`). End date for filtering posts in `MM-DD-YYYY` format (should be later than `start_date`). **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `url`, `post_id`, `title`, `content`, `date_posted`, `user_name`, `user_url`, `user_id`, `followers`, `likes`, `categories`, `source`, `attached_files`, `image_video_url`, `video_length`, `hashtags`, `comments_num`, `comments`, `post_type`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk0sjs4d21kdr7cnlv/profile_url?tab=overview). * **Engagement & Metrics**:\ `followers`, `likes`, `comments_num`. * **Media & Attachments**:\ `image_video_url`, `video_length`, `attached_files`. * **Additional Information**:\ `discovery_input`. This API enables users tso collect posts from a specific Pinterest profile, allowing for filtering by date, exclusion of specific posts, and retrieval of detailed post data including media, comments, and engagement metrics. ### Discover by Keywords This API allows users to discover Pinterest posts based on a specific keyword, enabling efficient content discovery. **Input Parameters**: The keyword to search for posts, such as "food" or any other relevant term. **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `url`, `post_id`, `title`, `content`, `date_posted`, `user_name`, `user_url`, `user_id`, `followers`, `likes`, `categories`, `source`, `attached_files`, `image_video_url`, `video_length`, `hashtags`, `comments_num`, `comments`, `post_type`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk0zv93c2m9qdph46z/keyword?id=all\&tab=overview). * **Engagement & Metrics**:\ `followers`, `likes`, `comments_num`. * **Media & Attachments**:\ `image_video_url`, `video_length`, `attached_files`. * **Additional Information**:\ `discovery_input`. This API enables users to search for Pinterest posts based on a specific keyword, providing detailed insights into the content, engagement, media, and associated metrics for efficient discovery. # Quora API Scrapers Source: https://docs.brightdata.com/api-reference/web-scraper-api/social-media-apis/quora ## Overview This API allows users to collect multiple posts based on a single input URL.
*   **Discovery functionality**: *   N/A
*   **Interesting Columns**: *   `author_name`, `title`, `over_all_answers`, `shares`.
## Posts API ### Collect by URL This API allows users to collect detailed information about a specific Quora post using the provided post URL. **Input Parameters** The URL of the Quora post. **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `url`, `post_id`, `title`, `post_date`, `originally_answered`, `over_all_answers`, `post_text`, `header`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lvz1rbj81afv3m6n5y?id=all\&tab=overview). * **Media & Links**:\ `pictures_urls`, `videos_urls`, `external_urls`. * **Engagement & Metrics**:\ `upvotes`, `shares`, `views`, `author_content_views`. * **Author Details**:\ `author_name`, `author_active_spaces`, `author_joined_date`, `author_about`, `author_education`. * **Top Comments**:\ `top_comments`. This API provides detailed insights into a Quora post, including content, media, author information, and engagement metrics, enabling efficient post analysis and content tracking. # Reddit API Scrapers Source: https://docs.brightdata.com/api-reference/web-scraper-api/social-media-apis/reddit ## Overview The Reddit API Suite offers multiple types of APIs, each designed for specific data collection needs from Reddit. Below is an overview of how these APIs connect and interact, based on the available features: This API allows users to collect multiple posts based on a single input URL.
*   **Discovery functionality**: *   Discover by subreddit URL. *   Discover by Keywords.
*   **Interesting Columns**: *   `url`, `title`, `num_comments`, `description`
This API allows users to collect multiple comments from a post using its URL.
*   **Discovery functionality**: *   N/A
*   **Interesting Columns**: *   `url`, `comment`, `community_name`, `num_replies`
## Posts API ### Collect by URL This API allows users to retrieve detailed post data from a specific Reddit post using the provided URL. **Input Parameters** The URL of the Reddit post to be retrieved. **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `post_id`, `url`, `title`, `description`, `num_comments`, `date_posted`, `tag`, `related_posts`, `comments`, `photos`, `videos`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lvz8ah06191smkebj4?id=all\&tab=overview). * **User Details**:\ `user_posted`. * **Community Details**:\ `community_name`, `community_url`, `community_description`, `community_members_num`, `community_rank`. * **Engagement Metrics**:\ `num_upvotes`. This API enables efficient retrieval of Reddit post information, including user engagement metrics, community details, and multimedia content, supporting advanced analysis and content tracking. ### Discover by Keywords This API allows users to discover Reddit posts by specific keywords, with options to filter by date and the number of posts to retrieve. **Input Parameters** The search term to find relevant posts. The date to filter posts by, ensuring results fall within the specified timeframe. The number of posts to collect. If not provided, all matching posts are retrieved. **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `post_id`, `url`, `user_posted`, `title`, `description`, `num_comments`, `date_posted`, `tag`, `related_posts`, `comments`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lvz8ah06191smkebj4/keyword?id=all\&tab=overview). * **Community Details**:\ `community_name`, `community_url`, `community_description`, `community_members_num`, `community_rank`. * **Media Details**:\ `photos`, `videos`. This API enables users to efficiently discover and analyze Reddit posts related to specific keywords, providing valuable insights into community discussions, post popularity, and associated media content. ### Discover by subreddit URL This API allows users to retrieve posts from a specific subreddit using the subreddit URL, with an option to sort results based on specific criteria. **Input Parameters** The URL of the subreddit to collect posts from. Determines the sorting method for the posts (e.g., `new`, `top`, `hot`). **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `post_id`, `url`, `user_posted`, `title`, `description`, `num_comments`, `date_posted`, `tag`, `related_posts`, `comments`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lvz8ah06191smkebj4/subreddit_url?id=all). * **Community Details**:\ `community_name`, `community_url`, `community_description`, `community_members_num`, `community_rank`. * **Media Details**:\ `photos`, `videos`. This API provides a streamlined way to explore and analyze posts within a subreddit, offering insights into post content, community dynamics, and associated media files. Perfect for monitoring subreddit activity or extracting post-level data for further analysis. ## Comments API ### Collect by URL This API allows users to collect detailed comment data from a specific Reddit post or comment URL, with the option to filter comments based on the number of days since they were published. **Input Parameters** The URL of the Reddit post or comment to retrieve comments from. Collect all comments published no later than the specified number of days. **Output Structure**:\ Includes comprehensive data points: * **Comment Details**:\ `comment_id`, `user_posted`, `comment`, `date_posted`, `replies`, `num_upvotes`, `num_replies`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lvzdpsdlw09j6t702?tab=overview). * **Post Details**:\ `post_url`, `post_id`, `post_language`, `post_state`, `post_type`, `images`. * **Community Details**:\ `community_name`, `community_url`, `community_description`, `community_members_num`, `community_rank`. * **Post Attributes**:\ `is_moderator`, `is_pinned`, `has_bot_in_username`, `is_locked`, `is_admin_post`, `is_archived_post`, `is_moderator_post`, `is_quarantined_post`, `is_not_safe_for_work_post`, `is_eligible_for_content_blocking_post`, `is_promoted_post`. This API provides comprehensive insights into Reddit comments and their associated posts, enabling users to analyze engagement, user activity, and content trends effectively. # TikTok API Scrapers Source: https://docs.brightdata.com/api-reference/web-scraper-api/social-media-apis/tiktok ## Overview The TikTok API Suite offers multiple types of APIs, each designed for specific data collection needs from TikTok. Below is an overview of how these APIs connect and interact, based on the available features: This API allows users to collect profile details based on a single input: profile URL.
*   **Discovery functionality**: *   Direct URL of the search
*   **Interesting Columns**: *   `nickname`, `awg_engagement_rate`, `followers`, `likes`
This API allows users to collect multiple posts based on a single input URL.
*   **Discovery functionality**: *   - Direct URL of the TikTok profile *   - Discover by keywords *   - Direct URL of the discovery
*   **Interesting Columns**: *   `url`, `share_count`, `description`, `hashtags`
This API allows users to collect multiple comments from a post using its URL.
*   **Discovery functionality**: *   N/A
*   **Interesting Columns**: *   `url`, `comment_text`, `commenter_url`, `num_likes`
## Profile API ### Collect by URL This API allows users to retrieve detailed TikTok profile information using the provided profile URL. **Input Parameters**: The TikTok profile URL. **Output Structure**:\ Includes comprehensive data points: * **Profile Details**:\ `account_id`, `nickname`, `biography`, `bio_link`, `predicted_lang`, `is_verified`, `followers`, `following`, `likes`, `videos_count`, `create_time`, `id`, `url`, `profile_pic_url`, `profile_pic_url_hd`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_l1villgoiiidt09ci?id=all\&tab=overview). * **Engagement Metrics**:\ `awg_engagement_rate`, `comment_engagement_rate`, `like_engagement_rate`, `like_count`, `digg_count`. * **Privacy & Settings**:\ `is_private`, `relation`, `open_favorite`, `comment_setting`, `duet_setting`, `stitch_setting`, `is_ad_virtual`, `room_id`, `is_under_age_18`. * **Discovery & Top Videos**:\ `region`, `top_videos`, `discovery_input`. This API allows users to retrieve detailed TikTok profile information, including engagement metrics, privacy settings, and top videos, offering insights into user activity and profile data. ### Discover by Search URL This API allows users to discover TikTok profiles based on a specific search URL and country, providing detailed profile information. **Input Parameters**: The TikTok search URL. The country from which to perform the search. **Output Structure**:\ Includes comprehensive data points: * **Profile Details**:\ `account_id`, `nickname`, `biography`, `bio_link`, `predicted_lang`, `is_verified`, `followers`, `following`, `likes`, `videos_count`, `create_time`, `id`, `url`, `profile_pic_url`, `profile_pic_url_hd`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_l1villgoiiidt09ci/search_url?id=all\&tab=overview). * **Engagement Metrics**:\ `awg_engagement_rate`, `comment_engagement_rate`, `like_engagement_rate`, `like_count`, `digg_count`. * **Privacy & Settings**:\ `is_private`, `relation`, `open_favorite`, `comment_setting`, `duet_setting`, `stitch_setting`, `is_ad_virtual`, `room_id`, `is_under_age_18`. * **Discovery & Top Videos**:\ `region`, `top_videos`, `discovery_input`. This API enables users to discover TikTok profiles based on search criteria, offering insights into user activity, engagement, privacy settings, and top content. It helps facilitate efficient discovery and analysis of TikTok users. ## Posts API ### Collect by URL This API enables users to collect detailed data from TikTok posts by providing a post URL. **Input Parameters**: The TikTok post URL. **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `post_id`, `description`, `create_time`, `share_count`, `collect_count`, `comment_count`, `play_count`, `video_duration`, `hashtags`, `original_sound`, `official_item`, `original_item`, `shortcode`, `video_url`, `music`, `cdn_url`, `width`, `carousel_images`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lu702nij2f790tmv9h?id=all\&tab=overview). * **Profile Details**:\ `profile_id`, `profile_username`, `profile_url`, `profile_avatar`, `profile_biography`, `account_id`, `profile_followers`, `is_verified`. * **Tagged Users and Media**:\ `tagged_user`, `carousel_images`. * **Additional Information:**:\ `tt_chain_token`, `secu_id` ### Discover by Profile URL This API allows users to retrieve posts from a TikTok profile based on a provided profile URL, with filtering options for the number of posts, date range, and post exclusions. **Input Parameters**: The TikTok profile URL. The number of posts to collect. If not provided, there is no limit. An array of post IDs to exclude from the collection. Start date for filtering posts (format: mm-dd-yyyy). Should be lower than `end_date`. End date for filtering posts (format: mm-dd-yyyy). Should be greater than `start_date`. Specify the type of posts to collect (e.g., "post" or "reel"). **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `post_id`, `description`, `create_time`, `share_count`, `collect_count`, `comment_count`, `play_count`, `video_duration`, `hashtags`, `original_sound`, `official_item`, `original_item`, `shortcode`, `video_url`, `music`, `cdn_url`, `width`, `carousel_images`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lu702nij2f790tmv9h/profile_url?id=all\&tab=overview). * **Profile Details**:\ `profile_id`, `profile_username`, `profile_url`, `profile_avatar`, `profile_biography`, `account_id`, `profile_followers`, `is_verified`. * **Tagged Users and Media**:\ `tagged_user`, `carousel_images`. * **Additional Information**:\ `tt_chain_token`, `secu_id`. This API allows users to discover and retrieve detailed information about posts from a specific TikTok profile, including post-specific metrics, profile details of the creator, and tagged users. It supports efficient content discovery and post analysis. ### Discover by Keywords This API allows users to search for TikTok posts based on specific keywords or hashtags, offering a powerful tool for discovering relevant content across TikTok's platform. **Input Parameters**: The keyword or hashtag to search for within TikTok posts. The number of posts to collect. If not provided, there is no limit. An array of post IDs to exclude from the collection. Specify the type of posts to collect (e.g., "post" or "reel"). **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `post_id`, `description`, `create_time`, `digg_count`, `share_count`, `collect_count`, `comment_count`, `play_count`, `video_duration`, `hashtags`, `original_sound`, `post_type`, `discovery_input`, `official_item`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lu702nij2f790tmv9h/keyword?id=all\&tab=overview). * **Profile Details**:\ `profile_id`, `profile_username`, `profile_url`, `profile_avatar`, `profile_biography`, `account_id`, `profile_followers`, `is_verified`. * **Tagged Users and Media**:\ `tagged_user`, `carousel_images`. * **Additional Information**:\ `tt_chain_token`, `secu_id`. This API allows users to discover posts on TikTok that match specific keywords or hashtags, providing insights into post details, profile information, and media. It’s a great tool for exploring trends, content, and users on TikTok. ### Discover by Discover URL This API allows users to collect detailed post data from a specific TikTok discover URL. **Input Parameters**: The TikTok discover URL from which posts will be retrieved. **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `post_id`, `description`, `create_time`, `digg_count`, `share_count`, `collect_count`, `comment_count`, `play_count`, `video_duration`, `hashtags`, `original_sound`, `post_type`, `discovery_input`, `official_item`, `original_item`, and more. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lu702nij2f790tmv9h/url?id=all\&tab=overview). * **Profile Details**:\ `profile_id`, `profile_username`, `profile_url`, `profile_avatar`, `profile_biography`, `account_id`, `profile_followers`, `is_verified`. * **Tagged Users and Media**:\ `tagged_user`, `carousel_images`. * **Additional Information**:\ `tt_chain_token`, `secu_id`. This API provides detailed insights into TikTok posts discovered via the discover URL, allowing for easy access to trending content, user profiles, and post metadata for analysis and exploration. ## Comments API ### Collect by URL This API allows users to collect detailed comment data from a specific TikTok post using the provided post URL. **Input Parameters**: The TikTok post URL. **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `post_url`, `post_id`, `post_date_created`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lkf2st302ap89utw5k?id=all\&tab=overview). * **Comment Details**:\ `date_created`, `comment_text`, `num_likes`, `num_replies`, `comment_id`, `comment_url`. * **Commenter Details**:\ `commenter_user_name`, `commenter_id`, `commenter_url`. This API provides detailed insights into TikTok post comments, including comment-specific metrics and information about the commenters, enabling effective comment analysis and interaction tracking. # Twitter API Scrapers Source: https://docs.brightdata.com/api-reference/web-scraper-api/social-media-apis/twitter ## Overview The Twitter API Suite offers multiple types of APIs, each designed for specific data collection needs from Twitter. Below is an overview of how these APIs connect and interact, based on the available features: This API allows users to collect profile details based on a single input: profile URL.
*   **Discovery functionality**: *   N/A
*   **Interesting Columns**: *   `url`, `profile_name`, `is_verified`, `followers`.
This API allows users to collect multiple posts based on a single input URL.
*   **Discovery functionality**: *   Discover by profile URL.
*   **Interesting Columns**: *   `name`, `description`, `replies`, `likes`.
## Profiles API ### Profiles by URL This API allows users to retrieve detailed Twitter profile information using the provided profile URL. **Input Parameters**: The Twitter profile URL. The number of posts to collect. If omitted, there is no limit. **Output Structure**:\ Includes comprehensive data points: * **Profile Details**:\ `x_id`, `url`, `id`, `profile_name`, `biography`, `is_verified`, `profile_image_link`, `external_link`, `date_joined`, `location`, `birth_date`, `posts_count`, `posts`, `suggested_profiles`, `is_business_account`, `is_government_account`, `category_name`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lwxmeb2u1cniijd7t4?id=all\&tab=overview). * **Engagement Metrics**:\ `followers`, `following`, `subscriptions`. * **Post Details**:\ `posts`. This API allows users to retrieve detailed Twitter profile information, including user engagement metrics, post details, and more, offering insights into user activity and profile data. ## Posts API ### Collect by URL This API allows users to retrieve detailed post information from a specific Twitter post using the provided post URL. **Input Parameters**: The Twitter post URL. API limitations: Up to 1000 posts per input. **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `id`, `user_posted`, `name`, `description`, `date_posted`, `url`, `tagged_users`, `replies`, `reposts`, `likes`, `views`, `external_url`, `hashtags`, `quotes`, `bookmarks`, `external_image_urls`, `videos`, `quoted_post`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lwxkxvnf1cynvib9co?id=all\&tab=overview). * **User Profile Details**:\ `followers`, `biography`, `posts_count`, `profile_image_link`, `following`, `is_verified`. * **Parent Post Details**:\ `parent_post_details`. This API allows users to retrieve detailed information about a specific Twitter post, including user details, engagement metrics, and media content, offering insights into post interactions and user activity. ### Discover by Profile URL This API allows users to discover and retrieve posts from a specific Twitter profile within a specified date range. **Input Parameters**: The URL of the Twitter profile. Start date for filtering posts in MM-DD-YYYY format. End date for filtering posts in MM-DD-YYYY format. **Output Structure**:\ Includes comprehensive data points: * **Post Details**:\ `id`, `user_posted`, `name`, `description`, `date_posted`, `url`, `tagged_users`, `replies`, `reposts`, `likes`, `views`, `external_url`, `hashtags`, `quotes`, `bookmarks`, `external_image_urls`, `videos`, `quoted_post`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lwxkxvnf1cynvib9co/profile_url?id=all\&tab=overview). * **User Profile Details**:\ `followers`, `biography`, `posts_count`, `profile_image_link`, `following`, `is_verified`. * **Parent Post Details**:\ `parent_post_details`. This API allows users to retrieve posts from a specific Twitter profile within a defined time frame, offering detailed insights into post interactions, user data, and media content. # Vimeo API Scrapers Source: https://docs.brightdata.com/api-reference/web-scraper-api/social-media-apis/vimeo ## Overview The Vimeo API Suite offers multiple types of APIs, each designed for specific data collection needs from Vimeo. Below is an overview of how these APIs connect and interact, based on the available features: This API allows users to collect multiple posts based on a single input URL.
*   **Discovery functionality**: *   Discover by profile URL. *   Discover by Keywords and License.
*   **Interesting Columns**: *   `title`, `video_length`, `views`, `likes`.
## Posts API ### Collect by URL This API allows users to collect detailed information about a specific Vimeo video using the provided video URL. **Input Parameters** The URL of the Vimeo video. **Output Structure**:\ Includes comprehensive data points: * **Video Details**\ `video_id`, `title`, `url`, `video_url`, `video_length`, `description`, `data_posted`, `transcript`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lxk88z3v1ketji4pn?id=all\&tab=overview). * **Engagement & Metrics**\ `views`, `likes`, `comments`, `collections`. * **Uploader Details**\ `uploader`, `uploader_url`, `uploader_id`, `avatar_img_uploader`. * **Video Media & Content**\ `preview_image`, `related_videos`, `music_track`. * **License & Quality**\ `license`, `license_info`, `video_quality`. * **Dimensions**\ `height`, `width`. This API provides detailed insights into a Vimeo video, including video content, uploader information, media links, engagement metrics, and more, enabling efficient video analysis and content tracking. ### Discover by URL This API allows users to discover Vimeo videos based on a specific URL and associated keywords, providing detailed video information and insights. **Input Parameters** The URL of the Vimeo post. The keyword to search for within the video's content. The number of pages of results to collect. **Output Structure**:\ Includes comprehensive data points: * **Video Details**\ `video_id`, `title`, `url`, `video_url`, `video_length`, `description`, `data_posted`, `transcript`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lxk88z3v1ketji4pn/url?id=all\&tab=overview). * **Engagement & Metrics**\ `views`, `likes`, `comments`, `collections`. * **Uploader Details**\ `uploader`, `uploader_url`, `uploader_id`, `avatar_img_uploader`. * **Video Media & Content**\ `preview_image`, `related_videos`, `music_track`. * **License & Quality**\ `license`, `license_info`, `video_quality`. * **Dimensions**\ `height`, `width`. This API allows users to discover Vimeo videos by URL and keyword, offering detailed insights into video content, uploader information, and engagement metrics. ### Discover by Keywords and License This API allows users to discover Vimeo videos based on specific keywords and license types, providing detailed video information and insights. **Input Parameters** The keyword to search for in the video content. The license type to filter the videos by (e.g., Creative Commons, Standard License). The number of pages of results to collect. **Output Structure**:\ Includes comprehensive data points: * **Video Details**\ `video_id`, `title`,`url`, `video_url`, `video_length`, `description`, `data_posted`, `transcript`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lxk88z3v1ketji4pn/keyword_and_license?id=all\&tab=overview). * **Engagement & Metrics**\ `views`, `likes`, `comments`, `collections`. * **Uploader Details**\ `uploader`, `uploader_url`, `uploader_id`, `avatar_img_uploader`. * **Video Media & Content**\ `preview_image`, `related_videos`, `music_track`. * **License & Quality**\ `license`, `license_info`, `video_quality`. * **Dimensions**\ `height`, `width`. This API allows users to discover Vimeo videos based on specific keywords and license types, offering detailed insights into video content, uploader information, and engagement metrics. # YouTube API Scrapers Source: https://docs.brightdata.com/api-reference/web-scraper-api/social-media-apis/youtube If you are looking to download videos from YouTube, please contact Bright Data sales at [sales@brightdata.com](mailto:sales@brightdata.com). ## Overview The Youtube API Suite offers multiple types of APIs, each designed for specific data collection needs from Youtube. Below is an overview of how these APIs connect and interact, based on the available features: This API allows users to collect profile details based on a single input: profile URL.
*   **Discovery functionality**: *   [Discover by Keywords](/api-reference/web-scraper-api/social-media-apis/youtube#discover-profiles-by-keywords)
*   **Interesting Columns**: *   `name`, `url`, `videos_count`, `views`.
This API allows users to collect multiple posts based on a single input URL.
*   **Discovery functionality**: *   Discover by keywords. *   Discover by search filters. *   Discover by hashtag. *   Discover by channel URL.
*   **Interesting Columns**: *   `url`, `title`, `likes`, `views`.
This API allows users to collect multiple comments from a post using its URL.
*   **Discovery functionality**: *   N/A
*   **Interesting Columns**: *   `comment_text`, `replies`, `likes`, `username`.
## Profiles API ### Collect by URL This API allows users to retrieve detailed YouTube channel information using the provided channel URL. **Input Parameters**: The YouTube channel URL. **Output Structure**:\ Includes comprehensive data points: * **Profile Details**:\ `url`, `handle`, `name`, `description`, `created_date`, `identifier`, `id`, `handle_md5`. * **Engagement Metrics**:\ `subscribers`, `videos_count`, `views`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk538t2k2p1k3oos71?id=all\&tab=overview). * **Media Assets**:\ `profile_image`, `banner_img`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk538t2k2p1k3oos71?id=all\&tab=overview). * **Additional Information**:\ `details`, `links`, `discovery_input`. This API provides insights into YouTube channel profiles, including engagement metrics, profile assets, and detailed channel information. ### Discover by Keywords This API allows users to discover YouTube channel profiles by searching with specific keywords related to the channel or its videos. **Input Parameters**: Keyword related to the channel or its videos. **Output Structure**:\ Includes comprehensive data points: * **Profile Details**:\ `url`, `handle`, `name`, `description`, `created_date`, `identifier`, `id`, `handle_md5`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk538t2k2p1k3oos71/keyword?id=all\&tab=overview). * **Engagement Metrics**:\ `subscribers`, `videos_count`, `views`. * **Media Assets**:\ `profile_image`, `banner_img`. * **Additional Information**:\ `details`, `links`, `discovery_input`. This API enables efficient discovery of YouTube channel profiles based on keywords, providing profile information, engagement metrics, and related details. ## Posts API ### Collect by URL This API allows users to retrieve detailed video information from YouTube using the provided video URL. **Input Parameters**: The YouTube video URL. **Output Structure**:\ Includes comprehensive data points: * **Video Details**:\ `url`, `title`, `video_url`, `video_length`, `video_id`, `post_type`, `date_posted`, `description`, `music`, `transcript`, `formatted_transcript`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk56epmy2i5g7lzu0k?id=all\&tab=overview). * **Engagement Metrics**:\ `likes`, `views`, `num_comments`, `related_videos`. * **Channel Details**:\ `youtuber`, `youtuber_id`, `handle_name`, `channel_url`, `subscribers`, `verified`, `avatar_img_channel`, `youtuber_md5`. * **Video Media & Quality**:\ `preview_image`, `viewport_frames`, `current_optimal_res`, `codecs`, `color`, `quality`, `quality_label`. * **Additional Information**:\ `discovery_input`, `shortcode`, `is_sponsored`, `license`. This API enables users to retrieve detailed insights about a YouTube video, including metadata, engagement metrics, and channel-specific details, supporting content analysis and tracking. ### Discover by Channel URL This API allows users to collect videos published by a specific YouTube channel using its URL, with optional filters like the number of posts, date range, and sorting order. **Input Parameters**: The YouTube channel URL. The number of videos to collect. If omitted, there is no limit. Start date for filtering videos (MM-DD-YYYY). End date for filtering videos (MM-DD-YYYY). Sort the results (e.g., by views, date, or relevance). **Output Structure**:\ Includes comprehensive data points: * **Video Details**:\ `url`, `title`, `video_url`, `video_id`, `video_length`, `date_posted`, `description`, `post_type`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk56epmy2i5g7lzu0k/url?id=all\&tab=overview). * **Channel Details**:\ `youtuber`, `youtuber_id`, `channel_url`, `handle_name`, `avatar_img_channel`, `subscribers`, `youtuber_md5`, `verified`. * **Engagement Metrics**:\ `likes`, `views`, `num_comments`, `is_sponsored`. * **Media and Technical Information**:\ `preview_image`, `related_videos`, `music`, `shortcode`, `viewport_frames`, `current_optimal_res`, `codecs`, `color`, `quality`, `quality_label`, `license`. * **Transcript**:\ `transcript`, `formatted_transcript`. This API allows efficient retrieval of YouTube videos from a specified channel with detailed video, channel, and engagement information, supporting filtering and sorting options for enhanced analysis. ### Discover by Keywords This API allows users to search for YouTube videos using specific keywords, with optional parameters for filtering results by date range and the number of posts to retrieve. **Input Parameters**: The keyword to search for videos. The number of videos to collect. Start date for filtering videos (MM-DD-YYYY). End date for filtering videos (MM-DD-YYYY). **Output Structure**:\ Includes comprehensive data points: * **Video Details**:\ `url`, `title`, `video_url`, `video_id`, `video_length`, `date_posted`, `description`, `post_type`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk56epmy2i5g7lzu0k/keyword?id=all\&tab=overview). * **Channel Details**:\ `youtuber`, `youtuber_id`, `channel_url`, `handle_name`, `avatar_img_channel`, `subscribers`, `youtuber_md5`, `verified`. * **Engagement Metrics**:\ `likes`, `views`, `num_comments`, `is_sponsored`. * **Media and Technical Information**:\ `preview_image`, `related_videos`, `music`, `shortcode`, `viewport_frames`, `current_optimal_res`, `codecs`, `color`, `quality`, `quality_label`, `license`. * **Transcript**:\ `transcript`, `formatted_transcript`. This API enables efficient discovery of YouTube videos using keywords, supporting advanced filtering by date and result count for targeted video analysis and retrieval. ### Discover by Search Filters This API allows users to search for YouTube videos using advanced search filters such as upload date, video type, duration, and additional features, enabling more refined and targeted video discovery. **Input Parameters**: The keyword to search for videos. Filter results by upload date (e.g., today, this week, this month). Specify video type (e.g., video, channel, playlist). Filter by video length (e.g., short, medium, long). Specify additional video features (e.g., 4K, HD, subtitles, live). **Output Structure**:\ Includes comprehensive data points: * **Video Details**:\ `url`, `title`, `video_url`, `video_id`, `video_length`, `date_posted`, `description`, `post_type`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk56epmy2i5g7lzu0k/search_filters?id=all\&tab=overview). * **Channel Details**:\ `youtuber`, `youtuber_id`, `channel_url`, `handle_name`, `avatar_img_channel`, `subscribers`, `youtuber_md5`, `verified`. * **Engagement Metrics**:\ `likes`, `views`, `num_comments`, `is_sponsored`. * **Media and Technical Information**:\ `preview_image`, `related_videos`, `music`, `shortcode`, `viewport_frames`, `current_optimal_res`, `codecs`, `color`, `quality`, `quality_label`, `license`. * **Transcript**:\ `transcript`, `formatted_transcript`. This API supports advanced search capabilities on YouTube, allowing users to filter results by multiple parameters for precise and efficient video discovery. ### Discover by Hashtag This API allows users to search for YouTube posts by a specific hashtag, enabling users to discover videos related to trending topics or specific themes. **Input Parameters**: The hashtag to search for (e.g., #fitness, #travel). The number of posts to collect. If omitted, there is no limit. An array of post IDs to exclude from the results. Start date for filtering posts in MM-DD-YYYY format (should be earlier than end\_date). End date for filtering posts in MM-DD-YYYY format (should be later than start\_date). **Output Structure**:\ Includes comprehensive data points: * **Video Details**:\ `url`, `title`, `video_url`, `video_id`, `video_length`, `date_posted`, `description`, `post_type`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk56epmy2i5g7lzu0k/hashtag?id=all\&tab=overview). * **Channel Details**:\ `youtuber`, `youtuber_id`, `channel_url`, `handle_name`, `avatar_img_channel`, `subscribers`, `youtuber_md5`, `verified`. * **Engagement Metrics**:\ `likes`, `views`, `num_comments`, `is_sponsored`. * **Media and Technical Information**:\ `preview_image`, `related_videos`, `music`, `shortcode`, `viewport_frames`, `current_optimal_res`, `codecs`, `color`, `quality`, `quality_label`, `license`. * **Transcript**:\ `transcript`, `formatted_transcript`. This API enables efficient video discovery by searching posts related to a specific hashtag, with filtering options to refine the results. ## Comments API ### Collect by URL This API allows users to retrieve detailed comment data from a specific YouTube video using the provided video URL. **Input Parameters**: The YouTube video URL. Number of times to load replies for comments. **Output Structure**:\ Includes comprehensive data points: * **Comment Details**:\ `comment_id`, `comment_text`, `likes`, `replies`, `replies_value`, `replies_without_names`. > For all data points, [click here](https://brightdata.com/cp/data_api/gd_lk9q0ew71spt1mxywf?id=all\&tab=overview). * **User Details**:\ `username`, `user_channel`, `username_md5`. * **Additional Information**:\ `date`, `url`, `video_id`. This API enables users to analyze YouTube video comments, including replies and user details, providing insights into engagement and audience interactions. # Synchronous Requests Source: https://docs.brightdata.com/api-reference/web-scraper-api/synchronous-requests dca-api POST /datasets/v3/scrape This endpoint allows users to fetch data efficiently and ensures seamless integration with their applications or workflows. ## How It Works This synchronous API endpoint allows users to send a scraping request and receive the results in real-time directly in the response, at the point of request - such as a terminal or application - without the need for external storage or manual downloads. This approach streamlines the data collection process by eliminating additional steps for retrieving results. You can specify the desired output format using the format parameter. If no format is provided, the response will default to JSON. ## Timeout Limit Please note that this synchronous request is subject to a 1 minute timeout limit. If the data retrieval process exceeds this limit, the API will return an HTTP 202 response, indicating that the request is still being processed. In such cases, you will receive a snapshot ID to monitor and retrieve the results asynchronously via the Monitor Snapshot and Download Snapshot endpoints. Example response on timeout: ```JSON 202 { "snapshot_id": "s_xxx", "message": "Your request is still in progress and cannot be retrieved in this call. Use the provided Snapshot ID to track progress via the Monitor Snapshot endpoint and download it once ready via the Download Snapshot endpoint." } ``` # Choose a delivery type on request level Source: https://docs.brightdata.com/api-reference/web-scraper-ide-api/Choose_a_delivery_type_on_request_level Instead of creating duplicate scrapers for each delivery type, you can choose a delivery type per job using API. hero-image.png Ensure that your scraper is updated to the latest version to improve success rate update-available.png type-batch.png on-completion.png Following error message will be returned from Batch API when delivery type is set to Realtime. ```json Error "error": "Cannot trigger a batch job with a real-time scraper. Use /trigger_immediate endpoint instead" ``` `dca/trigger` [Trigger a scraper for batch collection method](/api-reference/web-scraper-ide-api/Trigger_a_scraper_for_batch_collection_method) ```sh Shell curl "https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1" -H "Content-Type: application/json" -H "Authorization: Bearer API_KEY" -d '[{"url":"https://targetwebsite.com/product_id/"}]' ``` ```json Sample Response { "collection_id":"j_l3daejgw1wnpjxxxxx", "start_eta":"2022-05-19T17:28:48.056Z" } ``` `dca/trigger_immediate` [Trigger a scraper for real-time collection](/api-reference/web-scraper-ide-api/Receive_data_from_real_time_work_scraper) ```sh Shell curl "https://api.brightdata.com/dca/trigger_immediate?collector=ID_COLLECTOR" -H "Content-Type: application/json" -H "Authorization: Bearer API_KEY" -d '{"url":"https://targetwebsite.com/product_id/"}' ``` ```json Sample Response { "response_id":"z2805t1652973963340rg6252xxxxxx" } ``` Batch responses begin with `j_****` and real-time responses begin with `z****` * [Receive batch data](/api-reference/web-scraper-ide-api/Receive_batch_data) * [Receive data from real-time work scraper](/api-reference/web-scraper-ide-api/Receive_data_from_real_time_work_scraper) # Getting started with the API Source: https://docs.brightdata.com/api-reference/web-scraper-ide-api/Getting_started_wtih_the_API The Data scraper product offers a JSON API to control and inspect your data scrapers programmatically. 1. Signup (or login) to our control panel 2. Insert target URL (website to collect data from) or choose a ready-made template 3. Set up the delivery preference of scraper: Batch or Real-Time 4. Initiate by API NOT A DEVELOPER? Contact our support for easy onboarding. The data scraper is divided into two crawling systems: This is an efficient way of handling large amounts of data. Data is collected in batches over a period of time. After the data is collected, entered, and processed, the batch results are generated. You can deliver the results all at once or stream them to your system as they're collected. This is an ideal way to get a fast response for one request. This is useful in environments where you have a limited amount of time to crawl a page and get the data. The Data scraper API has four general workflows: 1. [Batch trigger + Push delivery](/api-reference/web-scraper-ide-api/Trigger_a_scraper_for_batch_collection_method) 2. [Batch trigger + API polling](/api-reference/web-scraper-ide-api/Receive_batch_data) 3. [Real-time trigger + Push delivery](/api-reference/web-scraper-ide-api/Trigger_a_scraper_for_real_time_collection) 4. [Real-time trigger + API polling](/api-reference/web-scraper-ide-api/Receive_data_from_real_time_work_scraper) # Receive Batch Data Source: https://docs.brightdata.com/api-reference/web-scraper-ide-api/Receive_batch_data web-scraper-ide-rest-api GET /dca/dataset Receive Batch data # Receive data from real-time work scraper Source: https://docs.brightdata.com/api-reference/web-scraper-ide-api/Receive_data_from_real_time_work_scraper web-scraper-ide-rest-api GET /dca/get_result Receive data from real-time work scraper # Trigger a scraper for batch collection method Source: https://docs.brightdata.com/api-reference/web-scraper-ide-api/Trigger_a_scraper_for_batch_collection_method web-scraper-ide-rest-api POST /dca/trigger Trigger a scraper for batch collection method # Trigger a scraper for real-time collection Source: https://docs.brightdata.com/api-reference/web-scraper-ide-api/Trigger_a_scraper_for_real_time_collection web-scraper-ide-rest-api POST /dca/trigger_immediate Trigger a scraper for real-time collection # IDE Job data API Source: https://docs.brightdata.com/api-reference/web-scraper-ide-api/job-data web-scraper-ide-rest-api GET /dca/log/{job_id} Get job data by job ID # Archive Source: https://docs.brightdata.com/archive-quickstart # Browser API Source: https://docs.brightdata.com/browser-api-quickstart # Browser Extension Source: https://docs.brightdata.com/browser-extension-quickstart # 添加静态 IP Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Add_Static_Datacenter_ISP_IPs POST /zone/ips Add Static IPs for Datacenter and ISP **Warning:** This API can modify your ac数量 settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an ac数量](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 添加区域 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Add_a_Zone POST /zone Add a new zone **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 添加计划示例 ```sh 添加共享数据中心 - 按使用区域付费 curl "https://api.brightdata.com/区域" -H "Content-Type: application/json" -H "身份验证: 持有者 API_KEY" -d "{'区域':{'名称':'区域_名称'},'计划':{'类型':'static','ips_类型':'shared','ip_alloc_preset':'shared_block'}}" ``` ```sh Add an ISP - Exclusive 区域 curl "https://api.brightdata.com/区域" -H "Content-Type: application/json" -H "身份验证: 持有者 API_KEY" -d "{'区域':{'名称':'区域_名称'},'计划':{'类型':'static','ips_类型':'dedicated','pool_ip_类型':'static_res','ips':3}}" ``` ```sh Add a 住宅 - Exclusive with Exclusive domain 'example.com' 区域 curl "https://api.brightdata.com/区域" -H "Content-Type: application/json" -H "身份验证: 持有者 API_KEY" -d "{'区域':{'名称':'区域_名称'},'计划':{'类型':'resident','VIP_类型':'domain','VIP':true,'VIP':3,'domain_whitelist':'example.com'}}" ``` * ```sh Add a 移动设备 - Shared 区域 curl "https://api.brightdata.com/区域" -H "Content-Type: application/json" -H "身份验证: 持有者 API_KEY" -d "{'区域':{'名称':'区域_名称'},'计划':{'类型':'resident','移动设备':true,'VIP_类型':'shared'}}" ``` ```sh Add a Web Unlcoker Zone curl "https://api.brightdata.com/区域" -H "Content-Type: application/json" -H "身份验证: 持有者 API_KEY" -d "{'区域':{'名称':'区域_名称'},'计划':{'类型':'unblocker'}}" ``` ```sh Add a SERP Zone curl "https://api.brightdata.com/区域" -H "Content-Type: application/json" -H "身份验证: 持有者 API_KEY" -d "{'区域':{'名称':'区域_名称'},'计划':{'类型':'resident','serp':true}}" ``` ```sh Shell curl "https://api.brightdata.com/zone" -H "Content-Type: application/json" -H "Authorization: Bearer API_KEY" -d "{'zone':{'name':'zone_name'},'plan':{'type':'static|resident','ips_type':'shared|dedicated|selective','ip_alloc_preset':'shared_block','bandwidth':'payperusage|unlimited','mobile':false,'city':false,'asn':false,'vip':false,'vips_type':'shared|vip|domain','vips':0,'domain_whitelist':'test.com fb.com','not_country':null,'vip_country':'any','exclusive_sec':0,'exclusive_type':'days','exclusive_num':0,'ips':0}}" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'https://api.brightdata.com/zone', json: { 'zone': { 'name': 'zone_name' }, 'plan': { 'type': 'static|resident', 'ips_type': 'shared|dedicated|selective', 'ip_alloc_preset': 'shared_block', 'bandwidth': 'payperusage|unlimited', 'mobile': false, 'city': false, 'asn': false, 'vip': false, 'vips_type': 'shared|vip|domain', 'vips': 0, 'domain_whitelist': 'test.com fb.com', 'not_country': null, 'vip_country': 'any', 'exclusive_sec': 0, 'exclusive_type': 'days', 'exclusive_num': 0, 'ips': 0 } }, headers: { 'Authorization': 'Bearer API_KEY' }, }).then(function(data) { console.log(data); }, function(err) { console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{'zone':{'name':'zone_name'},'plan':{'type':'static|resident','ips_type':'shared|dedicated|selective','ip_alloc_preset':'shared_block','bandwidth':'payperusage|unlimited','mobile':false,'city':false,'asn':false,'vip':false,'vips_type':'shared|vip|domain','vips':0,'domain_whitelist':'test.com fb.com','not_country':null,'vip_country':'any','exclusive_sec':0,'exclusive_type':'days','exclusive_num':0,'ips':0}}"; String res = Executor.newInstance() .addHeader("Authorization", "Bearer API_KEY") .execute(Request.Post("https://api.brightdata.com/zone") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("https://api.brightdata.com/zone"), Headers = { { "Authorization", "Bearer API_KEY" } }, Content = new StringContent(JsonConvert.SerializeObject(new { zone = new { name = "zone_name" }, plan = new { type = "static|resident", ips_type = "shared|dedicated|selective", ip_alloc_preset = "shared_block", bandwidth = "payperusage|unlimited", mobile = false, city = false, asn = false, vip = false, vips_type = "shared|vip|domain", vips = 0, domain_whitelist = "test.com fb.com", not_country = null, vip_country = "any", exclusive_sec = 0, exclusive_type = "days", exclusive_num = 0, ips = 0 } }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import requests data = { 'zone':{'name': 'zone_name'}, 'plan':{ 'type': 'static|resident', 'ips_type': 'shared|dedicated|selective', 'ip_alloc_preset':'shared_block', 'bandwidth':'payperusage|unlimited', 'mobile':False, 'city':False, 'asn':False, 'vip':False, 'vips_type':'shared|vip|domain', 'vips':0, 'domain_whitelist':'test.com fb.com', 'not_country':null, 'vip_country':'any', 'exclusive_sec':0, 'exclusive_type':'days', 'exclusive_num':0, 'ips':0 } } headers = {'Authorization': 'Bearer API_KEY'} r = requests.post( 'https://api.brightdata.com/zone', data=data, headers=headers ) print(r.content) ``` # 更改账户状态 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Change_account_status PUT /status **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 获取可用国家 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_Available_Countries GET /countrieslist List all available countries per zone type 将您的 API 令牌粘贴到授权字段中。要获取令牌,请[创建账户](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground),并了解如何[生成令牌](/cn/api-reference/unlocker/api_token)。 # 区域统计信息 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_Zone_Static_Datacenter_ISP_IPs GET /zone/ips Get Zone Static (Datacenter/ISP) IPs 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) ```json Response when "ip_per_country=true" { "gb":198, "de":282, "br":418, "au":115, "jp":292, "nl":421, "uz":333, "il":517, "kg":566, "az":498, "lv":484, "tw":372, "sg":184 } ``` # 区域信息 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_Zone_info GET /zone Get Zone status 请将 API 令牌粘贴到“授权”字段。 如需获取令牌,请创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/api-reference/unlocker/api_token) # 区域密码 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_Zone_passwords GET /zone/passwords Get Zone passwords 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 区域权限 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_Zone_permissions GET /zone/permissions Get Zone permissions 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 区域状态 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_Zone_status GET /zone/status Get Zone status 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 账户状态 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_account_status GET /status Get account status 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 活动区域 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_active_Zones GET /zone/get_active_zones Return all active zones 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 可用的 IP/区域 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_all_available_dedicated_residential_IPs_per_Zone GET /zone/route_vips Get all available residential dedicated IPs per Zone 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) 仅适用于专属固定区域 # 数据中心和 ISP 的可用 IP 数量 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_amount_of_available_Datacenter_and_ISP_IPs GET /zone/count_available_ips 请将 API 令牌粘贴到“授权”字段。 如需获取令牌,请创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/api-reference/unlocker/api_token) * 当前区域计划中可用的 IP: ```sh curl "https://api.brightdata.com/count_available_ips?zone=ZONE" -H "Authorization: Bearer API_KEY" ``` * 抽象计划,专用 IP: ```sh dedicated IPs curl "https://api.brightdata.com/count_available_ips?plan=\{\"ips_type\":\"dedicated\"\}" -H "Authorization: Bearer API_KEY" ``` * 抽象计划,共享 IP 位于美国: ```sh curl "https://api.brightdata.com/count_available_ips?plan=\{\"country\":\"us\",\"ips_type\":\"shared\"\}" -H "Authorization: Bearer API_KEY" ``` * 抽象计划,专用 IP 位于美国: ```sh curl "https://api.brightdata.com/count_available_ips?plan=\{\"country\":\"us\",\"ips_type\":\"dedicated\"\}" -H "Authorization: Bearer API_KEY" ``` * 抽象计划,共享 IP 位于美国丹佛: ```sh curl "https://api.brightdata.com/count_available_ips?plan=\{\"ips_type\":\"shared\",\"country_city\":\"us-denver\",\"city\":true\}" -H "Authorization: Bearer API_KEY" ``` * 抽象计划,共享 IP 位于美国,仅适用于以下域名:amazon.com 和 fb.com: ```sh curl "https://api.brightdata.com/count_available_ips?plan=\{\"ips_type\":\"selective\",\"country\":\"us\",\"domain_whitelist\":\"amazon.com%20fb.com\"\}" -H "Authorization: Bearer API_KEY" ``` * 抽象计划,共享 IP 位于美国,地理位置 IP 数据库:需同时存储在 `maxmind` 和 `dbip` 中: ```sh curl "https://api.brightdata.com/count_available_ips?plan=\{\"ips_type\":\"shared\",\"country\":\"us\",\"geo_db\":\{\"maxmind\":true,\"dbip\":true\}\}" -H "Authorization: Bearer API_KEY" ``` # 获取国家/地区城市 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_country_cities GET /cities Get a list of cities from a Country 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 获取当前服务状态 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_current_service_status GET /network_status/{NETWORK_TYPE} 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 列出所有国家/地区的可用城市(静态网络) Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_list_of_available_cities_of_static_network_per_country GET /zone/static/cities Get list of available cities of static network per country 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 静态区域的实时状态 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_live_status_of_Static_Datacenter_ISP_Zone_and_IPs_with_connectivity_problem GET /zone/ips/unavailable Get live status of Static (Datacenter/ISP) Zone and IPs with connectivity problem 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 最近尝试使用您的区域的 IP Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_recent_IPs_attempting_to_use_your_Zone GET /zone/recent_ips 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 可用数据中心和 ISP IP/区域 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_the_available_Data_center_ISP_IPs_per_Zone GET /zone/route_ips Get the available Data center/ISP IPs per Zone 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 区域的带宽统计信息 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_the_bandwidth_stats_for_a_Zone GET /zone/bw Get the bandwidth stats for a Zone 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 所有区域的带宽统计信息 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_the_bandwidth_stats_for_all_your_Zones GET /customer/bw Get the bandwidth stats for all your Zones 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) Sample response redacted # 区域总成本和带宽 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_the_total_cost_and_bandwidth_stats_for_a_Zone GET /zone/cost Get the total cost and bandwidth stats for a Zone 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 总余额 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Get_total_balance_through_API GET /customer/balance Get total balance through API 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 在区域间迁移静态 IP Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Migrate_Static_Datacenter_ISP_IPs_between_zones POST /zone/ips/migrate Migrate Static IPs between zones (Datacenter/ISP) **警告**:此 API 可能会修改账户设置、影响正常操作或产生额外费用。 请将 API 令牌粘贴到“授权”字段。 如需获取令牌,请创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/api-reference/unlocker/api_token) # 刷新静态 IP(数据中心/ISP) Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Refresh_Static_Datacenter_ISP_IPs POST /zone/ips/refresh **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) [查看刷新数据中心 IP 的定价](/cn/general/account/billing-and-pricing/faqs#what-is-the-cost-for-refreshing-data-center-IP) ```json Sample Response for Static IPs { "IP":[ "1.1.1.1", "1.1.1.2", "1.1.1.3" ], "new_IP":[ "1.1.1.1", "1.1.1.3" ] } ``` # 刷新专用住宅 IP Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Refresh_dedicated_residential_IPs POST /zone/ips/refresh **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) 每个专用住宅 IP 刷新费用为每个 IP 每次刷新 \$0.02 # 移除静态 IP(数据中心/ISP) Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Remove_Static_Datacenter_ISP_IPs DELETE /zone/ips **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 移除区域 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Remove_Zone DELETE /zone **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 移除专用住宅 IP Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Remove_dedicated_residential_IPs DELETE /zone/vips **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 切换 100% 正常运行时间 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Switch_100_uptime_ON_OFF_in_a_Static_zone POST /zone/switch_100uptime Switch automatic failover (100% uptime) ON/OFF in a Static zone **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 开启/关闭区域 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/Turn_on_off_Zone POST /zone/change_disable **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 建立 IP 白名单 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/allowlist-ip POST /zone/whitelist Add IP to zone's allowlist **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 建立域名白名单/黑名单 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/allowlist-or-denylist-domains POST /zone/domain_perm Add domain to Zone allowlist or denylist **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 将 IP 列入黑名单 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/denylist-ip POST /zone/blacklist Add IP to zone's denylist **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 获取待更换代理 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/get-proxies-pending-replacement GET /zone/proxies_pending_replacement Get list of all proxies in the provided zone, which are pending replacement # 从区域白名单/黑名单中移除域名 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/remove-domain-from-zone-allowlist-or-denylist DELETE /zone/domain_perm **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 从区域白名单中移除 IP Source: https://docs.brightdata.com/cn/api-reference/account-management-api/remove-ip-from-zone-allowlist DELETE /zone/whitelist **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 从区域黑名单中移除 IP Source: https://docs.brightdata.com/cn/api-reference/account-management-api/remove-ip-from-zone-denylist DELETE /zone/blacklist **Warning:** This API can modify your account settings, damage your operations or incur charges. 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # 特定区域的白名单 IP 列表 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/zone-ip-allowlist GET /zone/whitelist Get a list of allowlisted IPs for a specific Zone 请将 API 令牌粘贴到“授权”字段。 如需获取令牌,请创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/api-reference/unlocker/api_token) # 区域黑名单 IP 列表 Source: https://docs.brightdata.com/cn/api-reference/account-management-api/zone-ip-denylist GET /zone/blacklist Get a list of denylisted IPs for a specific Zone 将您的 API 令牌粘贴到身份验证字段。要获得令牌,创建账户并了解如何生成令牌 [Create an account](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground), [generate an API key](/cn/api-reference/unlocker/api_token) # API vs 本地访问 Source: https://docs.brightdata.com/cn/api-reference/api-vs-native-access 哪种方式更适合我? Bright Data 的代理和解锁平台(Unlocker API、SERP API 和代理产品)可通过两种方式访问: 1. **API 访问** - 通过标准化的 HTTP 方法(如 GET、POST、REST API 调用)和端点与 Bright Data 平台交互,以发送请求并检索结构化数据(JSON 或 HTML)。 适用于:与内部脚本或第三方应用程序的无缝、可扩展集成。 2. **本地访问** - 通过为每个产品定制的代理协议直接与 Bright Data 平台交互。 适用于:需要直接代理连接(IP:端口)的网页浏览器和[第三方工具](https://docs.brightdata.com/integrations)集成。 ## **请求示例** ### **API 访问** ```sh curl https://api.brightdata.com/request -H "Content-Type: application/json" -H "Authorization: Bearer " -d "{"zone": "zone_name","url": "https://www.example.com", "format": "raw"}" ``` ### **本地访问** ```sh curl -i --proxy brd.superproxy.io:33335 --proxy-user : "https://www.example.com" ``` ## **API vs 本地访问对比** | 参数 | API 访问 | 本地访问 | | ---------- | -------------------------- | ------------------- | | **推荐产品** | Unlocker API, SERP API, 代理 | 代理 - 适用于需要直接代理连接的情况 | | **SSL 证书** | 不需要 | 需要(数据中心和 ISP 代理除外) | | **账户验证** | 通常不需要 | 某些特定域名需要 | | **连接方式** | API 端点 | 代理端点 | | **身份验证** | API 令牌 | 用户名:密码 | | **输出格式** | HTML 或 JSON | HTML | ## **计费** 本地访问和 API 访问的计费 **相同**,费用基于您的 Bright Data 产品的常规定价。 # 交付快照 Source: https://docs.brightdata.com/cn/api-reference/marketplace-dataset-api/deliver-snapshot dca-api POST /datasets/snapshots/{id}/deliver Deliver the dataset snapshot # 快照内容 Source: https://docs.brightdata.com/cn/api-reference/marketplace-dataset-api/download-the-file-by-snapshot_id dca-api GET /datasets/snapshots/{id}/download Get dataset snapshot content # 过滤数据集 Source: https://docs.brightdata.com/cn/api-reference/marketplace-dataset-api/filter-dataset dca-api POST /datasets/filter Create a dataset snapshot based on a provided filter 将您的 API 令牌粘贴到授权字段中。要获取令牌,[创建一个账户](https://brightdata.com/?hs_signup=1\&utm_source=docs\&utm_campaign=playground),并了解如何[生成令牌](/api-reference/unlocker/api_token)。 ## 一般描述 * 调用此端点将启动异步作业,以过滤数据集并在您的账户中创建包含过滤数据的快照。 * 该作业的最大完成时间为 5 分钟。如果在此时间范围内未完成,作业将被取消。 * 创建数据集快照的费用取决于快照大小和记录单价。 * 过滤组的最大嵌套深度为 3。 ## 过滤语法 ### 运算符 下表显示了可用于字段过滤的运算符。 | 操作符 | 字段类型 | 描述 | | -------------------- | ------ | --------------------------------------------------------------------------------- | | = | 任意 | 等于 | | != | 任意 | 不等于 | | \< | 数字, 日期 | 小于 | | \<= | 数字, 日期 | 小于或等于 | | > | 数字, 日期 | 大于 | | >= | 数字, 日期 | 大于或等于 | | `in` | 任意 | 测试字段值是否等于筛选值中提供的任意值 | | `not_in` | 任意 | 测试字段值是否不等于筛选值中提供的所有值 | | `includes` | 数组, 文本 | 测试字段值是否包含筛选值。如果筛选值是一个单一字符串,则匹配字段值包含该字符串的记录。如果筛选值是一个字符串数组,则匹配字段值包含数组中的至少一个字符串的记录。 | | `not_includes` | 数组, 文本 | 测试字段值是否不包含筛选值。如果筛选值是一个单一字符串,则匹配字段值不包含该字符串的记录。如果筛选值是一个字符串数组,则匹配字段值不包含数组中的任何字符串的记录。 | | `array_includes` | 数组 | 测试筛选值是否在字段值中(精确匹配) | | `not_array_includes` | 数组 | 测试筛选值是否不在字段值中(精确匹配) | | `is_null` | 任意 | 测试字段值是否等于 NULL。操作符不接受任何值。 | | `is_not_null` | 任意 | 测试字段值是否不等于 NULL。操作符不接受任何值。 | ### 组合多个筛选器 可以使用两个逻辑运算符:'and'(与),'or'(或)将多个字段筛选器组合成筛选器组。\ API 支持最多 3 层嵌套的筛选器。\ 筛选器组示例: ```json { // operator can be one of ["and", "or"] "operator": "and", // an array of field filters "filters": [ { "name": "reviews_count", "opeartor": ">", "value": "200" }, { "name": "rating", "operator": ">", "value": "4.5" } ] } ``` # 获取快照元数据 Source: https://docs.brightdata.com/cn/api-reference/marketplace-dataset-api/get-snapshot-meta dca-api GET /datasets/snapshots/{id} Get dataset snapshot metadata # 获取快照部分 Source: https://docs.brightdata.com/cn/api-reference/marketplace-dataset-api/get-snapshot-parts dca-api GET /datasets/snapshots/{id}/parts Get dataset snapshot delivery parts. All query parameters used here need to match those used when downloading the snapshot to get accurate parts # 获取快照元数据 Source: https://docs.brightdata.com/cn/api-reference/marketplace-dataset-api/get-snapshots-meta dca-api GET /datasets/snapshots Get dataset snapshots metadata # 添加用户 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/add_a_user **API 端点:** `POST` `/api/lpm_user` ## `POST` 正文 要添加的用户电子邮件地址 {/* TODO: Missing Sample Response */} ```sh Shell curl "http://127.0.0.1:22999/api/lpm_user" -H "Content-Type: application/json" -d '{"email":"test@example.com"}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/lpm_user', json: {'email':'test@example.com'} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"email\":\"test@example.com\"}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/lpm_user") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/lpm_user"), Content = new StringContent(JsonConvert.SerializeObject(new { email = "test@example.com" }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'email':'test@example.com'} r = requests.post('http://127.0.0.1:22999/api/lpm_user', data=json.dumps(data)) print(r.content) ``` # 建立 IP 白名单 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/allowlist-ips **API 端点:** `PUT` `/api/wip` API 令牌 应通过“生成令牌 API”命令生成令牌 [Generate API Key](/cn/api-reference/proxy-manager/generate_token_for_token_based_authentication) ## `PUT` body 要列入白名单的 IP。例如,`ip="1.2.1.2"` ```sh Shell curl -X PUT "http://127.0.0.1:22999/api/wip" -H "Authorization: API key" -H "Content-Type: application/json" -d '{"ip":"1.2.1.2"}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'PUT', url: 'http://127.0.0.1:22999/api/wip', json: {'ip':'1.2.1.2'}, headers: {'Authorization': 'API key'}, }).then(function(data){ console.log(data);}, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ip\":\"1.2.1.2\"}"; String res =Executor.newInstance() .addHeader("Authorization", "API key") .execute(Request.Put("http://127.0.0.1:22999/api/wip") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent() .asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Put, RequestUri = new Uri("http://127.0.0.1:22999/api/wip"), Headers = {{"Authorization", "API key"}}, Content = new StringContent(JsonConvert.SerializeObject(new { ip = "1.2.1.2" }), Encoding.UTF8, "application/json") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import json import requests data = {'ip':'1.2.1.2'} headers = {'Authorization': 'API key'} r = requests.put( 'http://127.0.0.1:22999/api/wip', data=json.dumps(data), headers=headers ) print(r.content) ``` ```json 200 Successful response ``` ```json 400 Bad request. No IP was passed ``` ```json 403 Forbidden. No authentication provided ``` ```json 422 Unprocessable entity. Invalid IP was passed ``` # 封禁 IP(所有端口) Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/ban_an_ip_for_all_ports **API 端点:** `POST` `/api/banip` ## `POST` body 所要封禁的 IP。例如 `1.2.1.2` 封禁向指定域发送请求的 IP 在指定毫秒内封禁 IP ```sh Shell curl "http://127.0.0.1:22999/api/banip" -H "Content-Type: application/json" -d '{"ip":"1.2.1.2","domain":"example.com","ms":60000}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/banip', json: {'ip':'1.2.1.2','domain':'example.com','ms':60000} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ip\":\"1.2.1.2\",\"domain\":\"example.com\",\"ms\":60000}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/banip") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/banip"), Content = new StringContent(JsonConvert.SerializeObject(new { ip = "1.2.1.2", domain = "example.com", ms = 60000 }), Encoding.UTF8, "application/json") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'ip':'1.2.1.2','domain':'example.com','ms':60000} r = requests.post('http://127.0.0.1:22999/api/banip', data=json.dumps(data)) print(r.content) ``` # 封禁 IP Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/ban_an_ip_single_ **API 端点:** `POST` `/api/proxies/{PORT}/banip` ## 路径参数 现有代理服务器端口号 ## `POST` 正文 所要封禁的 IP。例如 `1.2.1.2` 封禁向指定域发送请求的 IP 在指定毫秒内封禁 IP ```sh Shell curl "http://127.0.0.1:22999/api/proxies/{PORT}/banip" -H "Content-Type: application/json" -d '{"ip":"1.2.1.2","domain":"example.com","ms":60000}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/proxies/{PORT}/banip', json: {'ip':'1.2.1.2','domain':'example.com','ms':60000} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ip\":\"1.2.1.2\",\"domain\":\"example.com\",\"ms\":60000}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/proxies/{PORT}/banip") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}/banip"), Content = new StringContent(JsonConvert.SerializeObject(new { ip = "1.2.1.2", domain = "example.com", ms = 60000 }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'ip':'1.2.1.2','domain':'example.com','ms':60000} r = requests.post('http://127.0.0.1:22999/api/proxies/{PORT}/banip', data=json.dumps(data)) print(r.content) ``` # 封禁多个 IP(阵列) Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/ban_ips_array_ **API 端点:** `POST` `/api/proxies/{PORT}/banips` ## 路径参数 现有代理服务器端口号 ## `POST` body 所要封禁的 IP。例如 `["10.0.0.1", "20.0.0.1"]` 封禁向指定域发送请求的 IP 在指定毫秒内封禁 IP ```sh Shell curl "http://127.0.0.1:22999/api/proxies/{PORT}/banips" -H "Content-Type: application/json" -d '{"ips":["10.0.0.1","20.0.0.1"],"domain":"example.com","ms":60000}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/proxies/{PORT}/banips', json: {'ips':['10.0.0.1','20.0.0.1'],'domain':'example.com','ms':60000} }).then(function(data){ console.log(data);}, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ips\":[\"10.0.0.1\",\"20.0.0.1\"],\"domain\":\"example.com\",\"ms\":60000}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/proxies/{PORT}/banips") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}/banips"), Content = new StringContent(JsonConvert.SerializeObject(new { ips = [ "10.0.0.1","20.0.0.1" ], domain = "example.com", ms = 60000 }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'ips':['10.0.0.1','20.0.0.1'],'domain':'example.com','ms':60000} r = requests.post('http://127.0.0.1:22999/api/proxies/{PORT}/banips', data=json.dumps(data)) print(r.content) ``` # 创建新的代理端口 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/create_a_new_proxy_port **API 端点:** `POST` `/api/proxies` ## `POST` 正文 HTTP 代理的端口 设置为 `persist` 以将代理保存到配置文件中。 将端口定义乘以给定的次数 用户 `[string]` 的列表。此选项必须与 `multiply_users` 一起使用 启用 SSL 分析 选择 SSL 库 | value | description | | ---------- | ----------- | | `open_ssl` | 打开 SSL 库 | | `flex_tls` | Flex TLS 库 | 所要监听的接口或 IP 客户名称 区域名称 区域密码 超级代理的主机名或 IP 超级代理端口 确定代理管理器和超级代理之间将使用哪种连接 | | | | ------- | - | | `http` | | | `https` | | | `socks` | | 超级代理出现故障时自动重试 对不安全的主机启用 SSL 连接/分析 国家/地区 城市 ASN 数据中心 IP gIP 来自外部供应商的代理列表。 格式:`[username:password@]ip[:port]` * proxy\[string] 外部供应商 IP 的默认用户名 外部供应商 IP 的默认密码 外部供应商 IP 的默认端口 DNS resolving | | | | -------- | - | | `local` | | | `remote` | | 通过 DNS 进行反向查询 通过文件进行反向查询 通过值进行反向查询 所有代理请求的会话 使用每个请求主机的会话维护每个主机的 IP 会话池大小 限制超过给定数量的请求 代理请求规则 出错时阻止或允许通过超级代理自动发送请求 对等 IP 的操作系统 请求标头 * `name[string]` * `value[string]` 请求调试信息 | | | | ------ | - | | `full` | | | `none` | | `x-lpm-authorization header` 解锁器移动 UA 浏览器使用的时区 ID 浏览器屏幕尺寸 浏览器中的 WebRTC 插件行为 带宽限制参数 * days \[integer] * bytes \[integer] * reenewable \[boolean] - 更新每个周期的字节限制或者使用单个周期,并在达到周期的最后一天时停止使用。 默认值为 true {/* TODO: Add description */} ```JSON Sample Response { "port":24000, "zone":"zone_name", "proxy_type":"persist", "customer":"customer_id", "password":"password", "whitelist_ips":[] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/proxies" -H "Content-Type: application/json" -d '{"proxy":{"port":24000,"zone":"ZONE","proxy_type":"persist","customer":"CUSTOMER","password":"password","whitelist_ips":[]}}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', method: 'POST', method: 'POST', url: 'http://127.0.0.1:22999/api/proxies', json: {'proxy':{'port':24000,'zone': 'ZONE','proxy_type':'persist','customer':'CUSTOMER','password':'password','whitelist_ips':[]}} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"proxy\":{\"port\":24000,\"zone\":\"ZONE\",\"proxy_type\":\"persist\",\"customer\":\"CUSTOMER\",\"password\":\"password\",\"whitelist_ips\":[]}}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/proxies") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` # 删除代理端口 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/delete_a_proxy_port **API 端点:** `DELETE` `/api/proxies/{PORT}` ## 路径参数 现有代理服务器端口编号 ```sh Shell curl -X DELETE "http://127.0.0.1:22999/api/proxies/{PORT}" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'DELETE', url: 'http://127.0.0.1:22999/api/proxies/{PORT}' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Delete("http://127.0.0.1:22999/api/proxies/{PORT}")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Delete, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.delete('http://127.0.0.1:22999/api/proxies/{PORT}') print(r.content) ``` # 删除代理端口(多个) Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/delete_proxy_ports **API 端点:** `POST /api/proxies/delete` ## `POST` body 端口数组 ```sh Shell curl "http://127.0.0.1:22999/api/proxies/delete" -H "Content-Type: application/json" -d '{"ports":[24000,24001]}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/proxies/delete', json: {'ports':[24000,24001]} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ports\":[24000,24001]}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/proxies/delete") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/delete"), Content = new StringContent(JsonConvert.SerializeObject(new { ports = [ 24000, 24001 ] }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'ports':[24000,24001]} r = requests.post('http://127.0.0.1:22999/api/proxies/delete', data=json.dumps(data)) print(r.content) ``` # 启用 SSL 分析(所有端口) Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/enable_ssl_analyzing_on_all_proxy_ports **API 端点:** `POST` `/api/enable_ssl` ```sh curl -X POST "http://127.0.0.1:22999/api/enable_ssl" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/enable_ssl' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/enable_ssl")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/enable_ssl") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.post('http://127.0.0.1:22999/api/enable_ssl') print(r.content) ``` # 生成 API 令牌 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/generate_token_for_token_based_authentication **API 端点:** `GET` `/api/gen_token` ```json Sample Response { "token": "RZD9vaQQaL6En7" } ``` ```sh Shell curl "http://127.0.0.1:22999/api/gen_token" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/gen_token' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/gen_token")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/gen_token") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/gen_token') print(r.content) ``` # 获取所有用户 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_all_users **API 端点:** `GET` `/api/lpm_users` ```JSON Sample Response [ { "email": "test@example.com", "password":"password" } ] ``` ```sh Shell curl "http://127.0.0.1:22999/api/lpm_users" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/lpm_users' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance().execute( Request.Get("http://127.0.0.1:22999/api/lpm_users") ).returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/lpm_users") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/lpm_users') print(r.content) ``` # 获取区域中分配的 gIP Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_allocated_gips_in_zone **API 端点:** `GET` `/api/allocated_vips` ## 查询参数 住宅/移动专属区域名称 ```json Sample Response { "ips": [ "gIP_1", "gIP_2" ] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/allocated_vips?zone=ZONE" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/allocated_vips?zone=ZONE' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/allocated_vips?zone=ZONE")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/allocated_vips?zone=ZONE") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/allocated_vips?zone=ZONE') print(r.content) ``` # 获取区域中分配的 IP Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_allocated_ips_in_zone **API 端点:** `GET` `/api/allocated_ips` ## 查询参数 静态(数据中心/互联网服务提供商)区域名称 ```JSON Sample Response { "ips": [ "10.0.0.1", "20.0.0.1" ] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/allocated_ips?zone=ZONE" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/allocated_ips?zone=ZONE' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/allocated_ips?zone=ZONE")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/allocated_ips?zone=ZONE") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/allocated_ips?zone=ZONE') print(r.content) ``` # 获取封禁的 IP Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_banned_ips **API 端点:** `GET` `/api/banlist/{PORT}` `full=true` 参数是可选的,可用于提供有关封禁列表中每个条目的更多详情 ```JSON Sample Response { "ips": [ "10.20.30.40", "50.60.70.80|example.com" ] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/banlist/{PORT}?full=true" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/banlist/{PORT}?full=true' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/banlist/{PORT}?full=true")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/banlist/{PORT}?full=true") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/banlist/{PORT}?full=true') print(r.content) ``` # 获取当前 NodeJS 版本 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_currently_running_nodejs_version **API 端点:** `GET` `/api/node_version` ```JSON Sample Response { "current": { "options": { "loose": false, "includePrerelease":false }, "loose": false, "raw":"v12.16.1\n", "major":12, "minor":16, "patch":1, "prerelease":[], "build":[], "version":"12.16.1" }, "satisfied": true, "recommended": ">=10.0" } ``` {/* TODO: Missing Code Examples */} # 为所有正在运行的代理获取有效配置 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_effective_configuration_for_all_running_proxies **API 端点:** `GET` `/api/proxies_running` ```JSON Sample Response [ { "port": 24000, "zone": "ZONE", "proxy_type": "persist", "customer": "CUSTOMER", "password": "password", "whitelist_ips":[] } ] ``` ```sh Shell curl "http://127.0.0.1:22999/api/proxies_running" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/proxies_running' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/proxies_running")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies_running") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/proxies_running') print(r.content) ``` # 获取已启用区域的配置 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_enabled_zones_configuration **API 端点:** `GET` `/api/zones` ```JSON Sample Response { "name": "ZONE", "perm": "country ip route_all route_dedicated", "plan": {}, "password":"password" } ``` ```sh Shell curl "http://127.0.0.1:22999/api/zones" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/zones' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/zones")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/zones") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import requests r = requests.get('http://127.0.0.1:22999/api/zones') print(r.content) ``` # 获取指定代理的显式配置 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_explicit_configuration_of_all_or_specified_proxies **API 端点:** `GET` `/api/proxies/{PORT}` 端口 参数是可选的。 您可以跳过它来获取所有代理\* ```JSON Sample Response [ {"port": 24000, "zone": "ZONE", "proxy_type": "persist", "customer": "CUSTOMER", "password": "password", "whitelist_ips": [] }, { "port": 33335, "zone": "ZONE", "listen_port": 33335, "customer": "CUSTOMER", "password":"password" } ] ``` ```sh Shell curl "http://127.0.0.1:22999/api/proxies/{PORT}" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/proxies/{PORT}' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/proxies/{PORT}")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/proxies/{PORT}') print(r.content) ``` # 获取常规设置 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_general_settings **API 端点:** `GET` `/api/settings` ```JSON Sample Response { "customer": "CUSTOMER", "zone": "ZONE", "password": "password", "www_whitelist_ips": [], "whitelist_ips": [], "fixed_whitelist_ips": [], "read_only": false, "config": "/home/user/proxy_manager/.luminati.json", "test_url":"http://lumtest.com/myip.json", "logs":1000, "log":"notice", "har_limit": 1024, "request_stats": true, "dropin": true, "pending_ips": [], "pending_www_ips": [], "zagent": false, "sync_config": true } ``` ```sh Shell curl "http://127.0.0.1:22999/api/settings" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/settings' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/settings")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/settings") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/settings') print(r.content) ``` # 获取 HAR 日志 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_har_logs **API 端点:** `GET` `/api/logs` 要从尾部获取的日志数量 要从尾部获取的日志数量 要获取的最大请求数量 URL 的正则表达式搜索查询 端口号的下限 端口号的上限 按状态码筛选请求 排序依据的参数 是降序排序方向 {/* TODO: Missing Sample Response */} ```sh Shell curl "http://127.0.0.1:22999/api/logs" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/logs' }).then(function(data){ console.log(data);}, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/logs")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/logs") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/logs') print(r.content) ``` # 获取代理管理器版本 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_proxy_manager_version **API 端点:** `GET` `/api/version` ```json Sample Response { "version":"1.280.385", "argv":"" } ``` ```sh Shell curl "http://127.0.0.1:22999/api/version" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/version' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/version")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/version") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/version') print(r.content) ``` # 获取代理端口状态 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_proxy_port_status **API 端点:** `GET` `/api/proxy_status/{端口}` 现有代理端口号 ```JSON Sample Response { "status":"ok", "status_details":[] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/proxy_status/{PORT}" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/proxy_status/{PORT}' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance().execute( Request.Get("http://127.0.0.1:22999/api/proxy_status/{PORT}") ).returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/proxy_status/{PORT}") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/proxy_status/{PORT}') print(r.content) ``` # 获取最新状态 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_recent_stats **API 端点:** `GET /api/recent_stats` ```JSON Sample Response { "ports": {}, "status_code": [], "hostname": [], "protocol": [], "total": 0, "success": 0, "ssl_enable":true} ``` ```sh Shell curl "http://127.0.0.1:22999/api/recent_stats" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/recent_stats' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance().execute( Request.Get("http://127.0.0.1:22999/api/recent_stats") ).returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/recent_stats") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/recent_stats') print(r.content) ``` # 从日志文件中获取结尾部分 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_tail_from_the_log_file **API 端点:** `GET` `/api/general_logs` 要从尾部获取的日志数量 {/* TODO: Missing Sample Response */} ```sh Shell curl "http://127.0.0.1:22999/api/general_logs?limit=5" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/general_logs?limit=5' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/general_logs?limit=5")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/general_logs?limit=5") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/general_logs?limit=5') print(r.content) ``` # 获取最新的代理管理器版本 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/get_the_latest_proxy_manager_versions **API 端点:** `GET` `/api/last_version` ```json Sample Response { "version": "1.280.385", "newer": false, "versions": [ { "ver": "1.280.385", "type": "stable", "changes": [ { "type": "star", "text": "Add render option for unblocker and serp zones" } ] } ] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/last_version" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/last_version' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/last_version")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/last_version") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.get('http://127.0.0.1:22999/api/last_version') print(r.content) ``` # 刷新区域中的 IP/gIP Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/refresh_ips_or_gips_in_zone **API 端点:** `POST` `/api/refresh_IP` ## `POST` body 区域名称 Static IPs. e.g. `IP=["ip1","ip2"]` gIPs. e.g. `VIP=["gip1","gip2"] [数组]` ```json Sample Response { "ips": [ { "ip":"10.0.0.1", "maxmind":"us" }, { "ip":"20.0.0.1", "maxmind":"us" } ] } ``` ```sh Shell curl "http://127.0.0.1:22999/api/refresh_ips" -H "Content-Type: application/json" -d '{"zone":"ZONE","ips":["10.0.0.1"]}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/refresh_ips', json: {'zone':'ZONE','ips':['10.0.0.1']} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"zone\":\"ZONE\",\"ips\":[\"10.0.0.1\"]}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/refresh_ips") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/refresh_ips"), Content = new StringContent(JsonConvert.SerializeObject(new { zone = "ZONE", ips = ["10.0.0.1"] }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import requests data = {'zone':'ZONE','ips':['10.0.0.1']} r = requests.post('http://127.0.0.1:22999/api/refresh_ips', data=json.dumps(data)) print(r.content) ``` # 刷新代理管理器端口会话 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/refresh_proxy_manager_port_sessions **API 端点:** `GET` `/api/refresh_sessions/{PORT}` ## 路径参数 现有代理端口号 ```sh Shell curl "http://127.0.0.1:22999/api/refresh_sessions/{PORT}" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'http://127.0.0.1:22999/api/refresh_sessions{PORT}' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Get("http://127.0.0.1:22999/api/refresh_sessions/{PORT}")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("http://127.0.0.1:22999/api/refresh_sessions/{PORT}") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import requests r = requests.get('http://127.0.0.1:22999/api/refresh_sessions/{PORT}') print(r.content) ``` # 重启 Proxy Manager Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/restart_proxy_manager **API 端点:** `POST` `/api/restart` ```sh Shell curl -X POST "http://127.0.0.1:22999/api/restart" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/restart' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/restart")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/restart") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.post('http://127.0.0.1:22999/api/restart') print(r.content) ``` # 关闭 Proxy Manager Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/shutdown_proxy_manager **API 端点:** `POST` `/api/shutdown` ```sh Shell curl -X POST "http://127.0.0.1:22999/api/shutdown" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/shutdown' }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/shutdown")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/shutdown") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import requests r = requests.post('http://127.0.0.1:22999/api/shutdown') print(r.content) ``` # 建立访问用户界面的 IP 白名单 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/ui-allowlist-ips **API 端点:** `PUT` `/api/add_whitelist_ip` API 令牌 应通过“生成令牌 API”命令生成令牌 [Generate API Key](/cn/api-reference/proxy-manager/generate_token_for_token_based_authentication) ## `PUT` body 要列入白名单的 IP,例如 `ip="1.2.1.2"` ```sh Shell curl -X PUT "http://127.0.0.1:22999/api/whitelist_ip" -H "Authorization: API key" -H "Content-Type: application/json" -d '{"ip":"1.2.1.2"}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'PUT', url: 'http://127.0.0.1:22999/api/add_whitelist_ip', json: {'ip':'1.2.1.2'}, headers: {'Authorization': 'API key'}, }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ip\":\"1.2.1.2\"}"; String res = Executor.newInstance() .addHeader("Authorization", "API key") .execute(Request.Put("http://127.0.0.1:22999/api/add_whitelist_ip") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent() .asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Put, RequestUri = new Uri("http://127.0.0.1:22999/api/add_whitelist_ip"), Headers = {{"Authorization", "API key"}}, Content = new StringContent(JsonConvert.SerializeObject(new { ip = "1.2.1.2" }), Encoding.UTF8, "application/json") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import json import requests data = {'ip':'1.2.1.2'} headers = {'Authorization': 'API key'} r = requests.put( 'http://127.0.0.1:22999/api/add_whitelist_ip', data=json.dumps(data) ) print(r.content) ``` # 解禁 IP Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/unban_an_ip **API 端点:** `POST` `/api/proxies/{PORT}/unbanip` ## 路径参数 现有代理服务器的端口号 ## `POST` body 要解禁的 IP。例如 `ip="1.2.1.2"` 解禁向指定域发送请求的 IP。 ```sh Shell curl "http://127.0.0.1:22999/api/proxies/{PORT}/unbanip" -H "Content-Type: application/json" -d '{"ip":"1.2.1.2","domain":"example.com"}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/proxies/{PORT}/unbanip', json: { 'ip': '1.2.1.2', 'domain': 'example.com' } }).then(function(data){ console.log(data);}, function(err){ console.error(err);}); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"ip\":\"1.2.1.2\",\"domain\":\"example.com\"}"; String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/proxies/{PORT}/unbanip") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}/unbanip"), Content = new StringContent(JsonConvert.SerializeObject(new { ip = "1.2.1.2", domain = "example.com" }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'ip':'1.2.1.2','domain':'example.com'} r = requests.post('http://127.0.0.1:22999/api/proxies/{PORT}/unbanip', data=json.dumps(data)) print(r.content) ``` # 解禁 IP(多个) Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/unban_ips **API 端点:** `POST` `/api/proxies/{PORT}/unbanips` ## 路径参数 现有代理服务器的端口号 ```sh Shell curl "http://127.0.0.1:22999/api/proxies/{PORT}/unbanips" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/proxies/{PORT}/unbanips' }).then(function(data){ console.log(data);}, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance().execute( Request.Post("http://127.0.0.1:22999/api/proxies/{PORT}/unbanips" )).returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}/unbanips") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.post('http://127.0.0.1:22999/api/proxies/{PORT}/unbanips') print(r.content) ``` # 更新代理端口 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/update_a_proxy_port **API 端点:** `PUT` `/api/proxies/{PORT}` ## 路径参数 现有代理服务器的端口号 ## `PUT` body HTTP 代理的端口 设置为 `persist` 以将代理保存至配置文件。 将端口定义复用给定次数 user \[string] 的列表。此项必须与 `multiply_users` 一起使用 启用 SSL 分析 选择 SSL 库 | 值 | 描述 | | ---------- | ---------- | | `open_ssl` | 打开 SSL 库 | | `flex_tls` | Flex TLS 库 | 所要监听的接口或 IP 客户名称 区域名称 区域密码 超级代理的主机名或 IP 超级代理端口 确定代理管理器和超级代理之间将使用哪种连接 | | | | ------- | - | | `http` | | | `https` | | | `socks` | | 超级代理出现故障时自动重试 对不安全的主机启用 SSL 连接/分析 国家/地区 城市 ASN 数据中心 IP gIP 来自外部供应商的代理列表。 格式:`[username:password@]ip[:port]` * `proxy[字符串]` 外部供应商 IP 的默认用户名 外部供应商 IP 的默认密码 外部供应商 IP 的默认端口 DNS 解析 | | | | -------- | - | | `local` | | | `remote` | | 通过 DNS 进行反向查询 通过文件进行反向查询 通过值进行反向查询 所有代理请求的会话 使用每个请求主机的会话维护每个主机的 IP 会话池大小 限制超过给定数量的请求 代理请求规则 出错时阻止或允许通过超级代理自动发送请求 对等 IP 的操作系统 请求标头 * name\[字符串] * 值\[字符串] 请求调试信息 | | | | ------ | - | | `full` | | | `none` | | `x-lpm-authorization header` 解锁器移动 UA 浏览器使用的时区 ID 浏览器屏幕尺寸 浏览器中的 WebRTC 插件行为 带宽限制参数 * days \[整数] * bytes \[整数] * reenewable \[布尔值] - 更新每个周期的字节限制或者使用单个周期,并在达到周期的最后一天时停止使用。 默认值为 “true” ```JSON Sample Response { "port":24000, "zone":"ZONE", "proxy_type":"persist", "customer":"CUSTOMER", "password":"password", "whitelist_ips":[] } ``` ```sh Shell curl -X PUT "http://127.0.0.1:22999/api/proxies/{PORT}" -H "Content-Type: application/json" -d '{"proxy":{"port":24000,"zone":"ZONE","proxy_type":"persist","customer":"CUSTOMER","password":"password","whitelist_ips":[]}}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'PUT', url: 'http://127.0.0.1:22999/api/proxies/{PORT}', json: {'proxy':{'port':24000,'zone': 'ZONE','proxy_type':'persist','customer':'CUSTOMER','password':'password','whitelist_ips':[]}} }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"proxy\":{\"port\":24000,\"zone\":\"ZONE\",\"proxy_type\":\"persist\",\"customer\":\"CUSTOMER\",\"password\":\"password\",\"whitelist_ips\":[]}}"; String res = Executor.newInstance() .execute(Request.Put("http://127.0.0.1:22999/api/proxies/{PORT}")) .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Put, RequestUri = new Uri("http://127.0.0.1:22999/api/proxies/{PORT}"), Content = new StringContent(JsonConvert.SerializeObject(new { proxy = new { port = 24000, zone = "ZONE", proxy_type = "persist", customer = "CUSTOMER", password = "password", whitelist_ips = [] } }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python import requests data = {'proxy':{'port':24000,'zone': 'ZONE','proxy_type':'persist','customer':'CUSTOMER','password':'password','whitelist_ips':[]}} r = requests.put('http://127.0.0.1:22999/api/proxies/{PORT}', data=json.dumps(data)) print(r.content) ``` # 更新常规设置 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/update_general_settings **API 端点:** `PUT` `/api/settings` ## `PUT` 正文: 默认区域 允许访问浏览器管理界面的白名单 IP 列表 所有代理的默认白名单 IP 列表,以允许相关 IP 访问代理 待存储的请求日志数量 启用请求统计 ```JSON Sample Response { "customer":"CUSTOMER", "zone":"ZONE", "password":"password", "www_whitelist_ips":[], "whitelist_ips":[], "fixed_whitelist_ips":[], "read_only":false, "config":"/home/user/proxy_manager/.luminati.json", "test_url":"http://lumtest.com/myip.json", "logs":1000, "log":"notice", "har_limit":1024, "request_stats":true, "dropin":true, "pending_ips":[], "pending_www_ips":[], "zagent":false, "sync_config":true } ``` ```sh Shell curl -X PUT "http://127.0.0.1:22999/api/settings" -H "Content-Type: application/json" -d '{"zone":"ZONE","www_whitelist_ips":[],"whitelist_ips":[],"logs":1000,"request_stats":true}' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'PUT', url: 'http://127.0.0.1:22999/api/settings', json: { 'zone': 'ZONE', 'www_whitelist_ips':'[]', 'whitelist_ips':'[]', 'logs':1000, 'request_stats':true } }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"zone\":\"ZONE\",\"www_whitelist_ips\":[],\"whitelist_ips\":[],\"logs\":1000,\"request_stats\":true}"; String res = Executor.newInstance() .execute(Request.Put("http://127.0.0.1:22999/api/settings")) .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method=HttpMethod.Put, RequestUri=new Uri("http://127.0.0.1:22999/api/settings"), Content=new StringContent(JsonConvert.SerializeObject( new { zone="ZONE", www_whitelist_ips=[], whitelist_ips=[], logs=1000, request_stats=true } ), Encoding.UTF8, "application/json" ) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print( 'If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests' ) import requests data = { 'zone': 'ZONE', 'www_whitelist_ips':[], 'whitelist_ips':[], 'logs':1000, 'request_stats':True } r = requests.put('http://127.0.0.1:22999/api/settings', data=json.dumps(data)) print(r.content) ``` # 升级代理管理器 Source: https://docs.brightdata.com/cn/api-reference/proxy-manager/upgrade_proxy_manager **API 端点:** `POST` `/api/upgrade` ```sh Shell curl -X POST "http://127.0.0.1:22999/api/upgrade" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'http://127.0.0.1:22999/api/upgrade' }).then(function(data){ console.log(data);}, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .execute(Request.Post("http://127.0.0.1:22999/api/upgrade")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("http://127.0.0.1:22999/api/upgrade") }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests r = requests.post('http://127.0.0.1:22999/api/upgrade') print(r.content) ``` # 请求和响应 C-Tag Source: https://docs.brightdata.com/cn/api-reference/proxy/c-tag 处理大量请求时,跟踪单个响应可能会很困难。C-Tag 是一种简单但功能强大的解决方案,可提升请求跟踪效率。 使用 C-Tag,用户可以在请求中添加唯一的 `c_tag` 标志。作为回应,企业会在标头中回传相同的标签。 这种无缝交换可确保每个响应都与相应的请求绑定,从而消除混乱并简化数据管理。 在请求中添加任何 C-Tag 文本,可在响应标头中获得相同的字符串: ```sh Sample Request curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--c_tag-: ``` ```js Output headers x-brd-c_tag: ``` # 运营商专属代理对等 IP(仅限移动设备) Source: https://docs.brightdata.com/cn/api-reference/proxy/carrier_specific_proxy_peer_ip_mobile_only_ * 您可以选择使用此列表中的特定运营商:a1、\[aircel] (%20)、airtel、att、celcom、chinamobile、claro、cox、digi、dt、docomo、dac、etisalat、idea、kyivstar、meo、mtnza、mts、optus、orange、qwest,reliance\_jio、robi、sprint、telefonica、telstra、time、vimpelcom、vimpelcom、vidacomza、vidafone、vivo、vivo、zain、vivabo、telenormyanmar、kcelljsc、swisscom、singtel、asiacell、windit、cellc、ooredoo、drei、umobile、callleon、proximus、2、mobitel、o2、bouygues、free、sfr、digicel * 运营商德国电信示例: ```sh Carrier: Deutsche Telekom brd-customer--zone--carrier-dt ``` ```sh Carrier: Sprint brd-customer--zone--carrier-sprint ``` # 配置 DNS 解析 Source: https://docs.brightdata.com/cn/api-reference/proxy/configuring_dns_resolution 可以指示 Bright Data 代理网络使用远程或本地 DNS 解析。 如果您的目标有两个以上的 IP(取决于进行 DNS 查询的位置),并且想要定向目标站点的特定 IP,这就会很有用。 对 Bright Data 的所有请求都以与超级代理 (brd.superproxy.io) 的连接开始。然后,超级代理将请求发送到不同国家的本地服务器,如果需要,这些服务器可以将请求发送给固定代理(“对等方”)。 使用本地DNS 解析时,域名由 Bright Data 超级代理网络服务器解析和缓存,经过解析的 IP 用于代理在实际国家/地区的真实请求。 当使用远程发出请求时,它们最初由超级代理 (brd.superproxy.io) 接收。超级代理在本地进行初步的 DNS 解析,只是为了检查域是否有效。 如果域有效,则请求将发送到距离代理所在国家/地区最近的 Bright Data 服务器。 * 如果 Bright Data 服务器与代理位于同一个国家/地区,则在服务器上执行 DNS,经过解析的 IP 用于实际请求。 此 DNS 解析通常比代理的 DNS 解析更快。 * 如果 Bright Data 服务器位于不同的国家/地区,则在实际代理上执行 DNS。 ```sh DNS (Local) curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--dns-local: ``` ```sh DNS (Remote) curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--dns-remote: ``` 使用远程 DNS 解析时,由于本地 DNS 解析延迟,您可能会发现一些性能下降。 # 地理位置定位 Source: https://docs.brightdata.com/cn/api-reference/proxy/geolocation-targeting 地理位置定位允许您根据“国家/地区”、“城市”、“州”、“ASN”或“邮政编码”来定位具体的位置 ## 国家/地区定位 您可以通过在用户名中添加国家/地区代码来定位特定国家/地区,例如 `username-country-country_code`。仅使用两个小写字母的国家/地区代码。 ```sh Shell curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--country-us: ``` ### EU Specific Targeting 网络解锁器 only 网络解锁器 allows random peer selection specifically from **EU-member countries** via the use of the `-country-eu` flag. When needing to target EU peers, this will improve success rates over targeting from just one EU country, while still keeping the peer within the EU. ## 城市定位 Residential proxies only. For DC & ISP networks you need to configure your zone and select the cities you would like to have proxies in. Using this parameter without pre-selection in DC & ISP is not permitted 您可以通过在用户名中添加城市名称来定位某个国家/地区的特定城市,例如 `username-country-country_code-city-city_code`。city\_code 必须是小写且不含空格,例如 `-city-sanfrancisco` ```sh Shell curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--country-us-city-sanfrancisco: ``` ## 州定位 Residential proxies only 对于相关郡县(例如美国和澳大利亚),您可以通过添加 `username-country-country_code-state-state_code` 来定位一个国家/地区的某个州。仅使用两个小写字母的州代码。 ```sh Country: USA curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--country-us-state-ny: ``` ```sh Country: Australia curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--country-au-state-wa: ``` ## ASN 定位 Residential proxies only 您可以从 ASN 列表中定位特定的 ASN: [http://bgp.potaroo.net/cidr/autnums.html](http://bgp.potaroo.net/cidr/autnums.html) 选择 ASN 国家/地区的方法如下:-asn-ASN\_NUMBER ```sh Shell curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--asn-56386: ``` ## 邮政编码定位 Residential proxies only 您可以通过特定区域的邮政编码来定位代理对等方。 以下是定位来自美国孟菲斯且邮政编码为12345的 IP 的示例: ```sh Shell curl "https://example.com" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--city-memphis-zip-12345: ``` # 获取每个区域可用的专用 IP Source: https://docs.brightdata.com/cn/api-reference/proxy/get_dedicated_ips_per_zone **API 端点**: `GET` `/api/zone/route_vips` **获取每个区域的所有可用专用 IP** 用于获取专用住宅或移动代理区域内 [gIP](/cn/proxy-networks/residential/configure-your-proxy#ip-groups-gips) 列表的 API 端点是: `GET /api/zone/route_vips` ```sh Shell curl -X GET "https://api.brightdata.com/zone/route_vips?zone=" -H "Authorization: Bearer " ``` ```js NodeJS var request = require('request'); var options = { 'method': 'GET', 'url': 'https://api.brightdata.com/zone/route_vips?zone=', 'headers': { 'Authorization': 'Bearer ' } }; request(options, function (error, response) { if (error) throw new Error(error); console.log(response.body); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .addHeader("Authorization", "Bearer ") .execute(Request.Get("https://api.brightdata.com/zone/route_vips?zone=")) .returnContent().asString(); System.out.println(res); }} ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("https://api.brightdata.com/zone/route_vips?zone=ZONE"), Headers = { {"Authorization", "Bearer API_KEY"} } }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python import requests url = "https://api.brightdata.com/zone/route_vips?zone=" payload={} headers = { 'Authorization': 'Bearer ' } response = requests.request("GET", url, headers=headers, data=payload) print(response.text) ``` ```json Sample Response [ "186XXXXXXXXXXX01", "186XXXXXXXXXXX00" ] ``` # 确保会话期间始终使用相同的节点 Source: https://docs.brightdata.com/cn/api-reference/proxy/keep_same_peer_in_session 您可以通过 API 指定,即使当前会话使用的节点离线,也不重新分配新的节点。 如果强制启用此请求参数,当会话使用的节点离线时,您的请求将收到 502 错误代码,同时返回以下错误消息: ```sh 错误消息 502 Proxy Error: server_error Failed to establish connection with peer ``` 对于由真实电脑和移动设备组成的住宅和移动网络代理区域,此工作流程非常有用。 要使用此功能,只需在会话名称后添加 -`const`,例如 `-session-mystring12345-const` ```sh Shell curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--session-mystring12345-const: ``` # 操作系统定位 Source: https://docs.brightdata.com/cn/api-reference/proxy/os_targeting Bright Data允许针对以下操作系统进行抓取: ```sh Windows curl "https://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--os-windows: ``` ```sh macOS curl "https://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--os-osx: ``` ```sh Android curl "https://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--os-android: ``` # 代理 REST API 测试平台 Source: https://docs.brightdata.com/cn/api-reference/proxy/proxy_api_post_playground proxy-rest-api POST /request 使用 REST API 访问代理网络 查看如何创建 API 令牌的说明:[API 令牌创建](/cn/general/account/api-token) # 请求错误处理 Source: https://docs.brightdata.com/cn/api-reference/proxy/request_error_handling | 值 | 描述 | | ---------- | -------------------------------------- | | `pass_dyn` | 如果请求无法通过代理对等节点传递,系统将自动通过超级代理传递请求。 | | `block` | 如果请求无法通过代理对等节点传递,则阻塞该请求,并且不通过超级代理进行发送。 | ```sh pass_dyn curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--route_err-pass_dyn: ``` ```sh block curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--route_err-block: ``` # IP Rotation in shared proxies pool Source: https://docs.brightdata.com/cn/api-reference/proxy/rotate_ips How does proxy IP rotation works, and how can you configure or control it. ## 代理 IP rotation by default By default, Bright Data assigns a random proxy from the available proxies in the zone, considering your configuration. For example: if you chose to target a specific country, for example Canada, using the `-country=ca` parameter, we will randomly assign a proxy in Canada out of all the proxies located in Canada that we have in the pool. The next request, will be directed to a different proxy, and so on, selected randomly from our pool. Executing requests in high rate or by multiple calling agents may result in reuse of the same proxy IPs. ## Forcing rotation with `session` parameter In order to force electing a different proxy, use the `-session` parameter with different value for every request. This will assure consequent requests will have different IPs. ## Using the same proxy for multiple requests: `session` To keep the same proxy IP address for multiple requests, use the `-session` parameter with a session identified you create and control. Each request, carrying the same `-session` value will be forwarded to the **same proxy IP**. The `-session` parameter value is generated by you and controlled by you, yet its format should include alphanumeric characters only. Using special characters like `-` or `*` will result in errors. If you work with native proxy access (within code libraries or 3rd party tools which take proxy ip:port and credentials) then you embed the session parameter in the user name parameter as shown below for session identified `mystring12345`. If you are using our proxy REST API, then the `session` parameter in request `body` shall carry the session identified value. ```sh Shell curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--session-mystring12345: ``` ## Losing session context The following describes scenarios where session context can be lost, and consequent requests will be executed by **different** proxies. ### Session Idle Time Session idle time is 5 minutes. In case there are more than 5 minutes idle time between two consequent requests, the second request, although carrying the same `session` parameter as the first, will use a randomly selected proxy from the pool. ### Session origin must be from same global region In order to keep the same session, you must ensure your calling process issuing the request is not switching between global regions. We recognize 3 global regions: 1. AMER: North, central and south America 2. EMEA: Europe, Middle East and Africa 3. APAC: Asia and Pacific Example: If you issue 2 consequent requests, one from **your company** server in Canada (in AMER region) and one from **your company server** in France, even if both requests will carry the same `session` parameter, they will be executed by different proxy peers. ### Other parameters must be identical In case 2 consequent requests carry different country, city or any other parameter: session context will be lost , even if they carry the same `session` parameter. ### Error on lost session By default, is session is lost or reset, Bright Data will randomly assign a proxy to your request. If you wish to avoid this default assignment, and error instead, use the `-const` parameter in your request. If the same peer is not available from any reason to 2 consequent requests, the second request will result in `HTTP Error 502` ```sh Shell curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--session-mystring12345-const: ``` # 选择特定的 gIP Source: https://docs.brightdata.com/cn/api-reference/proxy/select_a_specific_gip 该选项仅适用于分配了多个 IP 的住宅区域或移动区域。要将分配给您的区域的特定 `gIP` 作为目标,请使用 `-gip-gip_name` 请求参数。 ```sh curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--gip-us_7922_fl_hollywood_0: ``` # 选择特定的 IP Source: https://docs.brightdata.com/cn/api-reference/proxy/select_a_specific_ip 该选项仅适用于分配了多个 IP 的区域。 要将分配给您的区域的特定 IP 作为目标,请使用 `-ip-ip_address` 请求参数。 ```sh Shell curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--ip-1.1.1.1.1: ``` # 在特定国家/地区选择超级代理 Source: https://docs.brightdata.com/cn/api-reference/proxy/select_super_proxy_in_specific_country {/*tags:nosophie*/} 重要提示:本文仅适用于选择超级代理,而很少需要对超级代理执行相关操作。要选择特定的标准代理(例如,住宅代理、数据中心代理或 ISP 代理),请执行以下操作: * 在共享代理区域,您可以在代理用户名中添加 `-country` 标记(以及 `-city` 标记),如下所示:`username-country-country_code-city-city_code`。国家/地区代码是标准的 2 个字符代码。 `city_code` 必须是小写且没有空格,例如 `-city-sanfrancisco`。 * 在专用代理区域,您可以按照以下步骤在区域配置页面中选择国家/地区和城市: 1. 添加一个新代理区域,并将 IP 类型设置为“专用”。 2. 选择您要在此区域中定位的域名。 3. 选择要分配给该区域的专用代理数量。 4. 选择您要定位的国家/地区。 5. 可选项——选择您想要的国家/地区后,在城市分辨率中点击您想要定位的每个国家/地区旁边的“添加城市”。 6. 为每个国家/地区添加所选城市并保存区域。 7. 您的新专用代理区域将在您选择的国家/地区和城市之间平均分配您要求的 grproxies 数量。 如果您确实需要我们的负载均衡器在特定国家/地区提供超级代理: 选择超级代理国家/地区的方法如下:`servercountry-{country}.brd.superproxy.io`。 ```sh Shell curl "https://example.com" --proxy servercountry-gb.brd.superproxy.io:33335 --proxy-user brd-customer--zone-: ``` # 直接从超级代理发送请求 Source: https://docs.brightdata.com/cn/api-reference/proxy/send_requests_directly_from_super_proxy 您可以选择直接从超级代理而不是对等 IP 执行请求。 在这种情况下,通过在请求授权字符串中添加 -direct,请求的 IP 将是超级代理的 IP。 ```sh Shell curl "http://target.site" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone--direct: ``` # Unlocker API & SERP REST API 游乐场 Source: https://docs.brightdata.com/cn/api-reference/rest-api/proxy-rest-api-playground unlocker-rest-api POST /request # 所有分类数据 Source: https://docs.brightdata.com/cn/api-reference/scraping-shield-api/all-classification-data ## 查询参数 请求数据的起始时间范围 示例: `from=2018-07-01T00:00:00` 请求数据的结束时间范围 示例: `to=2018-07-02T00:00:00` 请求来源所在的国家 示例: `cn=uk` 对等 IP 所在的国家 示例: `peer_cn=us` 仅返回特定的分类 示例: `categories=ads` ```sh Request curl "https://api.brightdata.com/shield/class" -H "Authorization: " ``` ```json Response [ { "class":"Shopping", "req":1139485, "bw":61138544246 }, ... ] ``` # 按分类数据划分的域 Source: https://docs.brightdata.com/cn/api-reference/scraping-shield-api/domains-by-classification-data 返回所选分类的每个域的使用数据 ## 查询参数 请求数据的起始时间范围 示例: `from=2018-07-01T00:00:00` 请求数据的结束时间范围 示例: `to=2018-07-02T00:00:00` 请求来源所在的国家/地区 示例: `cn=uk` 对等 IP 所在的国家/地区 示例: `peer_cn=us` 仅返回特定的分类 示例: `categories=ads` 按 bw/requests 排序结果 * `order_by=bw` * `order_by=req` 按大小升序/降序排列结果 * `order=asc` * `order=desc` ```sh Request curl "https://api.brightdata.com/shield/domains_by_class?category=Ads" -H "Authorization: " ``` ```json Response [ { "domain":"static.addtoany.com", "req":16370, "bw":200590537 }, ... ] ``` # 按分类数据排列的示例 Source: https://docs.brightdata.com/cn/api-reference/scraping-shield-api/samples-by-classification-data 返回所选分类的示例请求数据 ## 查询参数 请求数据的起始时间范围 示例: `from=2018-07-01T00:00:00` 请求数据的结束时间范围 示例: `to=2018-07-02T00:00:00` 请求来源所在的国家/地区 示例: `cn=uk` 对等 IP 所在的国家/地区 示例: `peer_cn=us` 仅返回特定的分类 示例: `categories=ads` 仅返回特定的域名 示例: `host=example.com` ```sh Request curl "https://api.brightdata.com/shield/samples?category=Ads" -H "Authorization: " ``` ```json Response [ { "timestamp":"2024-02-07T11:35:45.197Z", "ip":"13.229.30.41", "zone":"zone_name", "bw":82345, "time":62594 }, ... ] ``` # 按分类汇总的区域数据 Source: https://docs.brightdata.com/cn/api-reference/scraping-shield-api/zones-by-classification-data 根据所选分类,返回每个区域的用量数据 ## 查询参数 请求数据的起始时间范围 示例: `from=2018-07-01T00:00:00` 请求数据的结束时间范围 示例: `to=2018-07-02T00:00:00` 请求来源所在的国家/地区 示例: `cn=uk` 对等 IP 所在的国家/地区 示例: `peer_cn=us` 仅返回特定分类的数据 示例: `categories=ads` 按 bw/requests 排序查询结果 * `order_by=bw` * `order_by=req` 按大小升序/降序排列结果 * `order=asc` * `order=desc` ```sh Request curl "https://api.brightdata.com/shield/zones_by_class?category=Ads" -H "Authorization: " ``` ```json Response [ { "zone":"v__dca", "req":5959860, "bw":390784389795 }, ... ] ``` # 发送您的第一个请求 Source: https://docs.brightdata.com/cn/api-reference/unlocker/sending_first_request 下面是一个使用 cURL 的最简单 Unlocker API 请求示例,该请求会返回一个 JSON: ```sh curl "https://geo.brdtest.com/welcome.txt" --proxy brd.superproxy.io:33335 --proxy-user brd-customer--zone-: ``` 您可以在代理区域的“访问参数”选项卡中找到您的 API 凭据,包括用户名(Customer\_ID)、区域名称和密码。 | | | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | | `brd.superproxy.io` | 我们的负载均衡器地址,它将为您的请求找到最快的超级代理 | | [`33335`](https://docs.brightdata.com/cn/general/faqs#which-port-shall-i-use-22225-or-33335) | 我们的超级代理的基础设施端口,用于接收您的请求 | | `-user brd-customer--zone-`  | 用户名验证。 它以最基本的形式定义了您的用户名以及您将在请求中使用哪个区域。  | | ``  | 区域密码。 所有区域都有用于身份验证的密码 | | [https://geo.brdtest.com/welcome.txt](https://geo.brdtest.com/welcome.txt) | 替换为您的目标域名。 这只是一个用于测试的服务器占位符。 | 有关所有 API 用例、集成和偏好的深入互动展示,请参阅我们的 API 示例页面: [https://brightdata.com/cp/zones/proxy\_examples](https://brightdata.com/cp/zones/proxy_examples) 您需要注册(免费)并登录 Bright Data 控制面板,才能访问此 API 工具。如果您添加了支付方式,您甚至会收到 5 美元的积分,以便开始使用! # 交付快照 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/delivery-apis/deliver-snapshot dca-api POST /datasets/v3/deliver/{snapshot_id} Deliver the snapshot content to the specified storage # 下载快照 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/delivery-apis/download-snapshot dca-api GET /datasets/v3/snapshot/{snapshot_id} Download the snapshot content 如果需要,结果可以分成多个部分: 1. 通过指定 `batch_size` 和 `part=1` 来下载第一部分。 2. 使用 [此端点](/scraping-automation/web-data-apis/web-scraper-api/management-apis#calculate-the-number-of-parts-for-delivery) 检查已创建的部分数量。 3. 通过更改 `part` 参数下载剩余部分。(`batch_size`、`format` 和 `compress` 在每次调用时必须保持不变) # 取消快照 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/management-apis/cancel-snapshot post /datasets/v3/snapshot/{snapshot_id}/cancel Cancel the running collection # 获取快照交付部分 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/management-apis/get-snapshot-delivery-parts get /datasets/v3/snapshot/{snapshot_id}/parts When requesting a delivery in batches (see available delivery APIs) use this endpoint to check how many parts were created. format, compress and batch_size should exactly match what was sent to the delivery/download API calls. # 获取快照输入 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/management-apis/get-snapshot-input get /datasets/v3/snapshot/{snapshot_id}/input Get the inputs that were used to trigger the collection # 获取快照 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/management-apis/get-snapshots get /datasets/v3/snapshots Get a list of triggered collections, the list contains only snapshots created for a specific dataset # 监控交付 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/management-apis/monitor-delivery get /datasets/v3/delivery/{delivery_id} The call returns the delivery status # 监控进度 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/management-apis/monitor-progress get /datasets/v3/progress/{snapshot_id} Get the status of a specific collection # Facebook API爬取工具 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/social-media-apis/facebook ## 概览 Facebook API套件提供多种类型的API,每种API都针对Facebook数据收集的特定需求进行了优化。以下是这些API的概述,包括它们如何连接和交互的方式: 此API允许用户根据单个输入URL(例如Facebook页面、群组或个人资料URL)收集多个帖子。
*   **发现功能**: *   需要Facebook页面、群组或个人资料的直接URL。
*   **重要字段**: *   `post_url`、`content`、`hashtags`、`likes`
此API允许用户使用帖子URL收集多个评论。
*   **发现功能**: *   `N/A`
*   **重要字段**: *   `commenter_url`、`comment_text`、`num_likes`、`num_replies`
由于Facebook数据需要登录,因此无法跨数据集进行发现。 该API套件旨在为用户提供灵活性,使他们能够输入特定URL,以批量或精确筛选的方式收集详细的帖子和评论数据。 ## 帖子API 帖子API允许用户从Facebook个人资料、群组和页面收集详细的帖子数据。API提供有关帖子、页面/个人资料详情以及附件和媒体的全面数据点。 ### 通过个人资料URL收集 此API允许用户通过提供个人资料URL收集Facebook帖子数据。 **输入参数** Facebook个人资料URL。 要收集的最近帖子数量。如果省略,则没有限制。 需要从结果中排除的帖子ID数组。 以MM-DD-YYYY格式指定的起始日期(应早于end\_date)。 以MM-DD-YYYY格式指定的结束日期(应晚于start\_date)。 **输出结构** 包括全面的数据点: * **帖子详情**: `post_id`、`content`、`hashtags`、`date_posted`、`num_comments`、`num_likes`等。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lkaxegm826bjpoo9m5?id=all\&tab=overview)。 * **页面/个人资料详情**: `page_name`、`page_category`、`page_followers`、`profile_handle`。 > 我们仅提供有关个人资料的有限数据点。 * **附件和媒体**: `attachments` 和 `post_image`(仅链接,不提供文件本身)、`video_view_count`。 ### 通过群组URL收集 此API允许用户通过提供群组URL收集详细的Facebook帖子。 **输入参数** Facebook群组URL。 要收集的帖子数量。如果省略,则没有限制。 需要从结果中排除的帖子ID数组。 以MM-DD-YYYY格式指定的起始日期(应早于end\_date)。 以MM-DD-YYYY格式指定的结束日期(应晚于start\_date)。 **输出结构** 包括全面的数据点: * **帖子详情**: `post_id`、`user_url`、`user_username`、`content`、`date_posted`、`hashtags`、`num_comments`、`num_shares`、`num_likes_type`等。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lz11l67o2cb3r0lkj3?id=all\&tab=overview)。 * **群组详情**: `group_name`、`group_id`、`group_url`、`group_intro`、`group_category`等。 * **用户详情**: `user_is_verified`、`profile_handle`等。 * **附件和外部链接**: `attachments`(仅链接,不提供文件本身)、`original_post_url`、`other_posts_url`、`post_external_link`等。 ### 通过帖子URL收集 此API允许用户使用帖子URL收集详细的Facebook帖子数据。 **输入参数** Facebook帖子URL。 **输出结构** 包括全面的数据点: * **帖子详情**: `post_id`、`content`、`hashtags`、`date_posted`、`num_comments`、`num_likes`等。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lyclm1571iy3mv57zw?id=all\&tab=overview)。 * **页面/个人资料详情**: `page_name`、`page_category`、`page_followers`、`profile_handle`。 > 我们仅提供有关个人资料的有限数据点。 * **附件和媒体**: `attachments` 和 `post_image`(仅链接,不提供文件本身)、`video_view_count`。 ## 评论 API 评论 API 允许用户从 Facebook 帖子中收集详细的评论数据。该 API 提供关于评论的全面数据点,包括评论详情、用户信息、帖子元数据、附件和媒体。 ### 通过帖子 URL 采集 该 API 允许用户通过提供帖子 URL,从 Facebook 帖子中收集详细的评论数据。 **输入参数** Facebook 帖子 URL。 要采集的评论数量。如果省略,则没有限制。 要排除的评论 ID 数组。 用于筛选评论的起始日期,格式为 MM-DD-YYYY(应早于 end\_date)。 用于筛选评论的结束日期,格式为 MM-DD-YYYY(应晚于 start\_date)。 **输出结构** 包含全面的数据点: * **评论详情**: `comment_id`、`comment_text`、`num_likes`、回复、`num_replies` 等。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lkay758p1eanlolqw8?id=all\&tab=overview)。 * **用户信息**: `user_name`、`user_id`、`user_url` > 我们仅提供有限的用户信息数据点。 * **帖子元数据**: `post_id`、`post_url`。 * **附件和媒体**: `attached_files`(仅链接,不包含文件本身)、`video_length`。 ## Reels API Reels API 允许用户从公共 Facebook 个人资料中收集详细的 Reels 数据。该 API 提供关于 Reels 的全面数据点,包括帖子详情、页面/个人资料信息、附件和媒体。 ### 通过个人资料 URL 采集 该 API 允许用户通过提供个人资料 URL,从公共 Facebook 个人资料中收集详细的 Reels 数据。 **输入参数** Facebook 个人资料 URL。 要采集的 Reels 数量(默认最多 1600)。 要排除的 Reels ID 数组。 用于筛选 Reels 的起始日期。 用于筛选 Reels 的结束日期。 **输出结构** 包含全面的数据点: * **Reels 详情**: `post_id`、`content`、`hashtags`、`date_posted`、`num_comments`、`num_likes`、`audio`(音轨详情)、`video_view_count` 等。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lyclm3ey2q6rww027t?id=all\&tab=overview)。 * **页面/个人资料信息**: `page_name`、`page_category`、`page_followers`、`profile_handle`。 > 我们仅提供有限的个人资料信息数据点。 * **附件和媒体**: `external_link`(仅链接,不包含文件本身)。 # Instagram API 抓取工具 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/social-media-apis/instagram ## 概述 Instagram API 套件提供多种类型的 API,每种 API 都针对特定的 Instagram 数据采集需求设计。以下是这些 API 的连接和交互概览,基于可用功能: 此 API 允许用户基于单一输入(个人资料 URL)收集个人资料详细信息。
*   **发现功能**: *   `N/A`
*   **关键字段**: *   `followers`、`post_count`、`post_hashtags`、`profile_name`。
此 API 允许用户基于单一输入 URL(如 Instagram Reels URL、搜索 URL 或个人资料 URL)收集多个帖子。
*   **发现功能**: *   - Instagram Reels 直接 URL *   - 搜索结果的直接 URL *   - 个人资料的直接 URL
*   **关键字段**: *   `url`、`followers`、`hashtags`、`engagement_score_view`。
此 API 允许用户使用帖子 URL 收集多个评论。
*   **发现功能**: *   `N/A`
*   **关键字段**: *   `comment_user`、`comment`、`likes_number`、`replies_number`。
该 API 套件旨在提供灵活的数据采集能力,用户可以输入特定的 URL 以批量或精准筛选的方式获取详细的帖子和评论数据。 ## Profiles API ### 通过 URL 采集 此 API 允许用户通过提供 Instagram 个人资料 URL 来收集详细的 Instagram 个人资料数据。它提供了 Instagram 个人资料的全面概览,包括商业和互动信息、帖子以及用户详细信息。 **输入参数** Instagram 个人资料 URL。 **输出结构**\ 包含全面的数据点: * **页面/个人资料详情**:\ `account`、`id`、`followers`、`posts_count`、`is_business_account`、`is_professional_account`、`is_verified`、`avg_engagement`、`profile_name`、`profile_url`、`profile_image_link` 等。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_l1vikfch901nx3by4?id=all\&tab=overview)。 *** ## Posts API ### 通过 URL 采集 此 API 允许用户通过提供帖子 URL 来收集 Instagram 帖子的详细数据。 **输入参数** Instagram 帖子 URL。 **输出结构**\ 包含全面的数据点: * **帖子详情**:\ `post_id`、`description`、`hashtags`、`date_posted`、`num_comments`、`likes`、`content_type`、`video_view_count`、`video_play_count` 等。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lk5ns7kz21pck8jpis?id=all\&tab=overview)。 * **页面/个人资料详情**:\ `user_posted`、`followers`、`posts_count`、`profile_image_link`、`is_verified`、`profile_url`。 > 仅提供有限的个人资料数据点。 * **附件和媒体**:\ `photos`、`videos`、`thumbnail`、`display_url`(仅链接,非文件)、`audio`。 *** ### 通过 URL 发现 此 API 允许用户通过提供 Instagram 个人资料 URL 并指定额外参数,发现该公共个人资料的最新帖子。 **输入参数** Instagram 个人资料 URL。 要采集的最新帖子数量。如果省略,则没有限制。 要从结果中排除的帖子 ID 数组。 过滤帖子时的起始日期,格式为 MM-DD-YYYY(应早于 `end_date`)。 过滤帖子时的结束日期,格式为 MM-DD-YYYY(应晚于 `start_date`)。 指定要采集的帖子类型(如 `post`、`reel`)。 **输出结构**\ 包含全面的数据点: * **帖子详情**:\ `post_id`、`description`、`hashtags`、`date_posted`、`num_comments`、`likes`、`video_view_count`、`video_play_count` 等。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lk5ns7kz21pck8jpis/url?id=all\&tab=overview)。 * **页面/个人资料详情**:\ `user_posted`、`followers`、`posts_count`、`profile_image_link`、`is_verified`、`profile_url`、`is_paid_partnership`、`partnership_details`、`user_posted_id`。 * **附件和媒体**:\ `photos`、`videos`、`thumbnail`、`audio`、`display_url`、`content_type`、`product_type`、`coauthor_producers`、`tagged_users`。 此 API 允许进行过滤、排除特定帖子,并按类型(普通帖子或短视频)在指定时间范围内采集帖子。它提供详细的帖子和个人资料信息,适用于数据采集和分析。 ## 评论 API ### 通过 URL 采集 此 API 允许用户通过提供 Instagram 帖子 URL 来收集特定帖子的最新评论。 此 API 仅获取最近 10 条评论及相关元数据。 **输入参数** Instagram 帖子 URL。 **输出结构**\ 包含全面的数据点: * **评论详情**:\ `comment_id`、`comment_user`、`comment_user_url`、`comment_date`、`comment`、`likes_number`、`replies_number`、`replies`、`hashtag_comment`、`tagged_users_in_comment` 等。 > 所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_ltppn085pokosxh13?id=all\&tab=overview)。 * **用户详情**:\ `user_name`、`user_id`、`user_url` > 我们仅提供有限的用户资料数据点。 * **帖子元数据**:\ `post_url`、`post_user`、`post_id`。 ## Reels API(短视频 API) ### 通过 URL 采集 此 API 允许用户通过提供 Reels 短视频的 URL,从公开的 Instagram 个人资料中收集详细数据。 **输入参数** Instagram Reels 短视频 URL。 **输出结构**\ 包含全面的数据点: * **短视频详情**:\ `post_id`、`description`、`hashtags`、`date_posted`、`tagged_users`、`num_comments`、`likes`、`views`、`video_play_count`、`length` 等。 > 所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lyclm20il4r5helnj?id=all\&tab=overview)。 * **页面/个人资料详情**:\ `user_posted`、`followers`、`posts_count`、`profile_image_link`、`is_verified`、`profile_url`。 > 我们仅提供有限的个人资料数据点。 * **附件和媒体**:\ `video_url`、`thumbnail`、`audio_url`。 ### 通过 URL 发现 此 API 允许用户通过个人资料 URL 或直接搜索 URL 发现 Instagram Reels 短视频。 **输入参数** Instagram 个人资料 URL 或直接搜索 URL。 需要收集的最新短视频数量。如果省略,则没有限制。 要排除的短视频 ID 数组。 过滤短视频的起始日期,格式为 MM-DD-YYYY。 过滤短视频的结束日期,格式为 MM-DD-YYYY(应晚于 `start_date`)。 **输出结构**\ 包含全面的数据点: * **短视频详情**:\ `post_id`、`description`、`hashtags`、`date_posted`、`num_comments`、`likes`、`views`、`video_play_count`、`top_comments`、`length`、`video_url`、`audio_url`、`content_id` 等。 > 所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lyclm20il4r5helnj/url?id=all\&tab=overview)。 * **个人资料详情**:\ `user_posted`、`followers`、`posts_count`、`following`。 * **附件和媒体**:\ `video_url`、`thumbnail`、`audio_url`(仅提供链接,不包含文件本身)。 此 API 提供有关 Instagram Reels 的详细信息,并支持按日期范围过滤、排除特定帖子以及限制收集的短视频数量。 # LinkedIn API Scrapers Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/social-media-apis/linkedin ## 概览 LinkedIn API 套件提供多种类型的 API,每种 API 旨在满足 LinkedIn 数据采集的特定需求。以下是这些 API 之间的连接和交互概述,基于可用功能: 该 API 允许用户根据一个输入:个人资料 URL 采集个人资料详情。
*   **发现功能**: *   根据姓名发现 LinkedIn 个人资料
*   **重要列、列名**: *   `name`,`country_code`,`current_company`,`about`。
该 API 允许用户根据一个输入 URL 采集多篇文章。
*   **发现功能**: *   - 直接 LinkedIn 个人资料 URL *   - 直接用户文章 URL *   - 直接公司 URL
*   **重要列、列名**: *   `url`,`title`,`hashtags`,`num_likes`。
该 API 允许用户根据一个输入 URL 采集职位信息。
*   **发现功能**: *   - 直接搜索 URL *   - 根据关键词发现
*   **重要列、列名**: *   `job_title`,`company_name`,`job_location`,`job_seniority_level`。
该 API 允许用户通过公司 URL 采集公司信息。
*   **发现功能**:N/A
*   **重要列、列名**: *   `name`,`country_code`,`followers`,`employees_in_linkedin`。
该 API 套件设计为提供灵活性,以满足定向数据采集需求。用户可以输入特定 URL 以获取详细的文章、个人资料、职位和公司数据,支持批量操作或精准筛选。 ## 个人资料 API ### 通过 URL 采集 此 API 允许用户通过提供 LinkedIn 个人资料 URL 来收集详细信息。 **输入参数**: LinkedIn 个人资料 URL。 **输出结构**:\ 包括全面的数据点: * **个人资料详情**:\ `linkedin_id`、`name`、`country_code`、`city`、`position`、`current_company`、`current_company_name`、`current_company_company_id`、`about` 等。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_l1viktl72bvl7bjuj0?id=all\&tab=overview)。 * **职业信息**:\ `experience`、`education`、`educations_details`、`certifications`、`languages`、`recommendations_count`、`recommendations`、`volunteer_experience`、`courses`、`publications`、`patents`、`projects`、`organizations` 和 `honors_and_awards`。 * **社交活动**:\ `posts`、`activity`、`followers`、`connections`、`people_also_viewed`。 * **媒体与身份识别**:\ `avatar`、`banner_image`、`url`、`input_url`、`linkedin_num_id`、`id`、`timestamp`。 此 API 提供丰富的数据点,可用于分析 LinkedIn 个人资料,包括职业经历、教育背景、推荐信息、社交活动,以及媒体和唯一身份标识信息。 ### 通过姓名发现 此 API 允许用户根据提供的姓和名查找 LinkedIn 个人资料。 **输入参数**: 个人的名字。 个人的姓氏。 **输出结构**:\ 包括全面的数据点: * **个人资料详情**:\ `linkedin_id`、`name`、`country_code`、`city`、`position`、`current_company`、`current_company_name`、`current_company_company_id`、`about` 等。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_l1viktl72bvl7bjuj0/name?id=all\&tab=overview)。 * **职业信息**:\ `experience`、`education`、`educations_details`、`certifications`、`languages`、`recommendations_count`、`recommendations`、`volunteer_experience`、`courses`、`publications`、`patents`、`projects`、`organizations` 和 `honors_and_awards`。 * **社交活动**:\ `posts`、`activity`、`followers`、`connections`、`people_also_viewed`。 * **媒体与身份识别**:\ `avatar`、`banner_image`、`url`、`input_url`、`linkedin_num_id`、`id`、`timestamp`。 此 API 可根据姓名高效检索 LinkedIn 个人资料,提供丰富的职业和社交信息,并附带相关的媒体资源。 ## 文章 API ### 通过 URL 收集 此 API 允许用户通过提供帖子 URL 来收集 Instagram 帖子的详细数据。 **输入参数**: LinkedIn 帖子的 URL。 **输出结构**:\ 包含全面的数据点: * **帖子详情**:\ `id`、`title`、`headline`、`post_text`、`post_text_html`、`date_posted`、`hashtags`、`embedded_links`、`images`、`videos`、`post_type`、`account_type` 等。 > 查看所有数据点,请 [点击这里](https://brightdata.com/cp/data_api/gd_lyy3tktm25m4avu764?id=all\&tab=overview)。 * **互动指标**:\ `num_likes`、`num_comments`、`top_visible_comments`。 * **用户详情**:\ `user_id`、`use_url`、`user_followers`、`user_posts`、`user_articles`。 * **相关内容**:\ `more_articles_by_user`、`more_relevant_posts`。 此 API 提供 LinkedIn 帖子的详细信息,包括文本、媒体附件、互动指标,以及额外的用户和相关内容数据,以便进行全面的帖子分析。 ### 通过 URL 发现 此 API 允许用户通过提供 LinkedIn 个人资料 URL 并可选择限制结果数量,来发现该用户撰写的文章。 **输入参数**: LinkedIn 个人资料 URL。 要检索的文章最大数量。如果省略,将收集所有可用的文章。 **输出结构**:\ 包含全面的数据点: * **文章详情**:\ `id`、`title`、`headline`、`post_text`、`post_text_html`、`date_posted`、`hashtags`、`embedded_links`、`images`、`videos`、`post_type`、`account_type` 等。 > 查看所有数据点,请 [点击这里](https://brightdata.com/cp/data_api/gd_lyy3tktm25m4avu764/url?id=all\&tab=overview)。 * **互动指标**:\ `num_likes`、`num_comments`、`top_visible_comments`。 * **用户详情**:\ `user_id`、`use_url`、`user_followers`、`user_posts`、`user_articles`。 * **相关内容**:\ `more_articles_by_user`、`more_relevant_posts`。 此 API 可获取 LinkedIn 用户撰写的文章的详细信息,包括文本、媒体、互动数据,以及相关的用户和内容数据,以便进行深入分析。 ### 通过个人资料 URL 发现 此 API 允许用户通过提供个人资料 URL 并应用可选的日期筛选器,来发现 LinkedIn 帖子。 **输入参数**: LinkedIn 个人资料 URL。 按发布时间筛选帖子的起始日期(YYYY-MM-DD 格式)。发布时间早于此日期的帖子将被排除。 按发布时间筛选帖子的结束日期(YYYY-MM-DD 格式)。发布时间晚于此日期的帖子将被排除。 **输出结构**:\ 包含全面的数据点: * **帖子详情**:\ `id`、`title`、`headline`、`post_text`、`post_text_html`、`date_posted`、`hashtags`、`embedded_links`、`images`、`videos`、`post_type`、`account_type` 等。 > 查看所有数据点,请 [点击这里](https://brightdata.com/cp/data_api/gd_lyy3tktm25m4avu764/company_url?id=all\&tab=overview)。 * **互动指标**:\ `num_likes`、`num_comments`、`top_visible_comments`。 * **用户详情**:\ `user_id`、`use_url`、`user_followers`、`user_posts`、`user_articles`。 * **相关内容**:\ `more_articles_by_user`、`more_relevant_posts`。 此 API 允许基于特定个人资料和时间范围精准发现 LinkedIn 帖子,提供详细的帖子数据、互动指标以及相关用户信息,以便进行深入分析。 ### 通过公司 URL 发现 此 API 允许用户使用公司 URL 发现公司发布的最新 LinkedIn 帖子,并通过可选的日期范围进行筛选。 **输入参数**: LinkedIn 公司资料 URL。 按发布时间筛选帖子的起始日期(YYYY-MM-DD 格式)。仅收集该日期或之后发布的帖子。 按发布时间筛选帖子的结束日期(YYYY-MM-DD 格式)。仅收集该日期或之前发布的帖子。 **输出结构**:\ 包含全面的数据点: * **帖子详情**:\ `post_id`、`title`、`post_text`、`date_posted`、`hashtags`、`embedded_links`、`images`、`videos`、`post_type`、`account_type` 等。 > 查看所有数据点,请 [点击这里](https://brightdata.com/cp/data_api/gd_lyy3tktm25m4avu764/company_url?id=all\&tab=overview)。 * **互动指标**:\ `num_likes`、`num_comments`、`top_visible_comments`。 * **公司详情**:\ `company_id`、`company_name`、`company_url`、`followers`。 * **相关内容**:\ `more_relevant_posts`、`additional_media_links`。 此 API 提供公司发布的帖子详细信息,帮助企业和分析师有效追踪动态、互动数据以及内容趋势。 ## 职位列表信息 API ### 通过 URL 收集 此 API 允许用户通过提供 LinkedIn 职位 URL 来收集详细的职位信息。 **输入参数**: LinkedIn 职位列表的 URL。 **输出结构**:\ 包含全面的数据点: * **职位详情**: `job_posting_id`、`job_title`、`job_summary`、`job_seniority_level`、`job_function`、`job_employment_type`、`job_industries`、`job_base_pay_range`、`base_salary`、`job_description_formatted` 等。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lpfll7v5hcqtkxl6l?id=all\&tab=overview)。 * **公司详情**: `company_name`、`company_id`、`company_url`、`company_logo`。 * **职位元数据**: `job_location`、`job_posted_time`、`job_posted_date`、`job_num_applicants`、`discovery_input`、`apply_link`、`country_code`、`title_id`。 * **职位发布者信息**: `job_poster`。 该 API 提供对 LinkedIn 职位列表的全面概览,包括职位详情、公司信息、申请人数统计和元数据,以增强职位分析和申请跟踪。 ### 通过关键词发现职位 此 API 允许用户根据关键词搜索、位置和其他筛选条件(如职位类型、经验水平等)发现 LinkedIn 职位。 **输入参数**: 目标职位所在的具体位置。 用于搜索职位的关键词或职位名称(例如 "产品经理")。使用引号可进行精确匹配。 以两位字母格式表示的国家代码(例如 `US`、`FR`)。 职位发布时间范围(例如 `past_week`、`past_month`)。 职位类型(例如 "全职"、"兼职")。 职位所需的经验水平(例如 "入门级"、"中高级")。 是否为远程职位(例如 "yes" 或 "no")。 用于筛选职位的公司名称。 设置为 `true` 时,排除不包含指定关键词的职位标题。 **输出结构**:\ 包含全面的数据点: * **职位详情**: `job_posting_id`、`job_title`、`job_summary`、`job_seniority_level`、`job_function`、`job_employment_type`、`job_industries`、`job_base_pay_range`、`job_description_formatted`、`base_salary` 等。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lpfll7v5hcqtkxl6l/keyword?id=all\&tab=overview)。 * **公司详情**: `company_name`、`company_id`、`company_url`、`company_logo`。 * **职位元数据**: `job_location`、`job_posted_time`、`job_posted_date`、`job_num_applicants`、`job_poster`、`apply_link`、`country_code`、`title_id`。 * **申请详情**: `job_poster`、`application_availability`、`discovery_input`。 此 API 允许用户根据特定关键词、位置、职位类型等筛选和发现 LinkedIn 职位,并提供详细的职位列表和公司信息,以便进一步分析。 ### 通过 URL 发现职位 此 API 允许用户从 LinkedIn 直接职位搜索 URL 发现职位列表,并根据特定 URL 查询提供职位详情、公司信息和职位元数据。 **输入参数**: 用于搜索职位的 LinkedIn 直接搜索 URL(例如按公司或职位名称)。 设置为 `true` 时,搜索将排除与指定关键词或条件不匹配的职位标题。 **输出结构**:\ 包含全面的数据点: * **职位详情**: `job_posting_id`、`job_title`、`job_summary`、`job_seniority_level`、`job_function`、`job_employment_type`、`job_industries`、`job_base_pay_range`、`job_description_formatted`、`base_salary` 等。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lpfll7v5hcqtkxl6l/url?id=all\&tab=overview)。 * **公司详情**: `company_name`、`company_id`、`company_url`、`company_logo`。 * **职位元数据**: `job_location`、`job_posted_time`、`job_posted_date`、`job_num_applicants`、`job_poster`、`apply_link`、`country_code`、`title_id`。 * **申请详情**: `job_poster`、`application_availability`、`discovery_input`。 此 API 允许用户从 LinkedIn 搜索 URL 获取详细的职位列表,提供职位机会、公司详情等信息,以实现高效的职位发现和申请跟踪。 ## 公司信息 API ### 通过 URL 获取 该 API 允许用户通过提供 LinkedIn 公司资料的 URL 来收集详细的公司信息。 **输入参数**: LinkedIn 公司资料的 URL。 **输出结构**:\ 包含全面的数据点: * **公司详情**:\ `id`、`name`、`about`、`slogan`、`description`、`specialties`、`organization_type`、`company_size`、`industries`、`founded` 等。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_l1vikfnt1wgvvqz95w?id=all\&tab=overview)。 * **商业信息**:\ `country_code`、`country_codes_array`、`locations`、`formatted_locations`、`headquarters`、`stock_info`、`funding`、`investors`、`crunchbase_url`、`get_directions_url`。 * **互动与网络**:\ `followers`、`employees`、`employees_in_linkedin`、`alumni`、`alumni_information`、`affiliated`、`similar`。 * **媒体与元数据**:\ `logo`、`image`、`url`、`updates`、`timestamp`。 该 API 提供 LinkedIn 公司资料的全面概览,包括业务详情、公司位置、校友网络、融资信息以及媒体资源。 # 社交媒体爬取 API 概览 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/social-media-apis/overview 社交媒体 API 套件提供广泛的数据收集工具,适用于多个平台。 #### 社交媒体 API 套件概述 社交媒体 API 套件旨在简化数据收集,同时提供对用户互动和内容传播的深入分析。用户可以参考详细指南,了解如何结合多个 API 以获取全面的数据。 #### 支持的平台 社交媒体 API 套件支持多个社交媒体平台,包括: 该 API 套件可用于分析用户资料、帖子及其互动情况(如评论和点赞)。以下是基于销售指南的 API 总览。 #### API 类型与数据连接 * **Profiles API(用户资料 API)**:收集用户或频道的资料数据,包括粉丝数、订阅数和内容统计信息。 * **Posts API(帖子 API)**:提供单个帖子或视频的数据,包括点赞数、观看数和标签信息。 * **Comments API(评论 API)**:分析用户与帖子的互动情况,例如评论的回复和点赞数据。 Facebook、YouTube、Instagram、TikTok 和 Pinterest 等平台均支持这些 API,从而实现一致且全面的社交数据收集。 #### 主要功能 1. **全面覆盖**:支持从 YouTube、TikTok 等视频平台到 Instagram、Facebook 等社交网络的广泛数据收集。 2. **最新数据**:每个 API 都支持按需数据获取,确保用户获取到最新的数据。 3. **可定制方案**:API 具有灵活性,允许用户自定义数据收集方式,并结合多个 API 以满足不同的需求。 4. **易于集成**:支持简单的 API 调用,方便集成到现有工作流中,降低开发者的学习成本。 5. **互联互通**:这些 API 可互相配合,用户可以利用一个 API 作为另一个 API 的输入,进行跨平台数据分析并获取全面的洞察信息。 > **示例:在 YouTube 上的数据分析** > > * 使用 **Profiles API** 获取特定频道的相关数据。 > * 通过 **Posts API** 获取该频道发布的视频数据。 > * 使用 **Comments API** 获取这些视频的评论数据。 这种互联方式适用于所有支持的平台,使用户能够分析用户资料、帖子及其相关互动,以进行深度数据研究。 # Pinterest API 抓取工具 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/social-media-apis/pinterest ## 概览 Pinterest API 套件提供多种 API,每种 API 旨在满足不同的 Pinterest 数据采集需求。以下是这些 API 之间的连接和交互方式概览,基于可用的功能: 该 API 允许用户根据单个输入(个人资料 URL)收集个人资料详细信息。
*   **发现功能**: *   通过关键词发现。
*   **重要字段**: *   `name`, `following_count`, `website`, `follower_count`。
该 API 允许用户基于单个输入收集多个帖子。
*   **发现功能**: *   - 通过个人资料 URL 发现。 *   - 通过关键词发现。
*   **重要字段**: *   `title`, `content`, `user_name`, `likes`。
## 个人资料 API ### 通过 URL 采集 该 API 允许用户使用提供的个人资料 URL 获取详细的 Pinterest 个人资料信息。 **输入参数**: Pinterest 个人资料 URL。 **输出结构**:\ 包含丰富的数据点: * **个人资料详情**:\ `url`, `profile_picture`, `name`, `nickname`, `website`, `bio`, `country_code`, `profile_id`。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lk0zv93c2m9qdph46z?id=all\&tab=overview)。 * **互动与指标**:\ `following_count`, `follower_count`, `boards_num`, `saved`。 * **附加信息**:\ `last_updated`, `posts_page_url`, `discovery_input`。 该 API 允许用户收集 Pinterest 个人资料的详细信息,包括用户统计数据、互动指标和个人资料详情。 *** ### 通过关键词发现 该 API 允许用户基于指定关键词发现 Pinterest 个人资料。 **输入参数**: 用于搜索个人资料的关键词。 **输出结构**:\ 包含丰富的数据点: * **个人资料详情**:\ `url`, `profile_picture`, `name`, `nickname`, `website`, `bio`, `country_code`, `profile_id`。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lk0zv93c2m9qdph46z/keyword?id=all\&tab=overview)。 * **互动与指标**:\ `following_count`, `follower_count`, `boards_num`, `saved`。 * **附加信息**:\ `last_updated`, `posts_page_url`, `discovery_input`。 该 API 使用户能够查找与特定关键词相关的 Pinterest 个人资料,提供用户统计数据、互动情况和个人资料详情。 ## 帖子 API ### 通过 URL 采集 该 API 允许用户使用提供的帖子 URL 收集特定 Pinterest 帖子的详细数据。 **输入参数**: Pinterest 帖子 URL。 **输出结构**:\ 包含丰富的数据点: * **帖子详情**:\ `url`, `post_id`, `title`, `content`, `date_posted`, `post_type`。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lk0sjs4d21kdr7cnlv?id=all\&tab=overview)。 * **用户详情**:\ `user_name`, `user_url`, `user_id`, `followers`。 * **帖子指标**:\ `likes`, `comments_num`, `comments`, `categories`。 * **媒体与附件**:\ `image_video_url`, `video_length`, `attached_files`。 * **标签与发现**:\ `hashtags`, `discovery_input`。 该 API 允许用户获取特定 Pinterest 帖子的详细信息,包括用户互动、帖子内容、媒体及其他相关信息。 *** ### 通过个人资料 URL 发现帖子 该 API 允许用户基于提供的个人资料 URL 获取特定 Pinterest 个人资料下的帖子。 **输入参数**: 要采集帖子的 Pinterest 个人资料 URL。 要采集的帖子数量。如果省略,则没有限制。 要排除的帖子 ID 数组。 过滤帖子起始日期,格式为 `MM-DD-YYYY`(应早于 `end_date`)。 过滤帖子结束日期,格式为 `MM-DD-YYYY`(应晚于 `start_date`)。 **输出结构**:\ 包含丰富的数据点: * **帖子详情**:\ `url`, `post_id`, `title`, `content`, `date_posted`, `user_name`, `user_url`, `user_id`, `followers`, `likes`, `categories`, `source`, `attached_files`, `image_video_url`, `video_length`, `hashtags`, `comments_num`, `comments`, `post_type`。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lk0sjs4d21kdr7cnlv/profile_url?tab=overview)。 * **互动与指标**:\ `followers`, `likes`, `comments_num`。 * **媒体与附件**:\ `image_video_url`, `video_length`, `attached_files`。 * **附加信息**:\ `discovery_input`。 该 API 允许用户从特定 Pinterest 个人资料中收集帖子,并支持按日期筛选、排除特定帖子,并获取包含媒体、评论和互动指标的详细帖子数据。 *** ### 通过关键词发现帖子 该 API 允许用户基于特定关键词发现 Pinterest 帖子,实现高效的内容发现。 **输入参数**: 用于搜索帖子的关键词,例如 "food" 或其他相关术语。 **输出结构**:\ 包含丰富的数据点: * **帖子详情**:\ `url`, `post_id`, `title`, `content`, `date_posted`, `user_name`, `user_url`, `user_id`, `followers`, `likes`, `categories`, `source`, `attached_files`, `image_video_url`, `video_length`, `hashtags`, `comments_num`, `comments`, `post_type`。 > 查看所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lk0zv93c2m9qdph46z/keyword?id=all\&tab=overview)。 * **互动与指标**:\ `followers`, `likes`, `comments_num`。 * **媒体与附件**:\ `image_video_url`, `video_length`, `attached_files`。 * **附加信息**:\ `discovery_input`。 该 API 允许用户基于特定关键词搜索 Pinterest 帖子,并提供详细的内容、互动、媒体及相关指标信息,以实现高效发现。 # Quora API 抓取器 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/social-media-apis/quora ## 概览 此 API 允许用户基于单个输入 URL 收集多个帖子。
*   **发现功能**: *   不适用
*   **重要字段**: *   `author_name`,`title`,`over_all_answers`,`shares`。
## Posts API ### 通过 URL 采集 此 API 允许用户使用提供的 Quora 帖子 URL,收集该帖子的详细信息。 **输入参数** Quora 帖子的 URL。 **输出结构**:\ 包含全面的数据点: * **帖子详情**:\ `url`,`post_id`,`title`,`post_date`,`originally_answered`,`over_all_answers`,`post_text`,`header`。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lvz1rbj81afv3m6n5y?id=all\&tab=overview)。 * **媒体 & 链接**:\ `pictures_urls`,`videos_urls`,`external_urls`。 * **互动 & 指标**:\ `upvotes`,`shares`,`views`,`author_content_views`。 * **作者详情**:\ `author_name`,`author_active_spaces`,`author_joined_date`,`author_about`,`author_education`。 * **热门评论**:\ `top_comments`。 此 API 提供对 Quora 帖子的详细见解,包括内容、媒体、作者信息和互动指标,助力高效的帖子分析和内容追踪。 # Reddit API 爬取器 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/social-media-apis/reddit ## 概览 Reddit API 套件提供多种类型的 API,每种 API 旨在满足不同的 Reddit 数据采集需求。以下是这些 API 之间的交互方式概览,基于可用的功能: 该 API 允许用户根据单个输入 URL 采集多个帖子。
*   **发现功能**: *   通过 Subreddit URL 发现。 *   通过关键词发现。
*   **重要字段**: *   `url`、`title`、`num_comments`、`description`
该 API 允许用户通过帖子 URL 采集多个评论。
*   **发现功能**: *   不适用
*   **重要字段**: *   `url`、`comment`、`community_name`、`num_replies`
## 帖子 API ### 通过 URL 采集 该 API 允许用户使用提供的 URL 检索特定 Reddit 帖子的详细数据。 **输入参数** 需要检索的 Reddit 帖子 URL。 **输出结构**:\ 包含全面的数据点: * **帖子详情**:\ `post_id`、`url`、`title`、`description`、`num_comments`、`date_posted`、`tag`、`related_posts`、`comments`、`photos`、`videos`。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lvz8ah06191smkebj4?id=all\&tab=overview)。 * **用户详情**:\ `user_posted`。 * **社区详情**:\ `community_name`、`community_url`、`community_description`、`community_members_num`、`community_rank`。 * **互动指标**:\ `num_upvotes`。 该 API 可高效获取 Reddit 帖子信息,包括用户互动数据、社区详情及多媒体内容,支持高级分析和内容跟踪。 ### 通过关键词发现 该 API 允许用户通过特定关键词发现 Reddit 帖子,并可选择按日期筛选和设置要检索的帖子数量。 **输入参数** 用于搜索相关帖子的关键词。 过滤帖子发布时间,以确保结果符合指定时间范围。 需要采集的帖子数量。如果未提供,则默认检索所有符合条件的帖子。 **输出结构**:\ 包含全面的数据点: * **帖子详情**:\ `post_id`、`url`、`user_posted`、`title`、`description`、`num_comments`、`date_posted`、`tag`、`related_posts`、`comments`。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lvz8ah06191smkebj4/keyword?id=all\&tab=overview)。 * **社区详情**:\ `community_name`、`community_url`、`community_description`、`community_members_num`、`community_rank`。 * **媒体详情**:\ `photos`、`videos`。 该 API 使用户能够高效发现并分析与特定关键词相关的 Reddit 帖子,提供关于社区讨论、帖子热度及相关媒体内容的宝贵洞察。 ### 通过 Subreddit URL 发现 该 API 允许用户使用 Subreddit URL 检索特定 Subreddit 中的帖子,并可选择基于特定标准对结果进行排序。 **输入参数** 需要采集帖子的 Subreddit URL。 确定帖子排序方式(例如 `new`、`top`、`hot`)。 **输出结构**:\ 包含全面的数据点: * **帖子详情**:\ `post_id`、`url`、`user_posted`、`title`、`description`、`num_comments`、`date_posted`、`tag`、`related_posts`、`comments`。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lvz8ah06191smkebj4/subreddit_url?id=all)。 * **社区详情**:\ `community_name`、`community_url`、`community_description`、`community_members_num`、`community_rank`。 * **媒体详情**:\ `photos`、`videos`。 该 API 提供了一种简化方式来探索和分析 Subreddit 内的帖子,帮助深入了解帖子内容、社区动态及相关媒体文件,非常适合监控 Subreddit 活动或提取帖子级别的数据进行进一步分析。 ## 评论 API ### 通过 URL 收集 该 API 允许用户从特定的 Reddit 帖子或评论 URL 采集详细的评论数据,并可选择根据发布时间筛选评论。 **输入参数** 要获取评论的 Reddit 帖子或评论的 URL。 仅采集指定天数内发布的所有评论。 **输出结构**\ 包含全面的数据点: * **评论详情**:\ `comment_id`, `user_posted`, `comment`, `date_posted`, `replies`, `num_upvotes`, `num_replies`。 > 所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lvzdpsdlw09j6t702?tab=overview)。 * **帖子详情**:\ `post_url`, `post_id`, `post_language`, `post_state`, `post_type`, `images`。 * **社区详情**:\ `community_name`, `community_url`, `community_description`, `community_members_num`, `community_rank`。 * **帖子属性**:\ `is_moderator`, `is_pinned`, `has_bot_in_username`, `is_locked`, `is_admin_post`, `is_archived_post`, `is_moderator_post`, `is_quarantined_post`, `is_not_safe_for_work_post`, `is_eligible_for_content_blocking_post`, `is_promoted_post`。 该 API 提供对 Reddit 评论及其关联帖子的全面洞察,帮助用户有效分析互动情况、用户活动和内容趋势。 # TikTok API 抓取器 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/social-media-apis/tiktok ## 概述 TikTok API 套件提供多种类型的 API,每种 API 都针对从 TikTok 采集数据的特定需求进行设计。以下是这些 API 之间的交互概述,基于可用功能: 该 API 允许用户通过单一输入(个人资料 URL)采集个人资料详情。
*   **发现功能**: *   搜索的直接 URL
*   **主要数据列**: *   `nickname`,`awg_engagement_rate`,`followers`,`likes`
该 API 允许用户通过单一输入 URL 采集多个帖子。
*   **发现功能**: *   - TikTok 个人资料的直接 URL *   - 通过关键词发现 *   - 发现的直接 URL
*   **主要数据列**: *   `url`,`share_count`,`description`,`hashtags`
该 API 允许用户使用帖子 URL 采集多个评论。
*   **发现功能**: *   N/A
*   **主要数据列**: *   `url`,`comment_text`,`commenter_url`,`num_likes`
## Profile API ### 通过 URL 采集 该 API 允许用户使用提供的个人资料 URL 检索详细的 TikTok 个人资料信息。 **输入参数**: TikTok 个人资料 URL。 **输出结构**:\ 包含全面的数据点: * **个人资料详情**:\ `account_id`,`nickname`,`biography`,`bio_link`,`predicted_lang`,`is_verified`,`followers`,`following`,`likes`,`videos_count`,`create_time`,`id`,`url`,`profile_pic_url`,`profile_pic_url_hd` 等。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_l1villgoiiidt09ci?id=all\&tab=overview)。 * **互动指标**:\ `awg_engagement_rate`,`comment_engagement_rate`,`like_engagement_rate`,`like_count`,`digg_count`。 * **隐私与设置**:\ `is_private`,`relation`,`open_favorite`,`comment_setting`,`duet_setting`,`stitch_setting`,`is_ad_virtual`,`room_id`,`is_under_age_18`。 * **发现与热门视频**:\ `region`,`top_videos`,`discovery_input`。 该 API 允许用户检索详细的 TikTok 个人资料信息,包括互动指标、隐私设置和热门视频,提供有关用户活动和个人资料数据的见解。 ### 通过搜索 URL 发现 该 API 允许用户根据特定的搜索 URL 和国家/地区发现 TikTok 个人资料,并提供详细的个人资料信息。 **输入参数**: TikTok 搜索 URL。 执行搜索的国家/地区。 **输出结构**:\ 包含全面的数据点: * **个人资料详情**:\ `account_id`,`nickname`,`biography`,`bio_link`,`predicted_lang`,`is_verified`,`followers`,`following`,`likes`,`videos_count`,`create_time`,`id`,`url`,`profile_pic_url`,`profile_pic_url_hd` 等。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_l1villgoiiidt09ci/search_url?id=all\&tab=overview)。 * **互动指标**:\ `awg_engagement_rate`,`comment_engagement_rate`,`like_engagement_rate`,`like_count`,`digg_count`。 * **隐私与设置**:\ `is_private`,`relation`,`open_favorite`,`comment_setting`,`duet_setting`,`stitch_setting`,`is_ad_virtual`,`room_id`,`is_under_age_18`。 * **发现与热门视频**:\ `region`,`top_videos`,`discovery_input`。 该 API 使用户能够根据搜索条件发现 TikTok 个人资料,并提供有关用户活动、互动、隐私设置和热门内容的见解,有助于高效发现和分析 TikTok 用户。 ## 帖子 API ### 通过 URL 采集 该 API 允许用户通过提供 TikTok 帖子 URL 来采集详细的帖子数据。 **输入参数**: TikTok 帖子 URL。 **输出结构**:\ 包含丰富的数据点: * **帖子详情**:\ `post_id`、`description`、`create_time`、`share_count`、`collect_count`、`comment_count`、`play_count`、`video_duration`、`hashtags`、`original_sound`、`official_item`、`original_item`、`shortcode`、`video_url`、`music`、`cdn_url`、`width`、`carousel_images` 等。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lu702nij2f790tmv9h?id=all\&tab=overview)。 * **个人资料详情**:\ `profile_id`、`profile_username`、`profile_url`、`profile_avatar`、`profile_biography`、`account_id`、`profile_followers`、`is_verified`。 * **标记的用户和媒体**:\ `tagged_user`、`carousel_images`。 * **附加信息**:\ `tt_chain_token`、`secu_id` *** ### 通过个人资料 URL 发现 该 API 允许用户基于提供的个人资料 URL 获取 TikTok 个人资料的帖子,并可通过帖子数量、日期范围和排除特定帖子的选项进行筛选。 **输入参数**: TikTok 个人资料 URL。 要采集的帖子数量。如果未提供,则不限制。 要排除的帖子 ID 数组。 过滤帖子的起始日期(格式:mm-dd-yyyy),应小于 `end_date`。 过滤帖子的结束日期(格式:mm-dd-yyyy),应大于 `start_date`。 指定要采集的帖子类型(例如 "post" 或 "reel")。 **输出结构**:\ 包含丰富的数据点: * **帖子详情**:\ `post_id`、`description`、`create_time`、`share_count`、`collect_count`、`comment_count`、`play_count`、`video_duration`、`hashtags`、`original_sound`、`official_item`、`original_item`、`shortcode`、`video_url`、`music`、`cdn_url`、`width`、`carousel_images` 等。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lu702nij2f790tmv9h/profile_url?id=all\&tab=overview)。 * **个人资料详情**:\ `profile_id`、`profile_username`、`profile_url`、`profile_avatar`、`profile_biography`、`account_id`、`profile_followers`、`is_verified`。 * **标记的用户和媒体**:\ `tagged_user`、`carousel_images`。 * **附加信息**:\ `tt_chain_token`、`secu_id`。 该 API 允许用户发现和检索特定 TikTok 个人资料的帖子详细信息,包括帖子数据、创作者的个人资料信息以及标记的用户。支持高效内容发现和帖子分析。 *** ### 通过关键词发现 该 API 允许用户基于特定关键词或标签搜索 TikTok 帖子,是发现 TikTok 平台相关内容的强大工具。 **输入参数**: 要搜索的关键词或标签。 要采集的帖子数量。如果未提供,则不限制。 要排除的帖子 ID 数组。 指定要采集的帖子类型(例如 "post" 或 "reel")。 **输出结构**:\ 包含丰富的数据点: * **帖子详情**:\ `post_id`、`description`、`create_time`、`digg_count`、`share_count`、`collect_count`、`comment_count`、`play_count`、`video_duration`、`hashtags`、`original_sound`、`post_type`、`discovery_input`、`official_item` 等。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lu702nij2f790tmv9h/keyword?id=all\&tab=overview)。 * **个人资料详情**:\ `profile_id`、`profile_username`、`profile_url`、`profile_avatar`、`profile_biography`、`account_id`、`profile_followers`、`is_verified`。 * **标记的用户和媒体**:\ `tagged_user`、`carousel_images`。 * **附加信息**:\ `tt_chain_token`、`secu_id`。 该 API 允许用户发现与特定关键词或标签匹配的 TikTok 帖子,并提供帖子详细信息、个人资料信息和媒体信息。这是探索 TikTok 趋势、内容和用户的绝佳工具。 *** ### 通过发现 URL 发现 该 API 允许用户从特定 TikTok 发现 URL 采集详细的帖子数据。 **输入参数**: TikTok 发现 URL。 **输出结构**:\ 包含丰富的数据点: * **帖子详情**:\ `post_id`、`description`、`create_time`、`digg_count`、`share_count`、`collect_count`、`comment_count`、`play_count`、`video_duration`、`hashtags`、`original_sound`、`post_type`、`discovery_input`、`official_item`、`original_item` 等。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lu702nij2f790tmv9h/url?id=all\&tab=overview)。 * **个人资料详情**:\ `profile_id`、`profile_username`、`profile_url`、`profile_avatar`、`profile_biography`、`account_id`、`profile_followers`、`is_verified`。 * **标记的用户和媒体**:\ `tagged_user`、`carousel_images`。 * **附加信息**:\ `tt_chain_token`、`secu_id`。 该 API 提供详细的 TikTok 发现 URL 相关帖子数据,方便访问热门内容、用户资料和帖子元数据,以便分析和探索。 ## 评论 API ### 按 URL 采集 该 API 允许用户通过提供的 TikTok 帖子 URL 采集该帖子的详细评论数据。 **输入参数**: TikTok 帖子 URL。 **输出结构**: 包含全面的数据点: * **帖子详情**: `post_url`,`post_id`,`post_date_created`。 > 获取所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lkf2st302ap89utw5k?id=all\&tab=overview)。 * **评论详情**: `date_created`,`comment_text`,`num_likes`,`num_replies`,`comment_id`,`comment_url`。 * **评论者详情**: `commenter_user_name`,`commenter_id`,`commenter_url`。 该 API 提供 TikTok 帖子的详细评论数据,包括评论相关的指标以及评论者信息,以便进行有效的评论分析和互动跟踪。 # Twitter API Scrapers Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/social-media-apis/twitter ## 概览 Twitter API 套件提供多种类型的 API,每种 API 都针对 Twitter 数据收集的特定需求而设计。以下是这些 API 如何连接和交互的概览,基于可用功能: 该 API 允许用户基于单一输入(个人资料 URL)收集个人资料详细信息。
*   **发现功能**: *   不适用
*   **重要字段**: *   `url`,`profile_name`,`is_verified`,`followers`。
该 API 允许用户基于单一输入 URL 收集多个帖子。
*   **发现功能**: *   通过个人资料 URL 发现。
*   **重要字段**: *   `name`,`description`,`replies`,`likes`。
## 个人资料 API ### 通过 URL 获取个人资料 该 API 允许用户使用提供的个人资料 URL 获取详细的 Twitter 个人资料信息。 **输入参数**: Twitter 个人资料 URL。 要收集的帖子数量。如果省略,则没有限制。 **输出结构**:\ 包含全面的数据点: * **个人资料详情**:\ `x_id`,`url`,`id`,`profile_name`,`biography`,`is_verified`,`profile_image_link`,`external_link`,`date_joined`,`location`,`birth_date`,`posts_count`,`posts`,`suggested_profiles`,`is_business_account`,`is_government_account`,`category_name`。 > 获取所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lwxmeb2u1cniijd7t4?id=all\&tab=overview)。 * **互动指标**:\ `followers`,`following`,`subscriptions`。 * **帖子详情**:\ `posts`。 该 API 允许用户获取详细的 Twitter 个人资料信息,包括用户互动指标、帖子详情等,提供对用户活动和个人资料数据的深入洞察。 ## 帖子 API ### 通过 URL 采集 该 API 允许用户使用提供的帖子 URL 获取特定 Twitter 帖子的详细信息。 **输入参数**: Twitter 帖子 URL。 API 限制:每个输入最多 1000 篇帖子。 **输出结构**:\ 包含全面的数据点: * **帖子详情**:\ `id`,`user_posted`,`name`,`description`,`date_posted`,`url`,`tagged_users`,`replies`,`reposts`,`likes`,`views`,`external_url`,`hashtags`,`quotes`,`bookmarks`,`external_image_urls`,`videos`,`quoted_post`。 > 获取所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lwxkxvnf1cynvib9co?id=all\&tab=overview)。 * **用户个人资料详情**:\ `followers`,`biography`,`posts_count`,`profile_image_link`,`following`,`is_verified`。 * **父级帖子详情**:\ `parent_post_details`。 该 API 允许用户获取特定 Twitter 帖子的详细信息,包括用户详情、互动指标和媒体内容,提供对帖子互动和用户活动的深入洞察。 ### 通过个人资料 URL 发现帖子 该 API 允许用户在指定日期范围内发现并获取特定 Twitter 个人资料的帖子。 **输入参数**: Twitter 个人资料 URL。 过滤帖子的起始日期,格式为 MM-DD-YYYY。 过滤帖子的结束日期,格式为 MM-DD-YYYY。 **输出结构**:\ 包含全面的数据点: * **帖子详情**:\ `id`,`user_posted`,`name`,`description`,`date_posted`,`url`,`tagged_users`,`replies`,`reposts`,`likes`,`views`,`external_url`,`hashtags`,`quotes`,`bookmarks`,`external_image_urls`,`videos`,`quoted_post`。 > 获取所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lwxkxvnf1cynvib9co/profile_url?id=all\&tab=overview)。 * **用户个人资料详情**:\ `followers`,`biography`,`posts_count`,`profile_image_link`,`following`,`is_verified`。 * **父级帖子详情**:\ `parent_post_details`。 该 API 允许用户在指定时间范围内从特定 Twitter 个人资料中获取帖子,提供详细的帖子互动、用户数据和媒体内容的洞察。 # Vimeo API 抓取器 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/social-media-apis/vimeo ## 概览 Vimeo API 套件提供多种 API,每种 API 都专为从 Vimeo 采集特定数据而设计。以下是这些 API 之间的连接和交互概览,基于可用功能: 此 API 允许用户基于单个输入 URL 采集多个帖子。
*   **发现功能**: *   通过个人资料 URL 发现。 *   通过关键词和许可证发现。
*   **有趣的字段**: *   `title`,`video_length`,`views`,`likes`。
## Posts API ### 通过 URL 采集 此 API 允许用户使用提供的视频 URL 采集特定 Vimeo 视频的详细信息。 **输入参数** Vimeo 视频的 URL。 **输出结构**:\ 包含全面的数据点: * **视频详情**\ `video_id`,`title`,`url`,`video_url`,`video_length`,`description`,`data_posted`,`transcript`。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lxk88z3v1ketji4pn?id=all\&tab=overview)。 * **互动与指标**\ `views`,`likes`,`comments`,`collections`。 * **上传者详情**\ `uploader`,`uploader_url`,`uploader_id`,`avatar_img_uploader`。 * **视频媒体与内容**\ `preview_image`,`related_videos`,`music_track`。 * **许可证与质量**\ `license`,`license_info`,`video_quality`。 * **视频尺寸**\ `height`,`width`。 此 API 提供 Vimeo 视频的详细洞察,包括视频内容、上传者信息、媒体链接、互动指标等,支持高效的视频分析与内容追踪。 ### 通过 URL 发现 此 API 允许用户基于特定 URL 和相关关键词发现 Vimeo 视频,并提供详细的视频信息与洞察。 **输入参数** Vimeo 帖子的 URL。 要在视频内容中搜索的关键词。 要采集的结果页数。 **输出结构**:\ 包含全面的数据点: * **视频详情**\ `video_id`,`title`,`url`,`video_url`,`video_length`,`description`,`data_posted`,`transcript`。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lxk88z3v1ketji4pn/url?id=all\&tab=overview)。 * **互动与指标**\ `views`,`likes`,`comments`,`collections`。 * **上传者详情**\ `uploader`,`uploader_url`,`uploader_id`,`avatar_img_uploader`。 * **视频媒体与内容**\ `preview_image`,`related_videos`,`music_track`。 * **许可证与质量**\ `license`,`license_info`,`video_quality`。 * **视频尺寸**\ `height`,`width`。 此 API 允许用户通过 URL 和关键词发现 Vimeo 视频,并提供详细的视频内容、上传者信息和互动指标。 ### 通过关键词和许可证发现 此 API 允许用户基于特定关键词和许可证类型发现 Vimeo 视频,并提供详细的视频信息与洞察。 **输入参数** 要在视频内容中搜索的关键词。 用于筛选视频的许可证类型(例如:Creative Commons、标准许可证)。 要采集的结果页数。 **输出结构**:\ 包含全面的数据点: * **视频详情**\ `video_id`,`title`,`url`,`video_url`,`video_length`,`description`,`data_posted`,`transcript`。 > 查看所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lxk88z3v1ketji4pn/keyword_and_license?id=all\&tab=overview)。 * **互动与指标**\ `views`,`likes`,`comments`,`collections`。 * **上传者详情**\ `uploader`,`uploader_url`,`uploader_id`,`avatar_img_uploader`。 * **视频媒体与内容**\ `preview_image`,`related_videos`,`music_track`。 * **许可证与质量**\ `license`,`license_info`,`video_quality`。 * **视频尺寸**\ `height`,`width`。 此 API 允许用户基于特定关键词和许可证类型发现 Vimeo 视频,并提供详细的视频内容、上传者信息和互动指标。 # YouTube API 爬取工具 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/social-media-apis/youtube ## 概述 YouTube API 套件提供多种 API,专为从 YouTube 采集特定数据而设计。以下是这些 API 的交互方式及可用功能概览: 此 API 允许用户基于单一输入(个人资料 URL)收集用户资料详情。
*   **发现功能**: *   [通过关键词发现](/cn/api-reference/web-scraper-api/social-media-apis/youtube#discover-profiles-by-keywords)
*   **重要字段**: *   `name`、`url`、`videos_count`、`views`。
此 API 允许用户基于单一输入 URL 收集多个帖子。
*   **发现功能**: *   通过关键词发现。 *   通过搜索过滤器发现。 *   通过标签发现。 *   通过频道 URL 发现。
*   **重要字段**: *   `url`、`title`、`likes`、`views`。
此 API 允许用户通过帖子 URL 收集多个评论。
*   **发现功能**: *   无
*   **重要字段**: *   `comment_text`、`replies`、`likes`、`username`。
## 用户资料 API ### 通过 URL 采集 此 API 允许用户使用提供的频道 URL 获取详细的 YouTube 频道信息。 **输入参数**: YouTube 频道 URL。 **输出结构**:\ 包含丰富的数据点: * **用户资料详情**:\ `url`、`handle`、`name`、`description`、`created_date`、`identifier`、`id`、`handle_md5`。 * **互动指标**:\ `subscribers`、`videos_count`、`views`。 > 所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lk538t2k2p1k3oos71?id=all\&tab=overview)。 * **媒体资源**:\ `profile_image`、`banner_img`。 > 所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lk538t2k2p1k3oos71?id=all\&tab=overview)。 * **附加信息**:\ `details`、`links`、`discovery_input`。 此 API 提供有关 YouTube 频道的详细信息,包括互动指标、用户资料图片及其他关键信息。 ### 通过关键词发现 此 API 允许用户通过搜索与频道或其视频相关的特定关键词来发现 YouTube 频道。 **输入参数**: 与频道或其视频相关的关键词。 **输出结构**:\ 包含丰富的数据点: * **用户资料详情**:\ `url`、`handle`、`name`、`description`、`created_date`、`identifier`、`id`、`handle_md5`。 > 所有数据点,[点击这里](https://brightdata.com/cp/data_api/gd_lk538t2k2p1k3oos71/keyword?id=all\&tab=overview)。 * **互动指标**:\ `subscribers`、`videos_count`、`views`。 * **媒体资源**:\ `profile_image`、`banner_img`。 * **附加信息**:\ `details`、`links`、`discovery_input`。 此 API 通过关键词高效发现 YouTube 频道,提供用户资料信息、互动指标及相关详细信息。 ## 帖子 API ### 通过 URL 收集 该 API 允许用户使用提供的视频 URL 从 YouTube 获取详细的视频信息。 **输入参数**: YouTube 视频 URL。 **输出结构**:\ 包含丰富的数据点: * **视频详情**:\ `url`,`title`,`video_url`,`video_length`,`video_id`,`post_type`,`date_posted`,`description`,`music`,`transcript`,`formatted_transcript`。 > 获取所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lk56epmy2i5g7lzu0k?id=all\&tab=overview)。 * **互动指标**:\ `likes`,`views`,`num_comments`,`related_videos`。 * **频道详情**:\ `youtuber`,`youtuber_id`,`handle_name`,`channel_url`,`subscribers`,`verified`,`avatar_img_channel`,`youtuber_md5`。 * **视频媒体与质量**:\ `preview_image`,`viewport_frames`,`current_optimal_res`,`codecs`,`color`,`quality`,`quality_label`。 * **附加信息**:\ `discovery_input`,`shortcode`,`is_sponsored`,`license`。 该 API 使用户能够获取 YouTube 视频的详细信息,包括元数据、互动指标和频道相关信息,支持内容分析与跟踪。 ### 通过频道 URL 发现 该 API 允许用户通过 YouTube 频道的 URL 收集该频道发布的视频,并可使用可选过滤条件,如视频数量、日期范围和排序顺序。 **输入参数**: YouTube 频道 URL。 要收集的视频数量。如果省略,则无上限。 视频过滤起始日期(MM-DD-YYYY)。 视频过滤结束日期(MM-DD-YYYY)。 结果排序方式(例如按浏览量、日期或相关性排序)。 **输出结构**:\ 包含丰富的数据点: * **视频详情**:\ `url`,`title`,`video_url`,`video_id`,`video_length`,`date_posted`,`description`,`post_type`。 > 获取所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lk56epmy2i5g7lzu0k/url?id=all\&tab=overview)。 * **频道详情**:\ `youtuber`,`youtuber_id`,`channel_url`,`handle_name`,`avatar_img_channel`,`subscribers`,`youtuber_md5`,`verified`。 * **互动指标**:\ `likes`,`views`,`num_comments`,`is_sponsored`。 * **媒体与技术信息**:\ `preview_image`,`related_videos`,`music`,`shortcode`,`viewport_frames`,`current_optimal_res`,`codecs`,`color`,`quality`,`quality_label`,`license`。 * **字幕**:\ `transcript`,`formatted_transcript`。 该 API 允许用户高效获取 YouTube 频道中的视频,提供详细的视频、频道和互动信息,并支持过滤和排序选项以增强分析能力。 ### 通过关键词发现 该 API 允许用户使用特定关键词搜索 YouTube 视频,并可通过可选参数按日期范围和视频数量进行筛选。 **输入参数**: 搜索视频的关键词。 要收集的视频数量。 视频过滤起始日期(MM-DD-YYYY)。 视频过滤结束日期(MM-DD-YYYY)。 **输出结构**:\ 包含丰富的数据点: * **视频详情**:\ `url`,`title`,`video_url`,`video_id`,`video_length`,`date_posted`,`description`,`post_type`。 > 获取所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lk56epmy2i5g7lzu0k/keyword?id=all\&tab=overview)。 * **频道详情**:\ `youtuber`,`youtuber_id`,`channel_url`,`handle_name`,`avatar_img_channel`,`subscribers`,`youtuber_md5`,`verified`。 * **互动指标**:\ `likes`,`views`,`num_comments`,`is_sponsored`。 * **媒体与技术信息**:\ `preview_image`,`related_videos`,`music`,`shortcode`,`viewport_frames`,`current_optimal_res`,`codecs`,`color`,`quality`,`quality_label`,`license`。 * **字幕**:\ `transcript`,`formatted_transcript`。 该 API 使用户能够通过关键词高效发现 YouTube 视频,并支持高级筛选功能,以实现精准的视频分析与检索。 ### 通过搜索过滤条件发现 该 API 允许用户使用高级搜索过滤条件搜索 YouTube 视频,如上传日期、视频类型、时长和其他特性,以实现更精准和有针对性的视频发现。 **输入参数**: 搜索视频的关键词。 按上传日期筛选结果(例如:今天、本周、本月)。 指定视频类型(例如:视频、频道、播放列表)。 按视频时长筛选(例如:短、中、长)。 指定视频附加特性(例如:4K、HD、字幕、直播)。 **输出结构**:\ 包含丰富的数据点: * **视频详情**:\ `url`,`title`,`video_url`,`video_id`,`video_length`,`date_posted`,`description`,`post_type`。 > 获取所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lk56epmy2i5g7lzu0k/search_filters?id=all\&tab=overview)。 * **频道详情**:\ `youtuber`,`youtuber_id`,`channel_url`,`handle_name`,`avatar_img_channel`,`subscribers`,`youtuber_md5`,`verified`。 * **互动指标**:\ `likes`,`views`,`num_comments`,`is_sponsored`。 * **媒体与技术信息**:\ `preview_image`,`related_videos`,`music`,`shortcode`,`viewport_frames`,`current_optimal_res`,`codecs`,`color`,`quality`,`quality_label`,`license`。 * **字幕**:\ `transcript`,`formatted_transcript`。 该 API 提供 YouTube 高级搜索功能,允许用户通过多个参数筛选结果,实现精准和高效的视频发现。 ### 通过标签发现 此 API 允许用户通过特定的标签搜索 YouTube 帖子,使用户能够发现与热门话题或特定主题相关的视频。 **输入参数:** 要搜索的标签 (例如:#fitness, #travel)。 要收集的帖子数量。如果省略,则没有限制。 需要从结果中排除的帖子 ID 数组。 以 MM-DD-YYYY 格式指定筛选帖子的起始日期 (应早于 end\_date)。 以 MM-DD-YYYY 格式指定筛选帖子的结束日期 (应晚于 start\_date)。 **输出结构:**\ 包含全面的数据点: * **视频详情:**\ `url`, `title`, `video_url`, `video_id`, `video_length`, `date_posted`, `description`, `post_type`。 > 获取所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lk56epmy2i5g7lzu0k/hashtag?id=all\&tab=overview)。 * **频道详情:**\ `youtuber`, `youtuber_id`, `channel_url`, `handle_name`, `avatar_img_channel`, `subscribers`, `youtuber_md5`, `verified`。 * **互动指标:**\ `likes`, `views`, `num_comments`, `is_sponsored`。 * **媒体和技术信息:**\ `preview_image`, `related_videos`, `music`, `shortcode`, `viewport_frames`, `current_optimal_res`, `codecs`, `color`, `quality`, `quality_label`, `license`。 * **字幕信息:**\ `transcript`, `formatted_transcript`。 此 API 通过搜索与特定标签相关的帖子,实现高效的视频发现,并提供筛选选项以优化搜索结果。 ## 评论 API ### 通过 URL 收集 此 API 允许用户使用提供的视频 URL 检索特定 YouTube 视频的详细评论数据。 **输入参数:** YouTube 视频的 URL。 加载评论回复的次数。 **输出结构:**\ 包含全面的数据点: * **评论详情:**\ `comment_id`, `comment_text`, `likes`, `replies`, `replies_value`, `replies_without_names`。 > 获取所有数据点,请[点击这里](https://brightdata.com/cp/data_api/gd_lk9q0ew71spt1mxywf?id=all\&tab=overview)。 * **用户详情:**\ `username`, `user_channel`, `username_md5`。 * **附加信息:**\ `date`, `url`, `video_id`。 此 API 使用户能够分析 YouTube 视频的评论,包括回复和用户详细信息,从而提供有关互动和受众参与度的洞察。 # 触发数据收集 API Source: https://docs.brightdata.com/cn/api-reference/web-scraper-api/trigger-a-collection dca-api POST /datasets/v3/trigger 了解如何使用 Web Scraper API 触发数据收集,提供发现和 PDP 抓取器的选项。自定义请求,设置交付选项,并高效地检索数据。 ## 正文 要供抓取器使用的输入。可以作为 JSON 或 CSV 文件提供: **`Content-Type: application/json`** 输入的 JSON 数组 > **示例**: `[{"url":"https://www.airbnb.com/rooms/50122531"}]` *** **`Content-Type: multipart/form-data`** 一个 CSV 文件,字段名为 `data` > **示例** (curl): `data=@path/to/your/file.csv` ## Web Scraper 类型 每种抓取器可能需要不同的输入。主要有两种类型的抓取器: ### 1. PDP 这些抓取器需要 URL 作为输入。PDP 抓取器从网页中提取详细的产品信息,如规格、定价和功能。 ### 2. Discovery Discovery 抓取器允许您通过搜索、类别、关键词等来探索和发现新实体/产品。 ae.com.png ## 请求示例 ### `PDP` 以 URL 输入 `PDP` 的输入格式始终是 URL,指向要抓取的页面。 ```sh Sample Request curl -H "Authorization: Bearer API_KEY" -H "Content-Type: application/json" -d '[{"url":"https://www.airbnb.com/rooms/50122531"},{"url":"https://www.airbnb.com/rooms/50127677"}]' "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_ld7ll037kqy322v05&format=json&uncompressed_webhook=true" ``` ### 基于 `discovery` 方法的 Discovery 输入 ```sh Sample Request curl -H "Authorization: Bearer x2x3fdaaddrer" -H "Content-Type: application/json" -d '[{"keyword":"light bulb"},{"keyword":"dog toys"},{"keyword":"home decor"}]' "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l7q7dkf244hwjntr0&endpoint=https://webhook-url.com&auth_header=QWxhZGRpbjpPcGVuU2VzYW1l¬ify=https://notify-me.com/&format=ndjson&uncompressed_webhook=true&type=discover_new&discover_by=keyword&limit_per_input=10" ``` `discovery` 的输入格式可以根据特定的抓取器有所不同。输入可以是: ```JSON keywords [{"keyword": "light bulb"},{"keyword": "dog toys"},{"keyword": "home decor"}] ``` ```JSON 搜索设置 [{"url": "https://www.amazon.com/s?i=luggage-intl-ship", "sort_by": "Best Sellers"}] ``` ```JSON 位置 [{"location": "Europe"},{"location": "Greece"},{"location": "United States"}] ``` 以及更多。了解每个抓取器需要的输入,请参见 [这里](https://brightdata.com/cp/data_api). # 根据请求级别选择交付类型 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-ide-api/Choose_a_delivery_type_on_request_level 您可以使用 API 为每个任务选择交付类型,而不是为每种交付类型创建重复的抓取工具。 hero-image.png 确保将您的抓取工具更新到最新版本以提高成功率 update-available.png type-batch.png on-completion.png 当交付类型设置为实时时,批处理 API 将返回以下错误消息。 ```json 错误 "error": "Cannot trigger a batch job with a real-time scraper. Use /trigger_immediate endpoint instead" ``` `dca/trigger` [触发抓取工具,获取批量采集方法](/cn/api-reference/web-scraper-ide-api/Trigger_a_scraper_for_batch_collection_method) ```sh Shell curl "https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1" -H "Content-Type: application/json" -H "Authorization: Bearer API_KEY" -d '[{"url":"https://targetwebsite.com/product_id/"}]' ``` ```json Sample Response { "collection_id":"j_l3daejgw1wnpjxxxxx", "start_eta":"2022-05-19T17:28:48.056Z" } ``` `dca/trigger_immediate` [Trigger a scraper for real-time collection](/cn/api-reference/web-scraper-ide-api/Receive_data_from_real_time_work_scraper) ```sh Shell curl "https://api.brightdata.com/dca/trigger_immediate?collector=ID_COLLECTOR" -H "Content-Type: application/json" -H "Authorization: Bearer API_KEY" -d '{"url":"https://targetwebsite.com/product_id/"}' ``` ```json Sample Response { "response_id":"z2805t1652973963340rg6252xxxxxx" } ``` 批量响应以 `j_****` 开头,实时响应以 `z****` 开头 * [接收批量数据](/cn/api-reference/web-scraper-ide-api/Receive_batch_data) * [由实时工作抓取工具接收数据](/cn/api-reference/web-scraper-ide-api/Receive_data_from_real_time_work_scraper) # 开始使用 API Source: https://docs.brightdata.com/cn/api-reference/web-scraper-ide-api/Getting_started_wtih_the_API 1. 注册(或登录)我们的控制面板 2. 插入目标 URL(作为收集数据来源的网站)或选择现成的模板 3. 设置抓取工具的交付偏好:批处理或实时 4. 由 API 启动 不是开发人员? 联系我们的支持人员,轻松上手。 数据抓取工具分为两个抓取系统: 这是处理大量数据的有效方法。 数据在一段时间内批量收集。 收集、输入和处理数据后,将生成批处理结果。 您可以一次性交付所有结果,也可以在收集结果时将其流式传输到您的系统。 This is an ideal way to get a fast response for one request. This is useful in environments where you have a limited amount of time to crawl a page and get the data. 数据抓取工具 API 有四个一般工作流程: 1. [批量触发器 + 推送交付](/cn/api-reference/web-scraper-ide-api/Trigger_a_scraper_for_batch_collection_method) 2. [批量触发器 + API 轮询](/cn/api-reference/web-scraper-ide-api/Receive_batch_data) 3. [实时触发器 + 推送交付](/cn/api-reference/web-scraper-ide-api/Trigger_a_scraper_for_real_time_collection) 4. [实时触发器 + API 轮询](/cn/api-reference/web-scraper-ide-api/Receive_data_from_real_time_work_scraper) # 接收批量数据 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-ide-api/Receive_batch_data **API 端点:** `GET` `/dca/dataset?id={id_dataset}` 数据集的唯一标识 身份验证 Bearer Key `-H "Authorization: Bearer API_KEY"` ```json Waiting for Dataset { "status": "building", "message": "Dataset is not ready yet, try again in XXs" } ``` ```json Dataset (Ready) [ { "Image": "https://target网站.com/product_ID.png", "Title": "product_name", "Price": "product_price", "input": { "url": "https://targetwebsite.com/product_id/" } } ] ``` **Shell** ```sh Shell curl "https://api.brightdata.com/dca/dataset?id=ID_DATASET" -H "Content-Type: application/json" -H "Authorization: " ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'https://api.brightdata.com/dca/dataset?id=ID_DATASET', headers: {'Content-Type': 'application/json','Authorization':'Bearer API_KEY'}, }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.entity.ContentType; import org.apache.http.HttpHost; import org.apache.http.client.fluent.*; public class Main { public static void main(String[] args) { String body = "[{\"url\":\"https://targetwebsite.com/product_id/\"}]"; String res = Executor.newInstance() .execute( Request.Post("https://api.brightdata.com/dca/trigger?collector=COLLECTOR_ID&queue_next=1") .addHeader("Authorization", "Bearer API_KEY") .addHeader("Content-Type", "application/json") .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res); } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("https://api.brightdata.com/dca/dataset?id=ID_DATASET"), Headers = { {"Content-Type", "application/json"}, {"Authorization", "Bearer API_KEY"} } }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests headers = {'Content-Type': 'application/json','Authorization': 'Bearer API_KEY'} r = requests.get('https://api.brightdata.com/dca/dataset?id=ID_DATASET', headers=headers) print(r.content) ``` # 从实时运行抓取器接收数据 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-ide-api/Receive_data_from_real_time_work_scraper **API 端点:** `GET` `/dca/get_result?response_id={ID_RESPONSE}` 响应的唯一标识 身份验证 Bearer Key `-H "Authorization: Bearer API_KEY"` ```json Sample (old scrapers) { "input": { "url": "https://targetwebsite.com/product_id/" }, "line_1": "Lorem ipsum dolor sit amet", "line_2": "consectetur adipisicing elit" } ``` ```json Sample (new scrapers) [ "line_1": "Lorem ipsum dolor sit amet", "line_2": "consectetur adipisicing elit" ] ``` ```sh Shell curl "https://api.brightdata.com/dca/get_result?response_id=ID_RESPONSE" -H "Authorization: Bearer API_KEY" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ url: 'https://api.brightdata.com/dca/get_result?response_id=ID_RESPONSE', headers: {'Authorization':'Bearer API_KEY'}, }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String res = Executor.newInstance() .addHeader("Authorization", "Bearer API_KEY") .execute(Request.Get("https://api.brightdata.com/dca/get_result?response_id=ID_RESPONSE")) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Get, RequestUri = new Uri("https://api.brightdata.com/dca/get_result?response_id=ID_RESPONSE"), Headers = { {"Authorization", "Bearer API_KEY"} } }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests headers = {'Authorization': 'Bearer API_KEY'} r = requests.get('https://api.brightdata.com/dca/get_result?response_id=ID_RESPONSE', headers=headers) print(r.content) ``` # 触发抓取工具,进行批量采集 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-ide-api/Trigger_a_scraper_for_batch_collection_method **API 端点:** `POST` `/dca/trigger?collector={ID_COLLECTOR}` ## `POST` Body raw JSON `-d '[{"url":"https://targetwebsite.com/product_id/"}]'` ## 查询参数 抓取工具的唯一标识 设置为 `dev` 以触发抓取工具的开发版本 `human_name` - 批量采集的数据名称,须易于识别和理解 ### 选择 `trigger_behavior` 参数: { /* TODO: Need more info on this? Does 1 means True and 0 means False? */} 如果抓取队列中已有任务,则将该作业推送至队列中 发送另一批请求,这些请求将在上一批请求完成后开始处理 { /* TODO: Need more info on this? Does 1 means True and 0 means False? */} 取消正在运行的作业,然后提交并运行新的请求 { /* TODO: Need more info on this? Does 1 means True and 0 means False? */} 禁止下载媒体文件 设置作业截止时间,作业将在指定时间后终止 * h:小时 * m:分钟 * s:秒 ## 标头 授权持有者密钥 `-H "Authorization: Bearer API_KEY"` JSON/CSV/Multipart `-H "Content-Type: application/json"` ```json Sample Response { "collection_id": "ID_DATASET", "start_eta": "2021-11-07T13:26:22.702Z" } ``` ```sh Shell curl "https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1" -H "Content-Type: application/json" -H "Authorization: Bearer API_KEY" -d '[{"url":"https://targetwebsite.com/product_id/"}]' ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1', json: [{'url': 'https://targetwebsite.com/product_id/'}], headers: {'Authorization': 'Bearer API_KEY'}, }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "[{\"url\":\"https://targetwebsite.com/product_id/\"}]"; String res = Executor.newInstance() .addHeader("Authorization", "Bearer API_KEY") .execute(Request.Post("https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1")) .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1"), Headers = { {"Authorization", "Bearer API_KEY"} }, Content = new StringContent(JsonConvert.SerializeObject([ new { url = "https://targetwebsite.com/product_id/" } ]), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = [{'url': 'https://targetwebsite.com/product_id/'}] headers = {'Content-Type': 'application/json','Authorization': 'Bearer API_KEY'} r = requests.post('https://api.brightdata.com/dca/trigger?collector=ID_COLLECTOR&queue_next=1', data=data, headers=headers) print(r.content) ``` # 触发抓取工具,进行实时采集 Source: https://docs.brightdata.com/cn/api-reference/web-scraper-ide-api/Trigger_a_scraper_for_real_time_collection **API 端点:** `POST` `/dca/trigger_immediate?collector={ID_COLLECTOR}` ## `POST` Body raw JSON `-d '[{"url":"https://targetwebsite.com/product_id/"}]'` ## 查询参数 抓取工具的唯一标识 设置为 `dev` 以触发抓取工具的开发版本 `human_name` - 实时采集的数据名称,须易于识别和理解 ## 标头 授权持有者密钥 `-H "Authorization: Bearer API_KEY"` JSON/CSV/Multipart `-H "Content-Type: application/json"` ```json Sample Response { "response_id": "ID_RESPONSE", } ``` ```sh Shell curl "https://api.brightdata.com/dca/trigger_immediate?collector=ID_COLLECTOR" -H "Content-Type: application/json" -H "Authorization: Bearer API_KEY" -d "{\"url\":\"https://targetwebsite.com/product_id/\"}" ``` ```js NodeJS #!/usr/bin/env node require('request-promise')({ method: 'POST', url: 'https://api.brightdata.com/dca/trigger_immediate?collector=ID_COLLECTOR', json: {'url': 'https://targetwebsite.com/product_id/'}, headers: {'Authorization': 'Bearer API_KEY'}, }).then(function(data){ console.log(data); }, function(err){ console.error(err); }); ``` ```java Java package example; import org.apache.http.HttpHost; import org.apache.http.client.fluent.\*; public class Example { public static void main(String[] args) throws Exception { String body = "{\"url\":\"https://targetwebsite.com/product_id/\"}"; String res = Executor.newInstance() .addHeader("Authorization", "Bearer API_KEY") .execute(Request.Post("https://api.brightdata.com/dca/trigger_immediate?collector=ID_COLLECTOR")) .bodyString(body, ContentType.APPLICATION_JSON)) .returnContent().asString(); System.out.println(res) } } ``` ```cs C# using System; using System.Net; using System.Net.Http; using System.Net.Http.Headers; public class Program { public static async Task Main() { var client = new HttpClient(); var requestMessage = new HttpRequestMessage { Method = HttpMethod.Post, RequestUri = new Uri("https://api.brightdata.com/dca/trigger_immediate?collector=ID_COLLECTOR"), Headers = { {"Authorization", "Bearer API_KEY"} }, Content = new StringContent(JsonConvert.SerializeObject( new { url = "https://targetwebsite.com/product_id/" }), Encoding.UTF8, "application/json")) }; var response = await client.SendAsync(requestMessage); var responseString = await response.Content.ReadAsStringAsync(); Console.WriteLine(responseString); } } ``` ```python Python #!/usr/bin/env python print('If you get error "ImportError: No module named requests", please install it:\n$ sudo pip install requests'); import requests data = {'url': 'https://targetwebsite.com/product_id/'} headers = {'Content-Type': 'application/json','Authorization': 'Bearer API_KEY'} r = requests.post('https://api.brightdata.com/dca/trigger_immediate?collector=ID_COLLECTOR', data=data, headers=headers) print(r.content) ``` # 抓取任何网站 Source: https://docs.brightdata.com/cn/datasets/custom-datasets/Scrape-any-web **解锁自定义数据集的强大功能,以抓取任何您想要的数据网站** 在从网站抓取数据时,没有一种通用的方法可以适用。网络上的每个域或URL都有其独特的结构、挑战和要求。无论您是想从电子商务网站、房地产门户、招聘网站、社交媒体平台或其他任何领域提取数据,Bright Data 的 **Custom Dataset(自定义数据集)** 产品都是解决方案。这就是为什么它是您所有网页抓取需求的终极工具。 **什么是 Custom Dataset(自定义数据集)?** Custom Dataset 是一种先进且用户友好的解决方案,旨在帮助您从任何网站精确提取所需的数据,无论您的需求有多复杂或具体。使用此产品,您无需担心构建自己的抓取基础设施、处理技术难题或应对网站不断变化的结构。 很简单:您提供要抓取的域或URL,分享所需的数据点或格式,Bright Data 负责完成其余的工作。换句话说,Custom Dataset 是您的全方位、按需提供的抓取解决方案,专为您的需求量身定制。 **为什么要为任何域或URL使用 Custom Dataset?** **根据您的需求量身定制** 不同的网站有不同的结构、布局和规则。Custom Dataset 专为适应任何网站或域名而设计。无论您是在寻找产品价格、竞争分析、市场趋势、潜在客户还是客户评价,Custom Dataset 都能个性化满足您的确切要求。 **适用于任何网站** 无论网站所属的行业或复杂性如何,Custom Dataset 都可以应用于从几乎任何在线域中抓取信息。它非常适合从动态内容、带有 CAPTCHA 保护或其他反爬机制的网站中提取数据。 **无需技术专业知识** 忘记花费数小时编写抓取程序或处理复杂的网页结构吧。Bright Data 的专家团队会处理所有技术难题,让您专注于利用数据推动业务增长。 **可扩展且高效** Custom Dataset 为扩展性而构建。无论您是需要用于一次性项目的小数据集,还是需要定期进行的大规模数据提取,这一解决方案都能轻松应对。 **可靠且准确的数据** 在数据准确性至关重要时,Bright Data 总能交付。Custom Dataset 确保您获得干净、结构化、可信赖的数据,并以您需要的格式(如 JSON、CSV、Excel 等)整齐打包。 **它是如何工作的?** **提交您的请求** 访问 Bright Data 的网站,分享您要抓取的域或URL。描述您的具体要求,例如特定的数据字段或您希望的数据交付格式。 **我们构建并执行抓取程序** 我们的专家团队将构建一个自定义管道并抓取请求的数据。您无需担心代理、浏览器自动化或错误处理的复杂性——我们为您处理所有细节。 **接收您的数据** 以您喜欢的格式获取高质量、即用型数据。利用这些数据做出明智决策、优化运营或获得竞争优势。 **为什么选择 Custom Dataset?** 当您寻找从特定域或URL抓取数据的解决方案时,Custom Dataset 是您可以信赖的产品。相比费力使用通用的抓取工具或编写自定义代码(这些方法可能并不适合所有网站),Bright Data 的 Custom Dataset 提供了一种精简且可靠的选择,确保您快速、无忧地获取所需数据。 无论您是企业、研究人员、初创公司还是开发者,Custom Dataset 都是您从任何URL随时访问任何数据的快捷通道。因此,下次您需要抓取网页时,请将 Custom Dataset 作为您的首选解决方案。 立即探索 Bright Data 的 Custom Dataset 产品,让您的数据驱动项目焕发生机! # 自定义数据集 API Source: https://docs.brightdata.com/cn/datasets/custom-datasets/custom-dataset-api 此更新可使您以更精确、更简化的方式请求和管理您的数据采集,从而根据您的特定需求更有效地生成数据集。 了解何时使用每个 API: ## 没有客户定义视图的初始采集: 3 个主要 API 端点在数据采集工作流程中具有不同的用途,可促进高效获取定制数据集的结构化流程。 ### 请求采集: 端点: `POST` [https://api.brightdata.com/datasets/request\_collection](https://api.brightdata.com/datasets/request_collection) **参数:** 数据集 ID `discover_new` 或 `url_collection` Array - json multipart - csv ```sh 示例 curl "https://api.brightdata.com/datasets/request_collection?dataset_id=gd_l1viktl72bvl7bjuj0&type=discover_new" -H "Authorization: Bearer API_KEY" -H "Content-Type: application/json" -k -d '[{"id":"user-id"}]' ``` 处理可能需要几分钟时间,具体取决于输入的数量。 当您请求查找(`discover_new`)时,找到所有链接 (PDP) 可能需要一些时间。 *** ### 查看以上采集的状态: 端点: `GET` [https://api.brightdata.com/datasets/request\_collection](https://api.brightdata.com/datasets/request_collection) **参数:** 从先前的 API 获取。 设置数据新鲜度。 如果数据在此期限内(例如,请求在 1 周之前采集,5 天前采集),则会发生 0 次新的抓取。 * 1 周: 604,800,000 毫秒 * 1 个月: 2,592,000,000 毫秒 ```sh 示例 curl -k "https://api.brightdata.com/datasets/request_collection?request_id=REQUEST_ID&freshness_ms=2592000000" -H "Authorization: Bearer API_KEY" ``` 说明发现的记录数量和新鲜度的响应: ```json { "dataset_id": request_job.dataset_id, "total_lines": 100, "fresh_count": 30, "name": "linkedin_companies custom input", "status": "done", "request_id": "XXXX", } ``` 请求仍在运行: ```json { "total_lines": 100, "status": "running", } ``` 一个(或多个)输入有问题: 在这种情况下,此 url 以 URL 形式发送 ```json { "request_id": "xxxx", "error": "Validation failed", "error_code": "validation", "validation_errors": [ { "line": "{\"URL\":\"https://www.tiktok.com/search?q=tjd\"}", "index": 1, "errors": [ ["url", "Required field"] ] } ] } ``` *** ### 启动采集: 端点: `POST` [https://api.brightdata.com/datasets/initiate\_collection](https://api.brightdata.com/datasets/initiate_collection) **参数:** 您正在查询的采集请求的唯一标识符。 以毫秒为单位的时间,表示所需的数据新鲜度。 以毫秒为单位的时间,表示所需的数据新鲜度。 ```sh 示例 curl -X POST -k "https://api.brightdata.com/datasets/initiate_collection" -d '{"request_id":"j_ln2x567b2961de0d1x","freshness_ms":2592000000}' -H "Authorization: Bearer API_KEY" -H "content-type: application/json" ``` ## 定义视图后的采集: *** ### 启动采集: 端点: `POST` [https://api.brightdata.com/datasets/initiate](https://api.brightdata.com/datasets/initiate) **参数:** `discover_new` 或 `url_collection` Array - json multipart - csv ```sh 示例 curl "https://api.brightdata.com/datasets/initiate?dataset_id=XXX_DATASET_ID&type=url_collection&view=XXX_VIEW_ID" -H "Authorization: Bearer API_KEY" -H "Content-Type: application/json" -k -d '[{"id":"user-id"}]' ``` 数据集将传送到为此视图配置的设置。 现在,利用这些增强的功能,用户可以更有效地定制其数据采集流程,确保生成的数据集符合其项目要求。 *** ### 如何检索已采集的快照结果 ```sh curl "https://api.brightdata.com/datasets/snapshots/snapshot_id/download" -H "Authorization: Bearer API_KEY" ``` # 交付选项 Source: https://docs.brightdata.com/cn/datasets/custom-datasets/delivery-options 本文介绍了快照准备就绪后获取数据集的相关选项。 要设置数据集的交付偏好,只需点击“交付设置”选项卡即可: * 选择文件格式: * JSON * NDJSON * CSV * JSON Lines * 选择如何接收数据: * Amazon S3 ([AWS S3 用户角色权限](/cn/datasets/custom-datasets/delivery-options#aws-s3-user-role-permissions)) * Google Cloud Storage ([如何查找您的 Google 云私钥](/cn/datasets/custom-datasets/delivery-options#how-to-find-your-google-cloud-private-key)) * Google Cloud PubSub * Microsoft Azure 存储 * SFTP/FTP * Snowflake ([Snowflake 交付配置指南](/cn/datasets/custom-datasets/delivery-options#snowflake-delivery-configuration-guide-only-visible-to-agents-and-admins)) ![](https://brightdata.zendesk.com/hc/article_attachments/22455157046161) 您还可以为每个快照设置交付首选项,为此,只需点击快照,然后点击“导出”,设置所需的存储参数即可: ![](https://brightdata.zendesk.com/hc/article_attachments/22455157047441) 可以通过点击“下载 + 格式化”按钮直接从 CP 下载数据集: ![](https://brightdata.zendesk.com/hc/article_attachments/22455219252497) # 常见问题解答 Source: https://docs.brightdata.com/cn/datasets/custom-datasets/faqs ... # 请求 Source: https://docs.brightdata.com/cn/datasets/custom-datasets/request ## 简介 Bright Data 的“请求定制数据集”功能提供了一种简化的、量身定制的数据收集方法。本指南概述了请求和获取适合特定需求的定制数据集的步骤。