SERP troubleshooting

Debugging SERP API

Use the x-brd-debug response header to extract detailed debug information about your requests. Activate by appending -debug-full to your proxy username:

curl -vk \
  -x brd-customer-$CUSTOMER_ID-zone-$ZONE-debug-full:$PASSWORD@brd.superproxy.io:44445 \
  "https://www.google.com/search?q=bright+data"

The x-brd-debug response header will look like this:

req_id=hl_d09913c7_a1lw123bkcg; bytes_up=2842; bytes_down=562418; billed=false; destination_ip=162.219.225.118; used_req_headers=accept-language,accept; peer_ip=r868133f79d0c3fa9d7c7ccca0151af2e; peer_country=us; render=false

Field	Description
`req_id`	Internal request ID - include in bug reports
`bytes_up`	Outgoing traffic recorded while processing the request
`bytes_down`	Incoming traffic recorded while processing the request
`billed`	Whether the request is considered billable
`destination_ip`	IP of the remote server used to fetch the data
`used_req_headers`	Custom headers relayed in the initial request
`peer_ip`	Unique identifier for the IP used - useful for validating IP rotation
`peer_country`	Country of the peer used for the request
`render`	Whether the result is browser-rendered HTML or a raw HTTP response

Common Error Codes

Error	Description
`404`	Invalid URL - likely broken or dead
`403`	URL is valid but access is forbidden
`502`	Most common SERP API error - see detail in `x-brd-error-code`
`407`	Incorrect credentials (password or zone name)
`429`	Rate limit / auto-throttle - contact support
`401` `411` `444`	Bad request - missing headers or cookies
`503`	Service unavailable - browser check failed

Example 429 response

HTTP/1.1 429 The request was auto-throttled due to low success rate
x-brd-error-code: sr_rate_limit
x-brd-error: The request was auto-throttled due to low success rate
date: Tue, 23 Jan 2024 17:07:19 GMT
connection: keep-alive

SERP API error catalog

Fast SERP runs inside the SERP API and shares its error codes. When a request fails, the error is returned in the x-brd-error and x-brd-err-msg response headers.

`x-brd-error`	`x-brd-err-msg`	Description
`query_rejected`	This query cannot be attempted at this time. Please try again later, after a minimum of 15 seconds.	In some cases the search engine flags a query as suspicious and blocks that query globally for roughly 15 seconds. To avoid wasted requests during this window, once Bright Data detects such a block, it bans the affected query for 15 seconds globally. Any attempt to send the query during that time returns this error. Retry after a minimum of 15 seconds.

Example query_rejected response

x-brd-error: query_rejected
x-brd-err-msg: This query cannot be attempted at this time. Please try again later, after a minimum of 15 seconds.

How to detect query truncation

Google sometimes returns results for a shorter version of your query than the one you sent. For example, a search for pizza in tlv can return results for pizza. This behavior applies to both the SERP API and Fast SERP text (web) search. Compare general.query with general.detected_query in the parsed response, then check for a spelling object to tell a genuine spelling correction apart from a truncated search:

If general.query and general.detected_query match, Google searched exactly what you sent.
If they differ, check for a spelling object:
- spelling present - Google auto-corrected the spelling and the results are valid for the corrected query.
- spelling absent - Google truncated (cloaked) your query and the results are for a shorter version of what you searched.

The example below shows an auto-corrected query, where detected_query differs from query but a spelling object confirms the results are valid:

{
  "general": {
    "query": "pizaa",
    "detected_query": "pizza"
  },
  "spelling": {
    "original_text": "pizaa",
    "original_link": "https://www.google.com/search?q=pizaa&nfpr=1",
    "auto_corrected_text": "pizza",
    "auto_corrected_link": "https://www.google.com/search?q=pizza&spell=1"
  },
  "organic": [ ... ]
}

For the general and spelling field definitions, see Parsed JSON results for the SERP API or Fast SERP web search for Fast SERP.

What happens when a query is blocked

When Bright Data detects that Google has flagged a query, that exact query is blocked across the SERP API and Fast SERP for 15 seconds. Any request for the same query during that window returns the query_rejected error. Retry the query after the 15-second window, or vary the query, to receive results. See SERP API error catalog for the full error response.

Get Success Rate Statistics Per Domain

Retrieve SERP API success rate stats from the past 7 days. Supports single domain or wildcard.

Single domain

curl "https://api.brightdata.com/unblocker/success_rate/google.com" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $API_KEY"

All monitored TLDs

curl "https://api.brightdata.com/unblocker/success_rate/google.*" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $API_KEY"

Once you provide your API key, replace all $API_KEY, $CUSTOMER_ID, $ZONE, and $PASSWORD placeholders accordingly.

Introduction

Product Guides

Debugging SERP API

Common Error Codes

SERP API error catalog

How to detect query truncation

What happens when a query is blocked

Get Success Rate Statistics Per Domain

​Debugging SERP API

​Common Error Codes

​SERP API error catalog

​How to detect query truncation

​What happens when a query is blocked

​Get Success Rate Statistics Per Domain

Debugging SERP API

Common Error Codes

SERP API error catalog

How to detect query truncation

What happens when a query is blocked

Get Success Rate Statistics Per Domain