同步触发爬虫进行实时数据采集

curl --request POST \
  --url 'https://api.brightdata.com/dca/crawl?collector=c_abc123&timeout=50s' \
  --header "Authorization: Bearer YOUR_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{"url": "https://www.ebay.com/itm/326972541236"}'

[
  {
    "product_title": "Apple iPad 11th Gen, 128 GB, Wi-Fi + 5G, 11 in - Silver - BRAND NEW SEALED",
    "price": {
      "value": 324.99,
      "currency": "USD",
      "symbol": "$"
    },
    "condition": "New: A brand-new, unused, unopened, undamaged item in its original packaging (where packaging is ...",
    "seller_feedback_percentage": "100% positive feedback",
    "seller_items_sold": "173 items sold",
    "seller_since": "Joined Nov 2012",
    "item_specifics": {
      "brand": "Apple",
      "model": "Apple iPad (11th generation)",
      "processor_model": "Apple A16 Bionic",
      "operating_system": "iPadOS"
    },
    "storage_options": [],
    "seller_ratings": {
      "accurate_description": "--",
      "shipping_cost": "--",
      "shipping_speed": "--",
      "communication": "4.9"
    },
    "product_images": [
      "https://i.ebayimg.com/images/g/4a8AAeSw7x5pboS0/s-l1600.webp",
      "https://i.ebayimg.com/images/g/4a8AAeSw7x5pboS0/s-l1600.webp"
    ],
    "feedback_count": 36,
    "financing_options": {
      "monthly_payment": {
        "value": 13.99,
        "currency": "USD",
        "symbol": "$"
      },
      "payment_plan_duration": "(12 monthly payment plan)",
      "apr": "13.99%"
    },
    "input": {
      "url": "https://www.ebay.com/itm/326972541236"
    },
    "seller_namestest": "jwc8109"
  }
]

{
  "error": "crawl_results_timeout",
  "message": "Timed out waiting for crawl results (timeout=NaNs). The page crawl is continuing in the background, you can poll a GET request to https://api.brightdata.com/dca/get_result?response_id=ID until the result is ready",
  "response_id": "RESPONSE_ID"
}

POST

dca

crawl

curl --request POST \
  --url 'https://api.brightdata.com/dca/crawl?collector=c_abc123&timeout=50s' \
  --header "Authorization: Bearer YOUR_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{"url": "https://www.ebay.com/itm/326972541236"}'

[
  {
    "product_title": "Apple iPad 11th Gen, 128 GB, Wi-Fi + 5G, 11 in - Silver - BRAND NEW SEALED",
    "price": {
      "value": 324.99,
      "currency": "USD",
      "symbol": "$"
    },
    "condition": "New: A brand-new, unused, unopened, undamaged item in its original packaging (where packaging is ...",
    "seller_feedback_percentage": "100% positive feedback",
    "seller_items_sold": "173 items sold",
    "seller_since": "Joined Nov 2012",
    "item_specifics": {
      "brand": "Apple",
      "model": "Apple iPad (11th generation)",
      "processor_model": "Apple A16 Bionic",
      "operating_system": "iPadOS"
    },
    "storage_options": [],
    "seller_ratings": {
      "accurate_description": "--",
      "shipping_cost": "--",
      "shipping_speed": "--",
      "communication": "4.9"
    },
    "product_images": [
      "https://i.ebayimg.com/images/g/4a8AAeSw7x5pboS0/s-l1600.webp",
      "https://i.ebayimg.com/images/g/4a8AAeSw7x5pboS0/s-l1600.webp"
    ],
    "feedback_count": 36,
    "financing_options": {
      "monthly_payment": {
        "value": 13.99,
        "currency": "USD",
        "symbol": "$"
      },
      "payment_plan_duration": "(12 monthly payment plan)",
      "apr": "13.99%"
    },
    "input": {
      "url": "https://www.ebay.com/itm/326972541236"
    },
    "seller_namestest": "jwc8109"
  }
]

{
  "error": "crawl_results_timeout",
  "message": "Timed out waiting for crawl results (timeout=NaNs). The page crawl is continuing in the background, you can poll a GET request to https://api.brightdata.com/dca/get_result?response_id=ID until the result is ready",
  "response_id": "RESPONSE_ID"
}

同步实时端点会保持 HTTP 请求处于打开状态，直到爬虫完成或达到 timeout，然后以 JSON 形式直接返回采集到的数据。这与异步实时端点不同，后者会立即返回一个 response_id。

你只能触发单输入 API。确保有效载荷是一个对象，而不是对象数组。

超时行为

timeout 查询参数的取值必须在 25s 到 50s 之间。

如果爬虫在超时时间内完成，响应返回 200 OK 以及采集到的数据。
如果到达超时时爬虫仍在运行，响应返回 202 Accepted 以及一个 response_id。使用该 response_id 调用实时数据端点以异步方式获取结果。

何时使用同步实时

在以下情况下使用同步实时（POST /dca/crawl）：

你需要在同一个 HTTP 请求中获得结果。
你只发送单个输入。
你的客户端可以保持请求打开，直到采集完成。
采集通常在配置的超时时间内完成。

在以下情况下使用异步实时（POST /dca/trigger_immediate）：

你希望先触发采集，稍后再获取结果。
你不希望保持 HTTP 请求处于打开状态。

在以下情况下使用批量采集：

你需要在一次运行中处理多个输入。

授权

Authorization

string

header

必填

Use your Bright Data API Key as a Bearer token in the Authorization header.

How to authenticate:

Obtain your API Key from the Bright Data account settings at https://brightdata.com/cp/setting/users
Include the API Key in the Authorization header of your requests
Format: Authorization: Bearer YOUR_API_KEY

Example:

Authorization: Bearer b5648e1096c6442f60a6c4bbbe73f8d2234d3d8324554bd6a7ec8f3f251f07df

Learn how to get your Bright Data API key: https://docs.brightdata.com/api-reference/authentication

查询参数

collector

string

必填

A unique identification of the collector to run

示例:

"c_abc123"

timeout

string

必填

How long the request stays open while waiting for the result. Must be between 25s and 50s. If the scraper finishes within the timeout, the response returns 200 OK with the collected data. If the scraper is still running when the timeout is reached, the response returns 202 Accepted with a response_id.

示例:

"50s"

version

string

Set to dev to trigger the development version of the scraper

请求体

application/json

A single input object. The fields must match the input schema defined for the collector. A URL-based scraper may require url, while a search scraper may require fields such as keyword and location.

Single input object whose fields match the collector input schema.

url

string<uri>

Example field for collectors that take a target URL. Replace with the fields your collector expects.

响应

触发爬虫进行实时收集（异步）在请求级别选择交付类型

概览

产品

管理 API

同步触发爬虫进行实时数据采集

超时行为

何时使用同步实时

授权

查询参数

请求体

响应

​超时行为

​何时使用同步实时

授权

查询参数

请求体

响应

超时行为

何时使用同步实时