> ## Documentation Index
> Fetch the complete documentation index at: https://docs.brightdata.com/llms.txt
> Use this file to discover all available pages before exploring further.

# 异步亚马逊数据采集

> 使用Bright Data异步/trigger端点触发大规模Amazon抓取任务、监控进度并检索结果。

# 使用异步端点大规模抓取亚马逊数据

本指南展示如何使用异步 `/trigger` 端点大规模抓取亚马逊数据。当您有超过 20 个 URL、需要发现功能或想要将数据传送到 webhook 或 S3 时，可以使用此方法。

<Tip>
  不确定是否使用同步或异步？请阅读 [了解同步与异步请求](/datasets/scrapers/concepts/sync-vs-async)。
</Tip>

## 前置要求

* 一个 [Bright Data 账户](https://www.bright.cn/cp/start)，具有活跃的 API 密钥
* 熟悉 [同步请求流程](/datasets/scrapers/amazon/send-first-request)

## 步骤 1：触发收集

向 `/trigger` 端点发送 `POST` 请求，包含您的输入 URL：

<CodeGroup>
  ```bash cURL theme={null}
  curl -X POST \
    "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l7q7dkf244hwjntr0&format=json" \
    -H "Authorization: Bearer YOUR_API_KEY" \
    -H "Content-Type: application/json" \
    -d '[
      {"url": "https://www.amazon.com/dp/B0CHHSFMRL"},
      {"url": "https://www.amazon.com/dp/B09V3KXJPB"},
      {"url": "https://www.amazon.com/dp/B0BDJ279KF"},
      {"url": "https://www.amazon.com/dp/B0BSHF7WHW"},
      {"url": "https://www.amazon.com/dp/B0D1XD1ZV3"}
    ]'
  ```

  ```python Python theme={null}
  import requests

  response = requests.post(
      "https://api.brightdata.com/datasets/v3/trigger",
      params={
          "dataset_id": "gd_l7q7dkf244hwjntr0",
          "format": "json",
      },
      headers={
          "Authorization": "Bearer YOUR_API_KEY",
          "Content-Type": "application/json",
      },
      json=[
          {"url": "https://www.amazon.com/dp/B0CHHSFMRL"},
          {"url": "https://www.amazon.com/dp/B09V3KXJPB"},
          {"url": "https://www.amazon.com/dp/B0BDJ279KF"},
          {"url": "https://www.amazon.com/dp/B0BSHF7WHW"},
          {"url": "https://www.amazon.com/dp/B0D1XD1ZV3"},
      ],
  )

  snapshot = response.json()
  print("Snapshot ID:", snapshot["snapshot_id"])
  ```

  ```javascript Node.js theme={null}
  const response = await fetch(
    "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l7q7dkf244hwjntr0&format=json",
    {
      method: "POST",
      headers: {
        "Authorization": "Bearer YOUR_API_KEY",
        "Content-Type": "application/json",
      },
      body: JSON.stringify([
        { url: "https://www.amazon.com/dp/B0CHHSFMRL" },
        { url: "https://www.amazon.com/dp/B09V3KXJPB" },
        { url: "https://www.amazon.com/dp/B0BDJ279KF" },
        { url: "https://www.amazon.com/dp/B0BSHF7WHW" },
        { url: "https://www.amazon.com/dp/B0D1XD1ZV3" },
      ]),
    }
  );

  const snapshot = await response.json();
  console.log("Snapshot ID:", snapshot.snapshot_id);
  ```
</CodeGroup>

您应该会看到一个 `200` 响应，其中包含 `snapshot_id`：

```json theme={null}
{
  "snapshot_id": "s_m1a2b3c4d5e6f7g8h"
}
```

保存此 ID。您需要它来检查进度和下载结果。

## 步骤 2：监控进度

轮询快照状态，直到显示 `ready`。根据 URL 数量，这可能需要 30 秒到几分钟。

<CodeGroup>
  ```bash cURL theme={null}
  curl "https://api.brightdata.com/datasets/v3/progress/s_m1a2b3c4d5e6f7g8h" \
    -H "Authorization: Bearer YOUR_API_KEY"
  ```

  ```python Python theme={null}
  import time

  snapshot_id = "s_m1a2b3c4d5e6f7g8h"

  while True:
      status_response = requests.get(
          f"https://api.brightdata.com/datasets/v3/progress/{snapshot_id}",
          headers={"Authorization": "Bearer YOUR_API_KEY"},
      )
      status = status_response.json().get("status")
      print(f"Status: {status}")

      if status == "ready":
          break
      time.sleep(10)
  ```

  ```javascript Node.js theme={null}
  const snapshotId = "s_m1a2b3c4d5e6f7g8h";

  let status = "collecting";
  while (status !== "ready") {
    const statusResponse = await fetch(
      `https://api.brightdata.com/datasets/v3/progress/${snapshotId}`,
      { headers: { "Authorization": "Bearer YOUR_API_KEY" } }
    );
    const statusData = await statusResponse.json();
    status = statusData.status;
    console.log("Status:", status);

    if (status !== "ready") {
      await new Promise((r) => setTimeout(r, 10000));
    }
  }
  ```
</CodeGroup>

状态值：

| 状态           | 含义        |
| :----------- | :-------- |
| `collecting` | 正在进行网页抓取  |
| `digesting`  | 正在处理数据    |
| `ready`      | 结果可供下载    |
| `failed`     | 收集过程中遇到错误 |

## 步骤 3：下载结果

状态为 `ready` 后，下载抓取的数据：

<CodeGroup>
  ```bash cURL theme={null}
  curl "https://api.brightdata.com/datasets/v3/snapshot/s_m1a2b3c4d5e6f7g8h?format=json" \
    -H "Authorization: Bearer YOUR_API_KEY" \
    -o results.json
  ```

  ```python Python theme={null}
  results_response = requests.get(
      f"https://api.brightdata.com/datasets/v3/snapshot/{snapshot_id}",
      params={"format": "json"},
      headers={"Authorization": "Bearer YOUR_API_KEY"},
  )

  results = results_response.json()
  print(f"Collected {len(results)} products")
  ```

  ```javascript Node.js theme={null}
  const resultsResponse = await fetch(
    `https://api.brightdata.com/datasets/v3/snapshot/${snapshotId}?format=json`,
    { headers: { "Authorization": "Bearer YOUR_API_KEY" } }
  );

  const results = await resultsResponse.json();
  console.log(`Collected ${results.length} products`);
  ```
</CodeGroup>

您已成功触发、监控并下载了批量亚马逊抓取任务。

## 使用 webhook 跳过轮询

如果您不想轮询状态，请向请求添加 `endpoint` 参数以自动接收结果：

```bash theme={null}
curl -X POST \
  "https://api.brightdata.com/datasets/v3/trigger?dataset_id=gd_l7q7dkf244hwjntr0&format=json&endpoint=https://your-server.com/webhook" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[{"url": "https://www.amazon.com/dp/B0CHHSFMRL"}]'
```

参见 [如何通过 webhook 接收亚马逊数据](/datasets/scrapers/amazon/data-delivery/webhooks) 了解完整设置。

## 限制和约束

| 约束           | 值       |
| :----------- | :------ |
| 最大输入文件大小     | 1 GB    |
| 最大并发批处理请求数   | 100     |
| 最大并发单输入请求数   | 1,500   |
| Webhook 传送大小 | 最高 1 GB |
| API 下载大小     | 最高 5 GB |

## 故障排除

<Accordion title="收到 429 请求过多错误？">
  您已超过并发请求限制。减少并行请求的数量或将输入合并为更少的较大批次。每个批次可以包含最多 1 GB 的输入数据。
</Accordion>

<Accordion title="快照状态显示 failed？">
  检查所有输入 URL 是否为有效的亚马逊 URL。查看快照响应中的错误详情，或在您的 Bright Data 仪表板的 [日志选项卡](https://www.bright.cn/cp/scrapers) 中查看。
</Accordion>

<Accordion title="结果不完整或缺少某些 URL？">
  某些 URL 可能在整个任务成功的情况下单独失败。检查快照响应中的任何 `errors` 字段。在单独的请求中重试失败的 URL。
</Accordion>

## 后续步骤

<CardGroup cols={2}>
  <Card title="设置 webhook" icon="webhook" href="/datasets/scrapers/amazon/data-delivery/webhooks">
    无需轮询即可接收结果。
  </Card>

  <Card title="传送到 S3" icon="bucket" href="/datasets/scrapers/amazon/data-delivery/amazon-s3">
    将结果直接发送到您的 S3 存储桶。
  </Card>
</CardGroup>