前提条件
- 一个带有有效 API 密钥的 Bright Data 账户
- 已完成 快速开始
请求结构
每个同步请求都遵循相同模式：POST https://api.brightdata.com/datasets/v3/scrape?dataset_id={DATASET_ID}&format=json
Authorization: Bearer YOUR_API_KEY
Content-Type: application/json
[{"url": "https://www.reddit.com/..."}]
dataset_id 和输入形式。
/scrape 端点专为实时的单 URL 请求设计。Bright Data 建议每个请求不超过 20 个 URL（这是最佳实践，而非硬性限制），端点最多等待 1 分钟。如果采集耗时更长，响应为 HTTP 202，返回 snapshot_id 而不是数据，采集将继续以异步方式进行。请通过异步请求获取结果，或使用 Webhook 投递接收结果。帖子 — 按 URL 收集
通过 URL 爬取特定 Reddit 帖子。 数据集 ID：gd_lvz8ah06191smkebj4
curl -X POST \
"https://api.brightdata.com/datasets/v3/scrape?dataset_id=gd_lvz8ah06191smkebj4&format=json" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '[{"url": "https://www.reddit.com/r/learnpython/comments/1w8chn8/found_a_for_loop_and_cant_find_an_explanation_can/"}]'
import requests
response = requests.post(
"https://api.brightdata.com/datasets/v3/scrape",
params={"dataset_id": "gd_lvz8ah06191smkebj4", "format": "json"},
headers={
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
json=[{"url": "https://www.reddit.com/r/learnpython/comments/1w8chn8/found_a_for_loop_and_cant_find_an_explanation_can/"}],
)
print(response.json())
const response = await fetch(
"https://api.brightdata.com/datasets/v3/scrape?dataset_id=gd_lvz8ah06191smkebj4&format=json",
{
method: "POST",
headers: {
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
body: JSON.stringify([
{ url: "https://www.reddit.com/r/learnpython/comments/1w8chn8/found_a_for_loop_and_cant_find_an_explanation_can/" }
]),
}
);
const data = await response.json();
console.log(data);
200 响应。处理时间取决于目标网站状况和爬虫负载。
完整帖子 — 按 URL 收集模式
Reddit 帖子抓取器返回哪些输出字段？
Reddit 帖子抓取器（gd_lvz8ah06191smkebj4，collect_by_url）返回 27 个输出字段。字段说明保留 API 返回的英文原文。
输出字段（27）
输出字段（27）
| 字段 | 类型 | 说明 |
|---|---|---|
post_id | text | Post ID |
url | url | URL of the post |
user_posted | text | Username of the post creator |
title | text | Title of the post |
description | text | Post text description |
num_comments | number | Number of comments |
date_posted | date | Post publication date |
community_name | text | Name of the community |
num_upvotes | number | Number of upvotes |
photos | array | URLs of attached photos |
videos | array | URLs of attached videos |
tag | text | The name of the tag |
related_posts | array | Related posts |
comments | array | Comments of the post |
community_url | url | URL of the community |
community_description | text | Description of the community |
community_members_num | number | Number of community members |
community_rank | object | Community ranked |
post_karma | number | post_karma |
bio_description | text | bio_description |
embedded_links | array | embedded_links |
description_markdown | text | Post content with original line breaks and Markdown formatting |
subreddit_icon_image | url | Subreddit icon image |
author_icon | url | Author icon link |
user_id | text | posting user unique id |
archived | text | archived |
locked_post | boolean | Indicator if the post is locked and new comments cannot be posted. |
/scrapers/gd_lvz8ah06191smkebj4.json
帖子 — 按子版块 URL 发现
爬取来自特定子版块的最近帖子，可选通过sort_by 排序。取值区分大小写并与 Reddit 上的标签一致，例如 Hot；小写 hot 会被校验错误拒绝。
数据集 ID： gd_lvz8ah06191smkebj4
发现端点最好通过
/trigger 异步使用，因为它们可能返回许多结果。下面的同步示例仅用于快速测试。# limit_per_input 限制每个输入返回的记录数
curl -X POST \
"https://api.brightdata.com/datasets/v3/scrape?dataset_id=gd_lvz8ah06191smkebj4&format=json&type=discover_new&discover_by=subreddit_url" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"input": [{"url": "https://www.reddit.com/r/learnpython/", "sort_by": "Hot"}], "limit_per_input": 10}'
import requests
response = requests.post(
"https://api.brightdata.com/datasets/v3/scrape",
params={
"dataset_id": "gd_lvz8ah06191smkebj4",
"format": "json",
"type": "discover_new",
"discover_by": "subreddit_url",
},
headers={
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
# limit_per_input 限制每个输入返回的记录数
json={
"input": [{"url": "https://www.reddit.com/r/learnpython/", "sort_by": "Hot"}],
"limit_per_input": 10,
},
)
print(response.json())
const response = await fetch(
"https://api.brightdata.com/datasets/v3/scrape?dataset_id=gd_lvz8ah06191smkebj4&format=json&type=discover_new&discover_by=subreddit_url",
{
method: "POST",
headers: {
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
// limit_per_input 限制每个输入返回的记录数
body: JSON.stringify({
input: [{ url: "https://www.reddit.com/r/learnpython/", sort_by: "Hot" }],
limit_per_input: 10,
}),
}
);
const data = await response.json();
console.log(data);
帖子 — 按关键词发现
查找匹配搜索词的 Reddit 帖子，按日期过滤。 数据集 ID：gd_lvz8ah06191smkebj4
# limit_per_input 限制每个输入返回的记录数
curl -X POST \
"https://api.brightdata.com/datasets/v3/scrape?dataset_id=gd_lvz8ah06191smkebj4&format=json&type=discover_new&discover_by=keyword" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"input": [{"keyword": "machine learning", "date": "Past week", "num_of_posts": 20}], "limit_per_input": 10}'
import requests
response = requests.post(
"https://api.brightdata.com/datasets/v3/scrape",
params={
"dataset_id": "gd_lvz8ah06191smkebj4",
"format": "json",
"type": "discover_new",
"discover_by": "keyword",
},
headers={
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
# limit_per_input 限制每个输入返回的记录数
json={
"input": [{"keyword": "machine learning", "date": "Past week", "num_of_posts": 20}],
"limit_per_input": 10,
},
)
print(response.json())
const response = await fetch(
"https://api.brightdata.com/datasets/v3/scrape?dataset_id=gd_lvz8ah06191smkebj4&format=json&type=discover_new&discover_by=keyword",
{
method: "POST",
headers: {
"Authorization": "Bearer YOUR_API_KEY",
"Content-Type": "application/json",
},
// limit_per_input 限制每个输入返回的记录数
body: JSON.stringify({
input: [{ keyword: "machine learning", date: "Past week", num_of_posts: 20 }],
limit_per_input: 10,
}),
}
);
const data = await response.json();
console.log(data);
评论 — 按 URL 收集
爬取来自 Reddit 帖子或特定评论线程的所有评论，可选按年龄过滤。 数据集 ID：gd_lvzdpsdlw09j6t702
curl -X POST \
"https://api.brightdata.com/datasets/v3/scrape?dataset_id=gd_lvzdpsdlw09j6t702&format=json" \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '[{"url": "https://www.reddit.com/r/learnpython/comments/1w8chn8/found_a_for_loop_and_cant_find_an_explanation_can/", "days_back": 7}]'
Reddit 评论抓取器返回哪些输出字段？
Reddit 评论抓取器（gd_lvzdpsdlw09j6t702，collect_by_url）返回 34 个输出字段。字段说明保留 API 返回的英文原文。
输出字段（34）
输出字段（34）
| 字段 | 类型 | 说明 |
|---|---|---|
url | url | URL of comment |
comment_id | text | ID of comment |
user_posted | text | Username of the user who posted the comment |
comment | text | Description of the comment |
date_posted | date | Date posted |
post_url | url | URL of the post this comment appears at |
post_id | text | ID of the post this comment appears at |
community_name | text | The name of the community the comment was posted at |
community_url | text | URL of the community where the comment was posted at |
community_description | text | Description of the community this comment was posted at |
community_members_num | text | Number of community members where the comment was posted at |
community_rank | object | |
replies | array | |
num_upvotes | number | Number of comment’s upvotes |
num_replies | number | Number of replies to the comment |
days_back | number | Reproduce from Input |
is_moderator | boolean | A post has a “MOD” icon - this means it was posted by a moderator or bot in some cases |
is_pinned | boolean | pinned or stickied comment, placed on top of all |
has_bot_in_username | boolean | The username has a substring “bot” - This parameter can be an indirect sign of a bot In most cases, but can also occur in common usernames |
is_locked | boolean | The post has a “lock” icon - this means that comments are blocked by moderators and no one but them can comment on them. This parameter can be an indirect sign of a bot |
is_admin_post | boolean | hidden is labeled as “distinguished admin” |
is_archived_post | boolean | Indicates whether the post is archived or not |
is_moderator_post | boolean | Specifies if the post is made by a moderator |
is_quarantined_post | boolean | Indicates if the post is quarantined |
is_not_safe_for_work_post | boolean | Flags whether the post contains content not safe for work |
is_eligible_for_content_blocking_post | boolean | Specifies if the post is eligible for content blocking |
is_promoted_post | boolean | Indicates if the post is promoted content |
post_language | text | Language in which the post is written |
post_state | text | Current state/status of the post |
post_type | text | Type or category of the post |
images | array | Images from the comment |
parent_comment_id | text | |
root_comment_id | text | Links all replies to the original root comment |
videos | array | Video and GIF URLs |
/scrapers/gd_lvzdpsdlw09j6t702.json
相关内容
异步批量请求
在单个异步请求中爬取数千个帖子或运行关键词发现任务。
API 参考
完整端点规范、参数和响应模式。