X (Twitter) Scraper API endpoints

This guide shows you how to send a synchronous request to each Bright Data X Scraper API endpoint. It covers profiles and posts.

Prerequisites

A Bright Data account with an active API key
Completed the Quickstart

Request structure

Every synchronous request follows the same pattern:

POST https://api.brightdata.com/datasets/v3/scrape?dataset_id={DATASET_ID}&format=json
Authorization: Bearer YOUR_API_KEY
Content-Type: application/json

[{"url": "https://x.com/..."}]

The only thing that changes between endpoints is the dataset_id and the input URL format.

The /scrape endpoint is built for real-time, single-URL requests. Bright Data recommends 20 URLs or fewer per request, a best practice rather than a hard limit, and the endpoint waits up to 1 minute. If collection takes longer, the response is HTTP 202 with a snapshot_id instead of data, and collection continues asynchronously. Fetch the result with async requests or have it pushed to you with webhook delivery.

How to scrape X (Twitter) profiles

Dataset ID: gd_lwxmeb2u1cniijd7t4

curl -X POST \
  "https://api.brightdata.com/datasets/v3/scrape?dataset_id=gd_lwxmeb2u1cniijd7t4&format=json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[{"url": "https://x.com/elonmusk"}]'

You should see a 200 response. Processing time varies with target-site conditions and scraper workload.

Example response

[
  {
    "followers": 241583960,
    "following": 1403,
    "is_verified": true,
    "profile_image_link": "https://pbs.twimg.com/profile_images/2053244804520427520/m8mdWZCG_400x400.jpg",
    "x_id": "44196397",
    "url": "https://x.com/elonmusk",
    "id": "elonmusk",
    "profile_name": "Elon Musk",
    "biography": "https://t.co/ZdBx5WABYx",
    "date_joined": "2009-06-02T20:12:29.000Z"
  }
]

Full Profiles response schema

Which output fields does the X (Twitter) profiles scraper return?

The X (Twitter) profiles scraper (gd_lwxmeb2u1cniijd7t4, collect_by_url) returns 22 output fields.

Output fields (22)

Field	Type	Description
`x_id`	text	Unique X id of each profile
`url`	url	Profile URL
`id`	text	ID of the post
`profile_name`	text	Profile username
`biography`	text	Bio of the profile
`is_verified`	boolean	Is the user verified
`profile_image_link`	url	Profile Image link
`external_link`	url	External link
`date_joined`	date	Date where the profile joined twitter
`following`	number	Number of profiles the user follows
`followers`	number	Number of followers of the profile
`subscriptions`	number	Number of subscriptions of the profile
`location`	text	Profile location
`birth_date`	date	Profile birth date
`posts_count`	number	Number of profile’s posts
`posts`	array
`suggested_profiles`	array	Suggested profiles
`is_business_account`	boolean	Is the profile a business account
`is_government_account`	boolean	Is it a government account
`category_name`	array	Category names
`max_number_of_posts`	number	The number of posts input from discovery
`banner_image`	url	The profile’s banner image

Machine-readable version with the input schema: /scrapers/gd_lwxmeb2u1cniijd7t4.json

How to scrape X (Twitter) posts

Dataset ID: gd_lwxkxvnf1cynvib9co

curl -X POST \
  "https://api.brightdata.com/datasets/v3/scrape?dataset_id=gd_lwxkxvnf1cynvib9co&format=json" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '[{"url": "https://x.com/elonmusk/status/1519480761749016577"}]'

Example response

[
  {
    "url": "https://x.com/elonmusk/status/1519480761749016577",
    "user_posted": "elonmusk",
    "description": "Next I’m buying Coca-Cola to put the cocaine back in",
    "date_posted": "2022-04-28T00:56:58.000Z",
    "likes": 4196406,
    "replies": 167707,
    "hashtags": null,
    "id": "1519480761749016577",
    "name": "Elon Musk",
    "reposts": 577446
  }
]

Full Posts response schema

Which output fields does the X (Twitter) posts scraper return?

The X (Twitter) posts scraper (gd_lwxkxvnf1cynvib9co, collect_by_url) returns 32 output fields.

Output fields (32)

Field	Type	Description
`id`	text	ID of the post
`user_posted`	text	Username of the post owner
`name`	text	Name of the post owner
`description`	text	Post text description
`date_posted`	date	Post publication date
`photos`	array	URLs of attached photos
`url`	url	URL of the post
`quoted_post`	object	Quoted post details
`tagged_users`	array	Tagged profiles from the post
`replies`	number	Number of replies
`reposts`	number	Number of reposts; There may be a difference between the value displayed on the web interface and the actual value collected from the server side, where the data is retrieved
`likes`	number	Number of likes
`views`	number	Number of views
`external_url`	url	External URL from a post
`hashtags`	array	Hashtags from the post
`followers`	number	Number of followers of the profile
`biography`	text	Bio of the profile
`posts_count`	number	Number of profile’s posts
`profile_image_link`	url	Profile image link
`following`	number	Number of profiles the user follows
`is_verified`	boolean	Is the user verified
`quotes`	number	Number of times the post was quoted
`bookmarks`	number	The number of times the post was bookmarked
`parent_post_details`	object	Details of the parent post this post is replying to
`external_image_urls`	array	The URL for the images attached to the post
`videos`	array	Videos attached to the post
`external_video_urls`	array	URLs of externally hosted videos attached to the post
`verification_type`	text	The type of verification badge assigned to the user’s profile (e.g. blue check, organization)
`user_id`	text	The user ID
`context_added`	text	Context added by the readers
`is_repost`	boolean	True when the post is a repost
`ai_generated_images`	array	List of images attached to the post, each with its URL and AI generation label

Machine-readable version with the input schema: /scrapers/gd_lwxkxvnf1cynvib9co.json

Quick reference: dataset IDs

Endpoint	Dataset ID	URL pattern
Profiles	`gd_lwxmeb2u1cniijd7t4`	`x.com/{username}`
Posts	`gd_lwxkxvnf1cynvib9co`	`x.com/{username}/status/{id}`

Output formats

Control the response format with the format query parameter:

Value	Description
`json`	JSON array
`ndjson`	Newline-delimited JSON, one record per line (default)
`csv`	Comma-separated values

These are the format values for /scrape. The full list, NDJSON (default), JSON, JSON lines, CSV and Parquet, and where each applies is in output formats.

Async batch requests

Scrape hundreds of URLs in a single batch job.

API reference

Full parameter and response field reference.

Get started

Data extraction

Web search

AI training data

Proxy networks

X (Twitter) Scraper API endpoints

Prerequisites

Request structure

How to scrape X (Twitter) profiles

Which output fields does the X (Twitter) profiles scraper return?

How to scrape X (Twitter) posts

Which output fields does the X (Twitter) posts scraper return?

Quick reference: dataset IDs

Output formats

Async batch requests

API reference

​Prerequisites

​Request structure

​How to scrape X (Twitter) profiles

​Which output fields does the X (Twitter) profiles scraper return?

​How to scrape X (Twitter) posts

​Which output fields does the X (Twitter) posts scraper return?

​Quick reference: dataset IDs

​Output formats

​Related

Async batch requests

API reference

Prerequisites

Request structure

How to scrape X (Twitter) profiles

Which output fields does the X (Twitter) profiles scraper return?

How to scrape X (Twitter) posts

Which output fields does the X (Twitter) posts scraper return?

Quick reference: dataset IDs

Output formats

Related