Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.brightdata.com/llms.txt

Use this file to discover all available pages before exploring further.

Building an AI startup?

You might be eligible for our Startup Program. Get fully funded access to the infrastructure you’re reading about right now (up to $20K value).
A comprehensive web scraping and data extraction plugin powered by Bright Data’s enterprise-grade infrastructure with intelligent auto-detection. Supports 50+ platforms including Amazon, LinkedIn, Instagram, YouTube, and more.
image
For the most up-to-date updates, please refer to this Repository

​
Available Tools

​
Structured Data Feeds

Extract structured data from popular platforms:
  • E-commerce: Amazon, eBay, Walmart, Best Buy, Etsy, Zara
  • Social Media: Instagram, Facebook, TikTok, YouTube, X (Twitter)
  • Professional: LinkedIn profiles, companies, jobs
  • Business: Crunchbase, ZoomInfo
  • Maps & Reviews: Google Maps, booking sites
  • News: News sources and articles

​
Scrape As Markdown

Convert any webpage into clean, readable markdown format perfect for:
  • Content analysis
  • Documentation extraction
  • Article processing

​
Search Engine

Get search results from major search engines:
  • Google
  • Bing
  • Yandex, etc.

​
Use Cases

  • E-commerce Monitoring: Track product prices and availability
  • Lead Generation: Extract business information from LinkedIn
  • Content Research: Gather articles and news for analysis
  • Market Research: Monitor competitor websites and social media
  • SEO Analysis: Track search engine results and rankings

​
How to Integrate Bright Data With Dify

1

Install the plugin

Install Bright Data Plugin from Dify Marketplace
2

Obtain Your Bright Data API Key

3

Create Your First Workflow

  1. Go to Dify Studio → Workflow
  2. Add one of the Bright Data Web Scraper tools:
  • Structured Data Feeds - Extract structured data from 20+ platforms
  • Scrape As Markdown - Convert any webpage to clean markdown
  • Search Engine - Get search results from Google, Bing, Yandex
  1. Enter your Bright Data API key when prompted
  2. You can connect an LLM node to process and summarize the scraped data
4

Example Workflow

see workflow in banner image
Sample Use Case: Extract Amazon product information and create a summary
  1. START → Input: Product URL
  2. STRUCTURED DATA FEEDS → Extract product details
  3. LLM → Summarize into easy-to-read text
  4. END → Output: Clean product summary

​
Important tips

  • Reference every stage of the workflow to the output of the previous stage
  • Set a high character limit in input fields (for the URL input field choose the “short paragraph” var option)

​
Advanced Option: Use Bright Data MCP

While the Bright Data plugin for Dify uses hosted APIs, advanced users can also integrate directly with Bright Data MCP (Model Context Protocol), a local server that exposes Bright Data’s full scraping and automation toolset via HTTP. You can call MCP tools from Dify using custom HTTP requests or external service nodes to unlock advanced capabilities like browser automation, structured data extraction, and real-time scraping.