A comprehensive web scraping and data extraction plugin powered by Bright Data’s enterprise-grade infrastructure with intelligent auto-detection. Supports 50+ platforms including Amazon, LinkedIn, Instagram, YouTube, and more.

Available Tools

Structured Data Feeds

Extract structured data from popular platforms:

  • E-commerce: Amazon, eBay, Walmart, Best Buy, Etsy, Zara
  • Social Media: Instagram, Facebook, TikTok, YouTube, X (Twitter)
  • Professional: LinkedIn profiles, companies, jobs
  • Business: Crunchbase, ZoomInfo
  • Maps & Reviews: Google Maps, booking sites
  • News: Reuters and other news sources

Scrape As Markdown

Convert any webpage into clean, readable markdown format perfect for:

  • Content analysis
  • Documentation extraction
  • Article processing

Search Engine

Get search results from major search engines:

  • Google
  • Bing
  • Yandex, etc.

Use Cases

  • E-commerce Monitoring: Track product prices and availability
  • Lead Generation: Extract business information from LinkedIn
  • Content Research: Gather articles and news for analysis
  • Market Research: Monitor competitor websites and social media
  • SEO Analysis: Track search engine results and rankings

How to Integrate Bright Data With Dify

1

Requirements

2

Obtain Your Bright Data API Key

3

Download the Plugin

Download the latest plugin package: brightdata_plugin.difypkg

4

Install in Dify

  1. Go to Dify.aiPluginsInstall Plugin
  2. Choose Choose Local Package File
  3. Upload the brightdata_plugin.difypkg file
5

Create Your First Workflow

  1. Go to Dify StudioWorkflow
  2. Add one of the Bright Data Web Scraper tools:
  • Structured Data Feeds - Extract structured data from 20+ platforms
  • Scrape As Markdown - Convert any webpage to clean markdown
  • Search Engine - Get search results from Google, Bing, Yandex
  1. Enter your Bright Data API key when prompted
  2. You can connect an LLM node to process and summarize the scraped data
6

Example Workflow

see workflow in banner image
Sample Use Case: Extract Amazon product information and create a summary

  1. START → Input: Product URL
  2. STRUCTURED DATA FEEDS → Extract product details
  3. LLM → Summarize into easy-to-read text
  4. END → Output: Clean product summary

Important tips

  • Reference every stage of the workflow to the output of the previous stage
  • Set a high character limit in input fields (for the URL input field choose the “short paragraph” var option)