Custom Scrapers
Custom Scrapers FAQs
What is Bright Data’s Custom Scrapers product?
What is Bright Data’s Custom Scrapers product?
Custom Scrapers is a tailored web scraping solution that allows you to extract structured data from any website. It’s designed for cases where Bright Data’s generic scrapers don’t meet your specific needs. You can choose to lead the project yourself (Custom Scraper route) or let Bright Data manage everything for you (Fully Managed route).
Who is Custom Scrapers for?
Who is Custom Scrapers for?
Custom Scrapers is ideal for:
- Businesses needing data from specific or complex websites
- Users with unique data requirements not covered by generic scrapers
- Enterprises looking for scalable, reliable, and accurate data extraction
- Teams that want either full control or a fully managed experience
What types of websites can Custom Scrapers handle?
What types of websites can Custom Scrapers handle?
Custom Scrapers can extract data from virtually any website, including:
- E-commerce platforms
- Real estate portals
- Job boards
- Social media sites
- Travel aggregators
- Business directories
- Sites with dynamic content, CAPTCHA, or anti-bot protections
How much does it cost?
How much does it cost?
We offer 2 pricing options:
| Option | Price | Additional information |
|---|---|---|
| One-time Project | $1200 starts at $4/CPM | A single delivery. If more records are needed, you will need to add funds to your balance. |
| Ongoing Project | $400/month starts at $4/CPM | 3-month minimum commitment |
Product Options
What’s the difference between the Custom Scraper and Fully Managed routes?
What’s the difference between the Custom Scraper and Fully Managed routes?
| Feature | Custom Scraper (Customer-Led) | Fully Managed Solution (Bright Data-Led) |
|---|---|---|
| Who defines the project? | You | Bright Data |
| Who manages development? | Bright Data (based on your specs) | Bright Data |
| Control over schema | Full control – you define and approve | Collaborative – we define based on your goals |
| Setup process | Submit form → Schedule validation meeting | Submit form → Bright Data contacts you |
| Ideal for | Users with clear requirements and technical input | Users who want a hands-off experience |
| Time to launch | Depends on complexity and customer readiness | Typically faster due to internal handling |
| Support | Ongoing support available | Full support and proactive management |
How do I choose between the two options?
How do I choose between the two options?
Choose the Custom Scraper route if:
- You have a clear idea of what data you need
- You want control over the schema and scraping logic
- You prefer to be hands-on during the setup
- You want Bright Data to handle everything
- You prefer a hands-off experience
- You’re looking for fast, expert-driven delivery
Process & Setup
How do I get started with Custom Scrapers?
How do I get started with Custom Scrapers?
- Choose your preferred route (Custom or Fully Managed)
- Submit your project details via the request form
- For Custom Scraper: Schedule a validation meeting with a Bright Data expert
- Bright Data builds and deploys your scraper
- Receive your data in your preferred format
What information do I need to provide?
What information do I need to provide?
For Custom Scraper:
- Target URLs or domains
- Scope of the project (e.g., frequency, volume)
- Desired data schema (fields to extract)
- Preferred output format (e.g., JSON, CSV)
- High-level project goals
- Types of data needed
- Any specific use cases or business context
What happens during the validation meeting?
What happens during the validation meeting?
In the Custom Scraper route, the validation meeting is where you and a Bright Data representative:
- Review your request
- Confirm the scope and complexity
- Finalize the data schema
- Align on timelines and next steps
Technical & Delivery
Do I need technical skills to use Custom Scrapers?
Do I need technical skills to use Custom Scrapers?
No. Bright Data’s team handles all the technical aspects—including scraper development, deployment, proxy management, and error handling. You simply define what you need, and we deliver the data.
What output formats are supported?
What output formats are supported?
We support multiple formats, including:
- JSON
- CSV
- Excel
- API integration (optional)
Can I schedule recurring data extractions?
Can I schedule recurring data extractions?
Yes. Custom Scrapers supports both one-time and recurring scraping jobs. You can define the frequency (e.g., daily, weekly, monthly) based on your needs.
Is the data validated or cleaned?
Is the data validated or cleaned?
Yes. We ensure the data is clean, structured, and ready to use. Our scrapers are built to deliver reliable and accurate results.
Performance & Support
How long does it take to build a custom scraper?
How long does it take to build a custom scraper?
Timelines vary depending on the complexity of the website and the scope of your request. Simpler projects may be completed in a few days, while more complex ones may take longer. The validation meeting helps clarify timelines.
What if the website structure changes?
What if the website structure changes?
Bright Data offers ongoing support and maintenance. If the target website changes, let us know and we can update the scraper to ensure continued data accuracy and functionality.
Is there a limit to how much data I can extract?
Is there a limit to how much data I can extract?
Custom Scrapers is built to scale. Whether you need data from a few pages or millions, we can accommodate your volume and frequency requirements.