> ## Documentation Index > Fetch the complete documentation index at: https://docs.brightdata.com/llms.txt > Use this file to discover all available pages before exploring further. # Data Enrichment > Build AI agents that fill CRM fields, enrich leads and complete records using the Bright Data search-and-extract pattern (660+ scrapers) for high-volume runs. You might be eligible for our Startup Program. Get fully funded access to the infrastructure you're reading about right now (up to \$20K value). # Data Enrichment Build AI agents that automatically fill CRM data, enrich leads, and complete customer records at enterprise scale. Master the search-and-extract pattern for enrichment operations, from LinkedIn company data collection to lead scoring workflows. Understand the enrichment workflow Start with a LinkedIn example *** ## Why Standard Scraping Falls Short \~50% success rate on protected sites like LinkedIn due to anti-bot measures Slow SERP responses (2-5 seconds) limit throughput Rate limits and IP bans break batch processing at scale Manual proxy management increases operational risk Unreliable beyond 1K concurrent enrichment jobs 95%+ enrichment success, including on protected sites 50K+ concurrent extractions with 99.99% uptime Automated proxy rotation, unblocking, and CAPTCHA solving GDPR/CCPA-ready with enterprise-grade security controls Global proxy network (400M+ monthly residential IPs) for broad market coverage *** ## How to handle enrichment complexity Address common challenges in enrichment systems: * **LinkedIn's aggressive anti-bot measures** - Automatically bypass with Web Unlocker * **CAPTCHA challenges** - Automatic CAPTCHA solving with no manual intervention * **Rate limiting** - Intelligent rate management and proxy rotation * **Data quality issues** - Built-in validation and error handling Bright Data's infrastructure solves these with automatic CAPTCHA solving, intelligent rate management, and production-ready reliability. Never get blocked by CAPTCHAs or bot detection Intelligent rate limiting and proxy rotation Built-in validation ensures data quality Robust error handling for production reliability *** ## How to scale enrichment Scale from enriching hundreds of leads to processing millions of records with the same infrastructure. Built for enrichment patterns like: * **Parallel processing** for throughput * **Error handling** for reliability * **Data validation** for quality Process thousands of leads simultaneously with enterprise-scale infrastructure Robust error handling ensures reliability at scale Built-in validation ensures high-quality enriched data *** ## The Enrichment Pattern The enrichment pattern typically follows these steps: 1. **Input** - Receive a list of leads or records that need enrichment 2. **Search** - Search for each lead using SERP API or web scraping 3. **Extract** - Extract relevant data from search results 4. **Validate** - Validate the extracted data for quality 5. **Enrich** - Add the enriched data to your CRM or database 6. **Monitor** - Monitor success rates and data quality Prepare your list of leads or records that need enrichment. Include identifiers like company names, domains, or email addresses. ```json theme={null} [ { "company_name": "Example Corp", "domain": "example.com", "email": "contact@example.com" } ] ``` Use SERP API or web scraping to search for each lead and find relevant information. ```javascript Node.js theme={null} const response = await fetch('https://api.brightdata.com/datasets/v3/trigger?dataset_id=YOUR_DATASET_ID', { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify([{ url: 'https://www.google.com/search', keyword: 'Example Corp company information', country: 'US' }]) }); ``` ```python Python theme={null} import requests response = requests.post( 'https://api.brightdata.com/datasets/v3/trigger?dataset_id=YOUR_DATASET_ID', headers={ 'Authorization': f'Bearer {api_key}', 'Content-Type': 'application/json' }, json=[{ 'url': 'https://www.google.com/search', 'keyword': 'Example Corp company information', 'country': 'US' }] ) ``` Extract relevant data from search results and validate for quality. Use data validation endpoints to ensure the extracted data meets your quality standards. Add the enriched data to your CRM or database. Successfully enriched leads are saved with complete contact information. *** ## Contact Enrichment - LinkedIn Example Enrich leads with LinkedIn company data: ### Step 1: Search LinkedIn Search for company information on LinkedIn: ```javascript Node.js theme={null} const response = await fetch('https://api.brightdata.com/datasets/v3/trigger?dataset_id=YOUR_LINKEDIN_DATASET_ID', { method: 'POST', headers: { 'Authorization': `Bearer ${apiKey}`, 'Content-Type': 'application/json' }, body: JSON.stringify([{ url: 'https://www.linkedin.com/company/example-corp', company_name: 'Example Corp' }]) }); ``` ```python Python theme={null} import requests response = requests.post( 'https://api.brightdata.com/datasets/v3/trigger?dataset_id=YOUR_LINKEDIN_DATASET_ID', headers={ 'Authorization': f'Bearer {api_key}', 'Content-Type': 'application/json' }, json=[{ 'url': 'https://www.linkedin.com/company/example-corp', 'company_name': 'Example Corp' }] ) ``` ### Step 2: Extract Company Data Extract company information: ```json theme={null} { "company_name": "Example Corp", "industry": "Technology", "employee_count": "1000-5000", "location": "San Francisco, CA", "website": "https://example.com", "description": "Leading technology company..." } ``` ### Step 3: Enrich Your CRM Add the enriched data to your CRM: ```javascript Node.js theme={null} // Add enriched data to your CRM await crm.addContact({ company_name: enrichedData.company_name, industry: enrichedData.industry, employee_count: enrichedData.employee_count, location: enrichedData.location, website: enrichedData.website }); ``` ```python Python theme={null} # Add enriched data to your CRM crm.add_contact( company_name=enriched_data['company_name'], industry=enriched_data['industry'], employee_count=enriched_data['employee_count'], location=enriched_data['location'], website=enriched_data['website'] ) ``` *** ## How to process leads in bulk Process large volumes of leads efficiently: ### How to process in parallel Process multiple leads simultaneously: ```javascript Node.js theme={null} const leads = [/* array of leads */]; const enrichmentPromises = leads.map(lead => enrichLead(lead) ); const enrichedLeads = await Promise.all(enrichmentPromises); ``` ```python Python theme={null} import asyncio leads = [/* list of leads */] async def enrich_lead(lead): # Enrichment logic pass async def enrich_all_leads(): tasks = [enrich_lead(lead) for lead in leads] enriched_leads = await asyncio.gather(*tasks) return enriched_leads ``` ### How to process in batches Process leads in batches to manage rate limits: ```javascript Node.js theme={null} async function processBatch(leads, batchSize = 10) { for (let i = 0; i < leads.length; i += batchSize) { const batch = leads.slice(i, i + batchSize); await Promise.all(batch.map(lead => enrichLead(lead))); // Wait between batches to respect rate limits await new Promise(resolve => setTimeout(resolve, 1000)); } } ``` ```python Python theme={null} import time def process_batch(leads, batch_size=10): for i in range(0, len(leads), batch_size): batch = leads[i:i + batch_size] for lead in batch: enrich_lead(lead) # Wait between batches to respect rate limits time.sleep(1) ``` *** ## Common Data Sources Company and professional data from LinkedIn Search results for company information and news Extract company information directly from websites Social media profiles and engagement data *** ## How to handle errors Implement robust error handling for production reliability: ```javascript Node.js theme={null} async function enrichLeadWithRetry(lead, maxRetries = 3) { for (let attempt = 1; attempt <= maxRetries; attempt++) { try { const result = await enrichLead(lead); return result; } catch (error) { if (attempt === maxRetries) { throw error; } // Exponential backoff await new Promise(resolve => setTimeout(resolve, Math.pow(2, attempt) * 1000)); } } } ``` ```python Python theme={null} import time def enrich_lead_with_retry(lead, max_retries=3): for attempt in range(1, max_retries + 1): try: result = enrich_lead(lead) return result except Exception as error: if attempt == max_retries: raise error # Exponential backoff time.sleep(2 ** attempt) ``` *** ## Pre-built enrichment templates Use pre-built templates for common enrichment workflows: Template for enriching leads with LinkedIn company data Template for validating and enriching email addresses Template for enriching contact information Template for collecting company intelligence data *** ## Additional Use Cases ### Product Catalog Enrichment Enrich product databases with up-to-date market data: 1. Ingest incomplete product records (SKU, name, category). 2. Scrape pricing, competitor data, and availability from live sources using [Web Unlocker](/scraping-automation/web-unlocker/introduction). 3. Merge and deduplicate entries across multiple vendors. 4. Generate enriched product descriptions, specifications, and metadata. 5. Sync enriched catalogs to inventory and recommendation systems. ### Research Data Normalization Standardize and complete research datasets at scale: 1. Ingest raw data from multiple CSV files or APIs. 2. Extract and validate company information, rankings, or trends using [SERP API](/scraping-automation/serp-api/introduction). 3. Normalize inconsistent formats and fill missing values. 4. Enrich with external APIs and verified data sources. 5. Output clean, deduplicated datasets for analytics pipelines and ML models. *** ## Next Steps Start collecting search results for enrichment Use pre-built LinkedIn scrapers for company data Use Deep Lookup for comprehensive data enrichment Explore pre-built scrapers for common data sources **Need help?** Check out our [Data Validation Guide](/datasets/data-validation/data-validation-for-customers) or [contact support](https://brightdata.com/contact).