Analyzing Scraping Browser Traffic & Resources

When scraping the web with Scraping Browser, various resources are loaded on every page load (like any browser). Understanding this traffic is crucial for optimizing browsing and managing data usage. You can analyze these resources using the Control Panel UI or via a custom Bright Data CDP command.

This custom CDP command allows you to monitor resource types and data usage in real-time during your browser session:

NodeJS - Example
  // Create a new page
  const page = await browser.newPage();
  
  // Establish a new CDP session
  const client = await page.createCDPSession();
  
  // Navigate to the desired webpage
  await page.goto('https://google.com', { waitUntil: 'networkidle2' });
  
  // Custom CDP command to get resource type statistics
  const { stats } = await client.send('Network.getResourceTypeStats');
  
  // Output the statistics
  console.log(stats);

The Network.getResourceTypeStats command returns an array of objects, each representing a different type of resource loaded by the webpage.

Sample Output
[
  { "resourceType": "Document", "data": 77776 },
  { "resourceType": "Stylesheet", "data": 2721 },
  { "resourceType": "Script", "data": 742028 },
  { "resourceType": "Image", "data": 33540 },
  { "resourceType": "XHR", "data": 4244 },
  { "resourceType": "Fetch", "data": 97329 },
  { "resourceType": "Font", "data": 68304 },
  { "resourceType": "Other", "data": 1918 }
]

Resource Types Explained

The retrieved resources can be categorized as follows:

​Resource Types Explained

Resource Types Explained