Analyzing Scraping Browser Traffic & Resources

The Control Panel provides a user-friendly interface to view the resource types and data usage for your browser sessions. Within your specific Scraping Browser zone, click on the Statistics tab.

This custom CDP command allows you to monitor resource types and data usage in real-time during your browser session:

  // Create a new page
  const page = await browser.newPage();
  
  // Establish a new CDP session
  const client = await page.createCDPSession();
  
  // Navigate to the desired webpage
  await page.goto('https://google.com', { waitUntil: 'networkidle2' });
  
  // Custom CDP command to get resource type statistics
  const { stats } = await client.send('Network.getResourceTypeStats');
  
  // Output the statistics
  console.log(stats);

The Network.getResourceTypeStats command returns an array of objects, each representing a different type of resource loaded by the webpage.

[
  { "resourceType": "Document", "data": 77776 },
  { "resourceType": "Stylesheet", "data": 2721 },
  { "resourceType": "Script", "data": 742028 },
  { "resourceType": "Image", "data": 33540 },
  { "resourceType": "XHR", "data": 4244 },
  { "resourceType": "Fetch", "data": 97329 },
  { "resourceType": "Font", "data": 68304 },
  { "resourceType": "Other", "data": 1918 }
]

Description
Various document types are loaded by the browser.

Examples
HTML, XML, Document, JSON, PDF, SVG, Markdown, TXT, CSV, DOC, DOCX, ODT, etc

Significance
Essential for rendering the primary structure and content of the web page. Typically minimal in size but crucial for page functionality.

Description
Files that are downloaded by you during the session.

Examples
Download

Significance
Can be substantial in size, directly affecting data consumption.

Description
Scripts and resources used for dynamic content.

Examples
XHR, Fetch, Script, Manifest, Module, Worker, etc

Significance
Often significant in size, especially in complex web applications.

Description
Multimedia content displayed on a webpage.

Examples
Image (JPEG, PNG, GIF, WebP, APNG, ICO, TIFF, BMP), Video, Audio, Font, etc

Significance
Typically one of the largest contributors to data usage.

Description
CSS files for styling web pages.

Examples
Stylesheet

Significance
Generally small but vital for user experience and visual consistency.

Description
Miscellaneous resources and data exchanges.

Examples
TextTrack, Prefetch, EventSource, WebSocket, SignedExchange, Ping, CSPViolationReport, Preflight, ServiceWorker, Beacon, Other, etc

Significance
Typically small in size but important for various functionalities and optimizations. Useful to monitor these for any unexpected data usage.