How to Configure the Browser API

To get started, grab your credentials - the Username and Password you will use with your web automation tool. You can find them in the Browser API zone you just created, in the “Overview” tab. We assume that you aleady have your preferred web automation tool installed. If not, please instal it.

Browser API Quick Start Examples

Run these basic examples to check that your Browser API is working (remember to swap in your credentials and target URL):

For advanced examples, such as handling captchas or implementing live DevTools, see the Code Examples section.

const puppeteer = require('puppeteer-core');  
// Enter your zone name and password below
const AUTH = 'SBR_ZONE_FULL_USERNAME:SBR_ZONE_PASSWORD';  
const SBR_WS_ENDPOINT = `wss://${AUTH}@brd.superproxy.io:9222`;  
  
async function main() {  
    console.log('Connecting to Browser API...');  
    const browser = await puppeteer.connect({  
        browserWSEndpoint: SBR_WS_ENDPOINT,  
   });  
    try {  
        console.log('Connected! Navigating...');  
        const page = await browser.newPage();
        // Enter your test URL below
        await page.goto('https://example.com', { timeout: 2 * 60 * 1000 });  
        console.log('Taking screenshot to page.png');  
        await page.screenshot({ path: './page.png', fullPage: true });  
   		console.log('Navigated! Scraping page content...');  
 		const html = await page.content();  
 		console.log(html)   
    } finally {  
        await browser.close();  
   }  
}  
  
if (require.main === module) {  
    main().catch(err => {  
        console.error(err.stack || err);  
        process.exit(1);  
   });  
}

const puppeteer = require('puppeteer-core');  
// Enter your zone name and password below
const AUTH = 'SBR_ZONE_FULL_USERNAME:SBR_ZONE_PASSWORD';  
const SBR_WS_ENDPOINT = `wss://${AUTH}@brd.superproxy.io:9222`;  
  
async function main() {  
    console.log('Connecting to Browser API...');  
    const browser = await puppeteer.connect({  
        browserWSEndpoint: SBR_WS_ENDPOINT,  
   });  
    try {  
        console.log('Connected! Navigating...');  
        const page = await browser.newPage();
        // Enter your test URL below
        await page.goto('https://example.com', { timeout: 2 * 60 * 1000 });  
        console.log('Taking screenshot to page.png');  
        await page.screenshot({ path: './page.png', fullPage: true });  
   		console.log('Navigated! Scraping page content...');  
 		const html = await page.content();  
 		console.log(html)   
    } finally {  
        await browser.close();  
   }  
}  
  
if (require.main === module) {  
    main().catch(err => {  
        console.error(err.stack || err);  
        process.exit(1);  
   });  
}

import asyncio
from playwright.async_api import async_playwright

AUTH = 'SBR_ZONE_FULL_USERNAME:SBR_ZONE_PASSWORD'
SBR_WS_CDP = f'wss://{AUTH}@brd.superproxy.io:9222'

async def run(pw):
    print('Connecting to Browser API...')
    browser = await pw.chromium.connect_over_cdp(SBR_WS_CDP)
    try:
        print('Connected! Navigating...')
        page = await browser.new_page()
        await page.goto('https://example.com', timeout=2*60*1000)
        print('Taking page screenshot to file page.png')
        await page.screenshot(path='./page.png', full_page=True)
        print('Navigated! Scraping page content...')
        html = await page.content()
        print(html)
    finally:
        await browser.close()

async def main():
    async with async_playwright() as playwright:
        await run(playwright)

if __name__ == '__main__':
    asyncio.run(main())

using PuppeteerSharp;  
using System.Net.WebSockets;  
using System.Text;  
  
var AUTH = "SBR_ZONE_FULL_USERNAME:SBR_ZONE_PASSWORD";  
var SBR_WS_ENDPOINT = $"wss://{AUTH}@brd.superproxy.io:9222";  
  
var Connect = (string ws) => Puppeteer.ConnectAsync(new ()  
{  
    BrowserWSEndpoint = ws,  
    WebSocketFactory = async (url, options, cToken)=>{  
        var socket = new ClientWebSocket();  
        var authBytes = Encoding.UTF8.GetBytes(new Uri(ws).UserInfo);  
        var authHeader = "Basic " + Convert.ToBase64String(authBytes);  
        socket.Options.SetRequestHeader("Authorization", authHeader);  
        socket.Options.KeepAliveInterval = TimeSpan.Zero;  
        await socket.ConnectAsync(url, cToken);  
        return socket;  
   },  
});  
  
Console.WriteLine("Connecting to Browser API...");  
using var browser = await Connect(SBR_WS_ENDPOINT);  
Console.WriteLine("Connected! Navigating...");  
var page = await browser.NewPageAsync();  
await page.GoToAsync("https://example.com", new NavigationOptions()  
{  
    Timeout = 2 * 60 * 1000,  
});  
Console.WriteLine("Taking page screenshot to file page.png");  
await page.ScreenshotAsync("./page.png", new ()  
{  
    FullPage = true,  
});  
Console.WriteLine("Navigated! Scraping page content...");  
var html = await page.GetContentAsync();   
Console.WriteLine(html);

Run the Script

Save the above code as script.js (don’t forget to enter your credentials!) and run it using this command:

node script.js

Save the above code as script.js (don’t forget to enter your credentials!) and run it using this command:

node script.js

Save the above code as main.py (don’t forget to enter your credentials!) and run it using this command:

python main.py

View live browser session

The Browser API Debugger enables developers to inspect, analyze, and fine-tune their code alongside Chrome Dev Tools, resulting in better control, visibility, and efficiency. You can integrate the following code snippet to launch devtools automatically for every session:

// Node.js Puppeteer - launch devtools locally  

// Add these requirements in addition to the other required imports
const { exec } = require('child_process');  
const chromeExecutable = 'google-chrome';  

// Add these functions to the top of the script  
const delay = ms => new Promise(resolve => setTimeout(resolve, ms));  
const openDevtools = async (page, client) => {  
    // get current frameId  
    const frameId = page.mainFrame()._id;  
    // get URL for devtools from Browser API  
    const { url: inspectUrl } = await client.send('Page.inspect', { frameId });  
    // open devtools URL in local chrome  
    exec(`"${chromeExecutable}" "${inspectUrl}"`, error => {  
        if (error)  
            throw new Error('Unable to open devtools: ' + error);  
    });  
    // wait for devtools ui to load  
    await delay(5000);  
};  

// Add these lines before navigating to the target URL  
const page = await browser.newPage();  
const client = await page.target().createCDPSession();  
await openDevtools(page, client);  
await page.goto('http://example.com');

Browser API sessions are structured to allow one initial navigation per session. This initial navigation refers to the first instance of loading the target site from which data is to be extracted. Following this, users are free to navigate the site using clicks, scrolls, and other interactive actions within the same session. However, to start a new scraping job, either on the same site or a different one, from the initial navigation stage, it is necessary to begin a new session.

Session Time Limits

Browser API has 2 kinds of timeouts aimed to safeguard our customers from uncontrolled usage.

Idle Session Timeout: in case a browser session is kept open for 5 minutes and above in an idle mode, meaning no usage going through it, Browser API will automatically timeout the session.
Maximum Session Length Timeout: Browser API session can last up to 30 minutes. Once the maximum session time is reached the session will automatically timeout.

Introduction

Product Guides

How to Configure the Browser API

Browser API Quick Start Examples

Run the Script

View live browser session

Single Navigation Per Session

Session Time Limits

Introduction

Product Guides

​Browser API Quick Start Examples

​Run the Script

​View live browser session

​Single Navigation Per Session

​Session Time Limits

Browser API Quick Start Examples

Run the Script

View live browser session

Single Navigation Per Session

Session Time Limits