Tutorials 4 min read

Headless Browser Proxies: Sessions, Bandwidth and Stability at Scale (2026)

Headless browsers are the most bandwidth-hungry way to use a proxy. Here is how to keep sessions stable and the GB bill down.

Headless browser proxies cost more and fail in different ways than proxies behind a plain HTTP client. A headless browser loads every script, image, font and tracking pixel on the page, opens many connections at once, and keeps sessions alive. That means three things matter more than with requests or curl: session design, bandwidth and timeouts. This guide covers all three with Playwright for Python, and the ideas carry straight over to Puppeteer and Selenium.

Do You Need a Headless Browser at All?

Before anything else: if the data is in the HTML or in a JSON endpoint the page calls, fetch it directly with an HTTP client. It's often 10 to 50 times less traffic. Open the browser's network tab, find the API call that returns the data, and request that. Use a headless browser only when the page truly needs JavaScript to render or to pass a challenge. How to use proxies for web scraping covers the lighter approach.

One Context, One Sticky Session

A single page load can make 100+ requests. If each leaves from a different IP, sites see a "user" whose address changes mid-page, which is a strong bot signal. Give each browser context its own sticky session and reuse it for the pages in that task:

import uuid
from playwright.sync_api import sync_playwright

def proxy_for(country='us', minutes=10):
    sid = uuid.uuid4().hex[:8]
    return {
        'server': 'http://global.nodeproxies.xyz:8080',
        'username': f'USERNAME-country-{country}-session-{sid}-time-{minutes}',
        'password': 'PASSWORD',
    }

with sync_playwright() as p:
    browser = p.chromium.launch(headless=True)
    context = browser.new_context(proxy=proxy_for())
    page = context.new_page()
    page.goto('https://api.ipify.org')
    print(page.inner_text('body'))
    context.close()
    browser.close()

Pick a session time longer than your task. When the task ends, close the context; the next task gets a fresh id and a fresh IP. The Node.js version is in Playwright proxy setup in Node.js.

Cut Bandwidth by Blocking Resources

Images, video and fonts are usually most of a page's weight and rarely needed for data. Block them at the context level:

BLOCK = {'image', 'media', 'font'}

def handle(route):
    if route.request.resource_type in BLOCK:
        return route.abort()
    return route.continue_()

context.route('**/*', handle)

Also consider blocking third-party analytics and ad domains by URL pattern, since they add requests without adding data. To see what's left, log sizes:

total = 0
def on_response(resp):
    global total
    try:
        total += len(resp.body())
    except Exception:
        pass
page.on('response', on_response)
# after navigation
print(f'{total / 1_000_000:.2f} MB')

This measures response bodies only, so real proxy traffic is a bit higher, but it's enough to compare before and after blocking.

What That Means in Money

Say a page weighs 3 MB with everything loaded and 0.6 MB with images, media and fonts blocked. For 20,000 pages that's 60 GB versus 12 GB. On residential per-GB pricing, that's about $33 (60 GB at the $0.55 tier) versus about $7 (12 GB at the $0.60 tier). The blocking code above is the single most effective cost saving for browser scraping. If your volume is high and steady, the unlimited residential plan, billed by the hour, may beat per-GB pricing; see how much proxy bandwidth do I need to estimate.

Timeouts and Retries

  • Raise navigation timeouts: residential exits add latency. 45 to 60 seconds is reasonable: context.set_default_navigation_timeout(60000).
  • Wait for the element you need, not for network idle. wait_until='networkidle' can hang forever on pages with live connections.
  • On a proxy-level failure (ERR_TUNNEL_CONNECTION_FAILED, ERR_PROXY_CONNECTION_FAILED, timeouts), close the context and retry with a new session id rather than retrying on the same IP.
  • Don't retry on 407 or 402 from the proxy. Those mean bad credentials, an inactive plan or no bandwidth left.

Concurrency and Memory

Each context costs memory, and each page more. On a typical 4 to 8 GB machine, a handful of concurrent contexts in one browser is safer than dozens. Scale out with more machines or containers before pushing a single browser hard. Spreading load also keeps request rates per site reasonable, which matters as much as the IPs; see why proxies get blocked.

Check the Exit Before the Real Work

A cheap sanity check at the start of each context catches misconfiguration early: load api.ipify.org (a few bytes), confirm it isn't your server's IP, and then navigate to the target. If you need to verify the country, use our what is my IP tool manually while setting up.

Running Headless Browsers on NodeProxies

Use http://global.nodeproxies.xyz:8080 with sticky usernames per context. Rotating residential proxies are the standard choice for JavaScript-heavy, protected sites, from $0.65/GB down to $0.35/GB at volume. Scrape responsibly: respect robots.txt and terms, as outlined in are proxies legal. Create an account to get started.

FAQ

Should a headless browser use rotating or sticky proxies?

Sticky, per context. A page makes many requests and they should all come from one IP.

How much bandwidth does a headless browser use per page?

Often 1 to 5 MB with everything loaded. Blocking images, media and fonts commonly cuts that by more than half. Measure your own targets.

Why does my headless browser time out through a proxy?

Residential exits are slower, and waiting for network idle can hang. Raise timeouts, wait for specific elements and retry with a new session.

NodeProxies

Ready to get started?

90M+ residential, mobile & datacenter proxies. Datacenter from $0.30/GB, residential from $0.42/GB. No contracts.

Create free account