Troubleshooting 4 min read

429 Too Many Requests with Proxies: How to Stop Rate Limits (2026)

A 429 is the clearest message a website can send: slow down. Here's how to work out what it's counting and pace your requests so it stops.

A 429 Too Many Requests with proxies means the website has counted your requests against a limit and you've crossed it. It's the most honest error a site can send: it isn't saying you're forbidden, only that you're going too fast. The tricky part is that sites count in different ways, and rotating proxies only help when the site counts per IP. This guide covers how to figure out what's being counted and how to pace requests so 429s stop.

What Is the Site Counting?

Rate limits are keyed on something. The usual keys are:

  • IP address. The classic limit: N requests per minute per IP. Rotating proxies spread the count across many IPs and usually solve it.
  • Session cookie or token. If you reuse the same cookie jar across rotating IPs, the site still sees one visitor. Rotation does nothing.
  • API key or account. Limits tied to a login follow the account, not the IP. No proxy changes that, and trying to bypass account limits usually breaks the terms you agreed to.
  • Endpoint. Some limits are global per path, such as a search endpoint that serves only so many queries per second in total.
  • Fingerprint. Advanced systems group requests by TLS and header signature, so many IPs with an identical unusual fingerprint can be counted together.

A Test to Find the Key

Run a short burst with each variable changed in turn and see which one makes 429s disappear:

  1. Same IP, same cookies: baseline. Note the request count at which 429 begins.
  2. Rotating IP (no session parameter), same cookies: if 429 still appears at the same count, the limit isn't per IP.
  3. Rotating IP, fresh cookies per request: if 429 disappears, the limit was per session.
  4. Rotating IP, fresh cookies, slower rate: if only slowing down helps, the limit is per endpoint or global.

In Python, a fresh cookie jar per request just means calling requests.get directly instead of reusing a requests.Session. With a Session, cookies persist across calls.

Read Retry-After and Rate-Limit Headers

Many sites tell you exactly how long to wait. Check the 429 response for Retry-After, and for headers such as RateLimit-Remaining, RateLimit-Reset or X-RateLimit-Reset. Waiting the time they give is faster than guessing, and it keeps your traffic well-behaved.

Build a Simple Rate Limiter

The durable fix is to control your own request rate rather than reacting after the fact. Here's an asyncio example with aiohttp that caps concurrency, spaces requests, and backs off on 429:

import asyncio
import random
import aiohttp

PROXY = "http://USERNAME:[email protected]:8080"
MAX_CONCURRENT = 5
MIN_DELAY = 0.5  # seconds between request starts per worker

sem = asyncio.Semaphore(MAX_CONCURRENT)

async def fetch(session, url, attempts=4):
    for i in range(attempts):
        async with sem:
            async with session.get(url, proxy=PROXY) as r:
                if r.status != 429:
                    return r.status, await r.text()
                retry_after = r.headers.get("Retry-After", "")
        wait = int(retry_after) if retry_after.isdigit() else 2 ** (i + 1)
        await asyncio.sleep(min(wait, 60) + random.random())
    return 429, ""

async def main(urls):
    timeout = aiohttp.ClientTimeout(total=40)
    async with aiohttp.ClientSession(timeout=timeout) as session:
        async def worker(url):
            await asyncio.sleep(random.uniform(0, MIN_DELAY * len(urls) / MAX_CONCURRENT))
            return await fetch(session, url)
        results = await asyncio.gather(*(worker(u) for u in urls))
    for url, (status, _) in zip(urls, results):
        print(status, url)

asyncio.run(main([f"https://example.com/?p={n}" for n in range(20)]))

Two details matter here. The semaphore is released before sleeping, so a backing-off request doesn't block other workers. And the random jitter breaks up the perfectly regular timing that rate limiters and bot detectors both notice.

When Rotation Helps and When It Doesn't

Rotating residential proxies are the right tool when the site limits per IP and you're collecting public pages at a reasonable overall pace. By default, every request without a session parameter gets a new IP. For flows that need the same IP for a few minutes, such as a paginated search, use a sticky session (USERNAME-session-abc123-time-10) and rotate between sessions instead of between requests. Rotating vs. sticky proxies shows both patterns.

Rotation is the wrong tool when the limit is per account or per API key. That limit exists by agreement; work within it, request a higher quota, or use an official data export. Also keep your total request rate sensible even when spread over many IPs: a site that serves you thousands of requests a second from rotating IPs is still carrying that load. Our notes on avoiding blocks when web scraping and on proxy legality cover where the line is.

Common Mistakes That Cause 429s

  • Retrying immediately in a tight loop after a 429, which counts as more requests.
  • Sharing one requests.Session (and its cookies) across all workers while rotating IPs.
  • Fetching the same URL repeatedly instead of caching it.
  • Loading every image, font and script when you only need the HTML.
  • Running many independent scripts against the same site without a shared rate limit.

Pacing Requests with NodeProxies

Use the HTTP gateway at global.nodeproxies.xyz:8080, leave rotation on by default, and add the limiter above. Since billing is per GB, a 429 response costs very little, but retry storms still add up, so the backoff pays for itself. Check per-GB rates on the pricing page or sign up and start with a small plan.

FAQ

Why do I still get 429 with rotating proxies?

The site is probably counting something other than IP, such as a cookie, an account, an endpoint or a fingerprint. Use the four-step test above to find out which.

How long should I wait after a 429?

Use the Retry-After header if the site sends one. Otherwise start at a couple of seconds and double the wait each time, with a cap and random jitter.

Is 429 the same as being blocked?

No. A 429 is a temporary rate limit that clears when you slow down. A 403 is a refusal. See 403 Forbidden with proxies for that case.

How many requests per second is safe?

There's no universal number. Start low, watch for 429s and response time increases, and stay below the rate where they begin.

NodeProxies

Ready to get started?

90M+ residential, mobile & datacenter proxies. Datacenter from $0.30/GB, residential from $0.42/GB. No contracts.

Create free account