Tutorials 4 min read

Python Proxies with httpx and aiohttp: Sync, Async and SOCKS5 (2026)

httpx changed its proxy argument, and aiohttp sets proxies per request. Here is current, working code for both, sync and async.

If you've outgrown requests and moved to httpx or aiohttp for async scraping, proxy setup works a little differently in each. httpx proxies are configured on the client, and the argument name changed in recent versions, which breaks a lot of older tutorials. aiohttp takes the proxy on each request. This guide gives current working code for both, plus the concurrency and session patterns that matter once you're making thousands of requests.

httpx: The Current Proxy Syntax

In httpx 0.26 and later, use proxy= (singular). The old proxies= dictionary was deprecated and then removed in 0.28, so code using it now fails with a TypeError.

import httpx

PROXY = 'http://USERNAME-country-us:[email protected]:8080'

with httpx.Client(proxy=PROXY, timeout=30) as client:
    r = client.get('https://api.ipify.org')
    print(r.text)

The proxy applies to every request from that client, both http and https URLs. If you need different proxies for different hosts, use mounts:

mounts = {
    'all://example.com': httpx.HTTPTransport(proxy=PROXY),
    'all://': httpx.HTTPTransport(),
}
client = httpx.Client(mounts=mounts)

httpx Async Client

import asyncio
import httpx

PROXY = 'http://USERNAME:[email protected]:8080'

async def main():
    async with httpx.AsyncClient(proxy=PROXY, timeout=30) as client:
        tasks = [client.get('https://api.ipify.org') for _ in range(5)]
        for r in await asyncio.gather(*tasks):
            print(r.text)

asyncio.run(main())

With no session in the username, each request leaves from a new IP, so you should see five different addresses.

httpx with SOCKS5

pip install 'httpx[socks]'
client = httpx.Client(proxy='socks5://USERNAME:[email protected]:1080')

Without the socks extra installed, httpx raises an error asking you to install it. For most scraping, the HTTP endpoint is simpler and needs no extra package; see SOCKS5 vs HTTP proxies.

aiohttp: Proxy Per Request

aiohttp passes the proxy to each call. Credentials can go in the URL or in proxy_auth:

import asyncio
import aiohttp

PROXY = 'http://global.nodeproxies.xyz:8080'
AUTH = aiohttp.BasicAuth('USERNAME-country-gb', 'PASSWORD')

async def main():
    timeout = aiohttp.ClientTimeout(total=30)
    async with aiohttp.ClientSession(timeout=timeout) as session:
        async with session.get('https://api.ipify.org', proxy=PROXY, proxy_auth=AUTH) as r:
            print(await r.text())

asyncio.run(main())

Because the proxy is per request, one aiohttp session can use many different proxy usernames, which is handy for per-task sticky sessions. aiohttp has no built-in SOCKS support; the aiohttp-socks package adds it via a connector.

Limit Concurrency with a Semaphore

Firing 1,000 coroutines at once doesn't make things faster; it causes timeouts and gets the target to throttle you. Cap in-flight requests:

sem = asyncio.Semaphore(20)

async def fetch(session, url):
    async with sem:
        for attempt in range(3):
            try:
                async with session.get(url, proxy=PROXY, proxy_auth=AUTH) as r:
                    if r.status == 200:
                        return await r.text()
                    if r.status in (402, 407):
                        raise RuntimeError(f'proxy rejected request: {r.status}')
            except (aiohttp.ClientError, asyncio.TimeoutError):
                pass
            await asyncio.sleep(2 ** attempt)
        return None

Note the two status codes that shouldn't be retried: 407 means your credentials or plan are wrong, and 402 means the plan is out of bandwidth. Retrying either just wastes time. Everything else (timeouts, connection resets, 5xx) is worth a retry with backoff. Pacing advice is in how to avoid blocks when web scraping.

Sticky Sessions for Multi-Step Tasks

When a task needs several requests from the same IP, such as fetching a page and then its pagination with a session cookie, build a username per task:

import uuid

def sticky_auth(minutes=10):
    sid = uuid.uuid4().hex[:8]
    return aiohttp.BasicAuth(f'USERNAME-country-us-session-{sid}-time-{minutes}', 'PASSWORD')

Create one per task and pass it on every request of that task. With httpx, create one client per task with the sticky username in the proxy URL.

Environment Variables: httpx vs aiohttp

The two libraries treat HTTP_PROXY and HTTPS_PROXY differently, which causes confusing results when a proxy is set in your shell:

  • httpx reads proxy environment variables by default (trust_env=True). An explicit proxy= argument overrides them, and trust_env=False ignores them entirely.
  • aiohttp ignores proxy environment variables unless you create the session with trust_env=True. A proxy= argument on the request always wins.
# httpx: ignore any proxy set in the environment
client = httpx.Client(trust_env=False)

# aiohttp: use HTTPS_PROXY from the environment
session = aiohttp.ClientSession(trust_env=True)

If requests unexpectedly go through a proxy, or unexpectedly don't, check the environment first with env | grep -i proxy.

Running httpx and aiohttp Through NodeProxies

Point either library at http://global.nodeproxies.xyz:8080 (or socks5:// on 1080) with options in the username. Start with rotating residential proxies for protected sites or datacenter proxies for easy targets. Before a large run, read how to use proxies for web scraping and make sure you respect each site's terms and rate limits. Create an account to get credentials.

FAQ

Why does httpx say unexpected keyword argument 'proxies'?

httpx 0.28 removed proxies=. Use proxy='http://user:pass@host:port' on the client, or mounts for per-host proxies.

How do I use a different proxy for each request in httpx?

httpx sets proxies per client, so create a client per proxy. aiohttp is easier for this because it takes proxy= on each request.

Does aiohttp support SOCKS5 proxies?

Not natively. Install aiohttp-socks and use its ProxyConnector with a socks5:// URL.

Should I retry a 407 from the proxy?

No. A 407 means the credentials are wrong or the plan isn't active. Fix the cause instead of retrying.

NodeProxies

Ready to get started?

90M+ residential, mobile & datacenter proxies. Datacenter from $0.30/GB, residential from $0.42/GB. No contracts.

Create free account