If you track keyword rankings for clients, you already know the same query returns different results in Manchester and Miami. Google personalises and localises results heavily, and it also protects search from automated traffic. Proxies for Google SERP scraping solve both problems: they let your checks come from the location you care about, and they spread queries across IPs so no single address hammers Google. This guide covers localized SERPs, rotation, CAPTCHAs and a Python starting point.
Google's terms restrict automated queries. For small volumes, the official Custom Search JSON API or a licensed SERP data provider may be the simpler route. If you collect results yourself, keep volumes modest and read are proxies legal first.
Why Rank Tracking Needs Proxies
- Location: local packs, maps results and even organic order change with the searcher's country and city.
- Volume: a few hundred queries from one IP quickly leads to Google's unusual traffic page and CAPTCHAs.
- Clean results: queries from your office are influenced by your own search history and location; a fresh IP gives a neutral view.
Localized SERPs With Country and City Targeting
Use two layers of localisation together. First, the IP: route each query through the country you are tracking, and for local SEO through the city. With NodeProxies residential plans that is USERNAME-country-us for national US results or USERNAME-country-us-state-NY1-city-NYC1 for New York. Second, the URL parameters: hl sets the interface language and gl sets the country Google should localise for.
Keeping the IP, hl and gl consistent with each other gives you the closest match to what a real local searcher sees. For a UK client, that means a UK IP with gl=uk and hl=en; for Germany, a German IP with gl=de and hl=de.
Rotate the IP on Every Query
A SERP check is a single request with no login or cart, so there is no reason to keep an IP. Rotate per query: with no session in the NodeProxies username, every request gets a new residential IP. That is the opposite of social media work, where you want sticky sessions; our rotating vs sticky proxies guide explains when each applies.
Residential IPs are the practical choice here. Google challenges datacenter ranges far more aggressively, so the cheaper per-GB rate tends to be eaten by retries. Residential proxies start at $0.65/GB, and SERP pages fetched as raw HTML are relatively small.
Python Example: Fetching a Localized Google SERP
import random
import time
from urllib.parse import urlencode
import requests
USER, PASS = 'USERNAME', 'PASSWORD'
UA = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36'
def serp(query, country='us', lang='en', tries=3):
proxy = f'http://{USER}-country-{country}:{PASS}@global.nodeproxies.xyz:8080'
url = 'https://www.google.com/search?' + urlencode({'q': query, 'hl': lang, 'gl': country})
headers = {'User-Agent': UA, 'Accept-Language': f'{lang},{lang};q=0.9'}
for attempt in range(tries):
try:
r = requests.get(url, headers=headers, timeout=30,
proxies={'http': proxy, 'https': proxy})
if r.status_code == 200 and '/sorry/' not in r.url:
return r.text
except requests.RequestException as e:
print('request failed:', e)
time.sleep(2 ** attempt + random.random()) # back off, next try gets a new IP
return None
for kw in ['running shoes', 'trail running shoes']:
html = serp(kw, country='us')
print(kw, 'ok' if html else 'blocked')
time.sleep(random.uniform(3, 8))When Google decides traffic looks automated, it redirects to a URL containing /sorry/ or returns 429, which is what the check above catches. Note that Google increasingly relies on JavaScript, so plain HTTP fetches may return limited or different markup. If you need the fully rendered page, run a headless browser such as Playwright through the same proxy and block images to save bandwidth; see proxies with Node.js, Axios and Puppeteer for browser setups.
Handling CAPTCHAs at a High Level
The best CAPTCHA strategy is not triggering them. In order of impact:
- Slow down. Add random delays between queries and cap concurrency per country.
- Rotate per query so no IP accumulates a history of searches.
- Send consistent, realistic headers with a language that matches hl and the IP's country.
- On a block, back off and retry later on a fresh IP rather than retrying immediately.
- Track your block rate. If it climbs, reduce volume before anything else.
We do not recommend building CAPTCHA-solving into your pipeline as a first resort; it adds cost and treats the symptom. Our article on why proxies get blocked covers the underlying signals.
Planning Volume and Cost
Work it out from your keyword list: keywords times locations times checks per week gives queries per week. Multiply by the average bytes of a SERP response (measure len(r.content) on a sample) and add a margin for retries. Rank tracking for most sites does not need daily checks on every keyword; daily for priority terms and weekly for the long tail keeps volume and cost down. For more on SEO-specific setups, see our proxies for SEO page.
Running SERP Checks With NodeProxies
- Create an account; no KYC and no contract.
- Top up from $3 and buy a residential plan, from 100 MB for a first test.
- Set country and, for local SEO, state and city in the username.
- Run a small batch, measure your block rate and bytes per query, then choose a volume tier on the pricing page.
FAQ
What proxies are best for Google SERP scraping?
Rotating residential proxies with country and city targeting. They give localized results and are challenged far less than datacenter IPs.
How do I check Google rankings for a specific city?
Send the query through an IP in that city, for example with city targeting in your NodeProxies username, and set the gl and hl parameters to match the country and language.
Why does Google show 'unusual traffic' when I use a proxy?
Too many queries from one IP, a datacenter IP, or automated-looking headers. Rotate per query, slow down and use residential IPs.
Is there an official alternative to scraping Google?
Google offers the Custom Search JSON API, which has quotas and differs from the live SERP. Many teams also buy data from licensed SERP API providers.
