Scrape Google Autocomplete & Related Searches for Free (2026)

Google's autocomplete dropdown is one of the richest free keyword sources on the internet, and almost nobody scrapes it the easy way. Those suggestions are not guesses — they are real, popular queries Google has decided are worth surfacing as you type.

Most tutorials send a headless browser to the homepage, simulate keystrokes, and read the dropdown out of the DOM. That works, but it is slow, fragile, and completely unnecessary, because the same data lives behind a tiny public JSON endpoint that needs no browser and no proxy.

This guide shows you that hidden endpoint, the parameters that matter, how to recursively expand a single seed into a keyword tree of hundreds of suggestions, and how to pull "related searches" from the SERP itself — all with working Python you can run today.

The hidden free endpoint

Open your browser's network tab, go to Google, and start typing. You will see requests fire off to a host called suggestqueries.google.com. That is the feed behind the dropdown, and you can hit it directly:

https://suggestqueries.google.com/complete/search?client=firefox&q=best+coffee

Request that URL and you get back a compact JSON array, not an HTML page. With client=firefox the shape is delightfully simple — element zero is your query echoed back, and element one is the list of suggestions:

["best coffee", ["best coffee maker", "best coffee grinder",
  "best coffee beans", "best coffee shops near me",
  "best coffee in the world", "best coffee for espresso"]]

No JavaScript to execute, no consent wall to click through, no markup to parse with brittle selectors. It is a plain HTTP GET that returns structured data, which makes it the gentlest scrape in this whole series.

This matters more than it sounds. Every other free route in this cluster fights the rendered page — a headless browser, megabytes of images and fonts, a consent banner, ever-shifting CSS class names. The suggest endpoint sidesteps all of it. The payload is a few hundred bytes, parses with the standard library, and has barely changed in years, which is rare for anything Google-adjacent. If you only ever scrape one Google surface, this is the one to start with. Here is a complete fetch-and-parse in a few lines of Python:

# pip install requests
import requests

ENDPOINT = "https://suggestqueries.google.com/complete/search"
HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
        "(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
    )
}

def autocomplete(query, hl="en", gl="us", timeout=10):
    """Return Google autocomplete suggestions for a single query."""
    params = {"client": "firefox", "q": query, "hl": hl, "gl": gl}
    resp = requests.get(ENDPOINT, params=params, headers=HEADERS, timeout=timeout)
    resp.raise_for_status()
    data = resp.json()          # shape: [query, [suggestions...], ...]
    return data[1] if len(data) > 1 else []

for s in autocomplete("best coffee"):
    print(s)

That is a working keyword discovery tool in roughly fifteen lines, with no browser and no proxy. The client=firefox trick is what gives you clean JSON; other client values return JSONP or a different encoding that is fiddlier to parse, so stick with firefox unless you have a reason not to.

The parameters that matter

The endpoint accepts a handful of query parameters, but only four genuinely change your results. Getting hl and gl right is the difference between US English suggestions and, say, German ones — the same discipline as the locale parameters in scraping the Google SERP for free.

Keep hl and gl consistent with the market you are researching — a US coffee shop and a German one get very different autocomplete trees, and a mismatch quietly returns the wrong region's demand. Everything else is noise for keyword work; client, q, hl, and gl are the whole toolkit.

Recursive expansion into a keyword tree

A single request returns up to about ten suggestions, which is useful but small. The trick to turning one seed into hundreds of keywords is breadth: append every letter of the alphabet and every question word to your seed, and fire one request per prefix.

Typing best coffee a surfaces a different slice than best coffee b, and prefixing with question words — how best coffee, why best coffee — pulls out the question-shaped demand that powers FAQ and content briefs. This is exactly the kind of expansion behind a free AnswerThePublic alternative built on autocomplete and PAA.

import string
import time

QUESTION_WORDS = ["who", "what", "when", "where", "why", "how",
                  "which", "are", "can", "does", "is", "will"]

def expand(seed, hl="en", gl="us", delay=0.4):
    """Expand one seed into a de-duplicated set of suggestions."""
    found = set()

    # 1) the bare seed
    found.update(autocomplete(seed, hl, gl))

    # 2) seed + each letter a-z  (e.g. "best coffee a", "best coffee b" ...)
    for letter in string.ascii_lowercase:
        found.update(autocomplete(f"{seed} {letter}", hl, gl))
        time.sleep(delay)

    # 3) question word + seed  (e.g. "how best coffee", "why best coffee" ...)
    for word in QUESTION_WORDS:
        found.update(autocomplete(f"{word} {seed}", hl, gl))
        time.sleep(delay)

    return sorted(found)

tree = expand("best coffee")
print(f"{len(tree)} unique suggestions")
for kw in tree[:40]:
    print(" -", kw)

Because every suggestion lands in a Python set, duplicates collapse automatically — the same phrase surfaced by three different prefixes is stored once. A single two-level pass like this on one seed routinely yields a few hundred unique, real-world queries.

You can go deeper by re-feeding the strongest results as new seeds, but mind the multiplication: each new seed is another 38-plus requests. One seed at two levels is 38 requests; ten promising children at two levels each is another 380. That is still tiny by scraping standards, but it is the moment to add caching so you never re-expand a prefix you have already seen.

A subtle bonus of the alphabet sweep is that it surfaces long-tail intent the bare seed hides. The unmodified query best coffee returns the broad, head-term suggestions; best coffee f pulls "best coffee for cold brew" and "best coffee french press" — specific, lower-competition phrases that are gold for content. The question-word prefixes do the same for informational intent, which is precisely what feeds People Also Ask boxes. For organising the output into themes and intent buckets, the workflow in our guide to keyword research with a SERP API picks up exactly where this tree leaves off.

Scraping related searches from the SERP

Autocomplete is the demand before the click. The other free signal lives after it — the "Related searches" block at the bottom of a results page and the "People also search for" suggestions. These come from the SERP HTML, not the suggest endpoint, so you fetch a results page and parse it.

Unlike autocomplete, this needs the actual SERP. The cleanest way to grab and parse it is covered in parsing Google SERP features in Python; the snippet below shows just the related-search extraction once you have the page HTML in hand:

# pip install requests beautifulsoup4
import requests
from bs4 import BeautifulSoup

HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
        "(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
    )
}

def related_searches(query, hl="en", gl="us"):
    """Pull the 'Related searches' phrases from a Google results page."""
    resp = requests.get(
        "https://www.google.com/search",
        params={"q": query, "hl": hl, "gl": gl},
        headers=HEADERS, timeout=15,
    )
    soup = BeautifulSoup(resp.text, "html.parser")

    related = set()
    # Related-search links route through /search?q=... with the phrase as text.
    for a in soup.select('a[href*="/search?"]'):
        text = a.get_text(strip=True)
        # keep multi-word phrases, skip nav links like "Next" or "Images"
        if text and " " in text and len(text) < 80:
            related.add(text)

    return sorted(related)

for phrase in related_searches("best coffee maker"):
    print(phrase)

One caveat: a single unauthenticated IP can hit the suggest endpoint all day, but the full SERP is far more sensitive to volume and may serve a consent page or a challenge instead of results. Treat related-search scraping as the heavier of the two routes, and apply the same care you would for any direct SERP scrape — including the bandwidth discipline of blocking heavy page resources if you switch to a browser.

Rate-limit care: it's free, so be polite

The suggest endpoint is generous, but it is not a buffet. Because it is free and undocumented, the only thing protecting your access is your own restraint — hammer it with a tight loop of thousands of requests and you will start seeing empty responses or HTTP errors, and you risk getting the IP throttled.

A polite expander — a few hundred milliseconds between calls, errors handled, prefixes cached — can run from a single residential or even home IP for a long time. If you genuinely need to expand thousands of seeds quickly you can route requests through any residential proxy provider (Bright Data, Oxylabs, Decodo, IPRoyal, SOAX, NetNut, and DataImpulse are all common choices), but for ordinary keyword research that is overkill.

import requests

def autocomplete_safe(query, hl="en", gl="us", timeout=10):
    """Same as autocomplete() but returns [] on any error instead of crashing."""
    try:
        params = {"client": "firefox", "q": query, "hl": hl, "gl": gl}
        resp = requests.get(ENDPOINT, params=params, headers=HEADERS, timeout=timeout)
        if resp.status_code != 200:
            return []           # back off upstream, don't hot-loop
        data = resp.json()
        return data[1] if len(data) > 1 else []
    except (requests.RequestException, ValueError):
        return []

How this differs from real search volume

Here is the most important caveat in the whole guide: autocomplete tells you which queries are popular enough to suggest, but it gives you no number. There is no monthly search count, no competition score, no cost-per-click attached to a suggestion.

That makes autocomplete a demand-discovery tool, not a demand-measurement tool. It is brilliant for finding the questions and phrasings real people type, and useless for deciding whether a keyword gets 100 or 100,000 searches a month. For that you need a volume source — and there are honest ways to estimate it that do not require a Google Ads account, which we cover in estimating keyword search volume without Google Ads.

The right move is to combine them: use autocomplete and related searches to discover the full shape of demand for free, then layer a volume estimate on the shortlist that survives. Good keyword research is discovery first, measurement second — a point made well in this Nielsen Norman Group primer on keyword research.

There is a second, quieter signal in autocomplete that volume tools miss entirely: ordering. Google returns suggestions roughly in popularity order, so the first item for a prefix is usually a stronger query than the last. It is not a precise rank, and you should not treat it as one, but when you are choosing between two near-identical phrasings the one Google lists first is the safer bet. Combined with the breadth of the alphabet sweep, that ordering gives you a free, directional sense of relative demand long before you spend anything on a measurement tool.

The verdict

For sheer cost-to-value, scraping Google autocomplete is the best deal in keyword research. A single undocumented JSON endpoint, a recursive a–z and question-word expander, and a set for de-duplication get you hundreds of real queries for nothing, with no browser and no proxy.

The honest limits are two. First, the endpoint is public and undocumented — not a supported API — so it can change shape or tighten limits without warning, and you should keep a fallback. Second, it gives you discovery, never volume, so pair it with an estimation method before you commit to a content plan.

When you outgrow the free route — you need related searches, People Also Ask, and full SERP features at volume, reliably, without babysitting a scraper — a managed SERP API hands you all of it as clean JSON from one call. That is where the build-it-yourself maths stops making sense and a hosted endpoint wins.

FAQ

Is the Google suggest endpoint official?

No. The suggestqueries.google.com/complete/search endpoint is the internal feed that powers the autocomplete dropdown in the search box. It is public and undocumented, not a supported API with a contract, so Google can change its shape or rate limits without notice. It works reliably today and returns plain JSON, but you should treat it as a best-effort source and not build a paid product on it without a fallback.

Do I need a proxy for autocomplete?

Usually not. The suggest endpoint returns a tiny JSON payload, needs no browser and no JavaScript, and a single IP can pull thousands of suggestions if you are polite — add a short delay between requests and avoid bursts. You only need rotating IPs if you are expanding a huge keyword tree fast enough that one IP starts seeing throttling, which for most research workloads never happens.

How is this different from Google Ads search volume?

Autocomplete tells you which queries are common enough that Google bothers to suggest them, but it gives you no number — no monthly searches, no competition, no cost-per-click. Google Ads Keyword Planner gives bucketed volume ranges instead. Autocomplete is best for discovering the shape of demand — the questions and phrasings real people type — while volume tools tell you roughly how big each one is.

How many suggestions can I pull per query?

A single suggest request typically returns up to about ten suggestions for one prefix. The way you scale past that is breadth, not depth: append each letter a–z and each question word to your seed, fire one request per prefix, and de-duplicate the results into a set. A two-level expansion of one seed can yield hundreds to a couple of thousand unique suggestions.