Bulk Keyword Rank Checking: The Capacity Math for 1M Keywords Overnight

Bulk keyword rank checking is a capacity-planning problem before it is a programming problem. A million keywords in an 8-hour overnight window is a sustained 39 requests per second — about 2,300 a minute — and as of July 2026 most SERP API plans' documented rate limits cannot reach a tenth of that. Whether your batch fits comes down to four numbers: window length, provider cap, measured latency, and retry headroom. This post works all four, with real measurements and verified prices.

It deliberately stays at the arithmetic layer. For the client-side engineering that executes the plan (concurrency control, backoff, resumable state), see running millions of SERP requests; for pipeline resilience (queues, circuit breakers, dead-letter handling), see the SERP scraping architecture guide; and for the full per-provider policy comparison the numbers below come from, see SERP API rate limits compared.

Step 1: the window decides your rate

Every bulk check starts with one division. Take the batch size, divide by the window, and leave headroom — you will need it for retries:

required rate = keywords / (window seconds × 0.9)

1,000,000 / (8 × 3600 × 0.9)  ≈  39 requests/second
                              ≈  2,315/minute
                              ≈  138,889/hour

That last line is the number to hold onto: a 1M-overnight batch needs ~139,000 searches per hour, sustained. Now compare it with what providers actually permit in writing.

The 1M-overnight feasibility table

Every rate below was read from the provider's own pricing page or docs on July 17, 2026 (full sourcing in our rate-limit comparison). “Minimum time” is 1,000,000 divided by the documented maximum rate — the physics ceiling, before latency, retries, or queueing.

Two honest observations. First, on published numbers alone, only Value SERP's top tiers and SearchApi's largest plans clear a literal million-in-a-night — every other lane needs either a custom arrangement or a longer window. Second, we're in that second group too: our standard $500+ allocation clears 30,000 checks a night (a million a month on a steady schedule), and a bigger window is a conversation, not a checkbox. We'd rather print that than let you discover it mid-migration.

Note that the table is really a two-axis check, and both axes have to pass. Axis one is rate: can the plan legally sustain ~139K/hour? Axis two is purchasability: does any published plan even sell 1M searches in a month? SerpApi fails on both at once — the 6,000/hour cap is 23× too slow and the largest published quota is 3% of the batch. A provider can also pass one axis and fail the other: SearchApi's percentage rule scales rate with plan size, so the rate axis passes exactly when the quota axis does. Run both checks against the plan you'd actually buy — the biggest number on a pricing page is rarely the tier your budget lands on.

Step 2: measure latency, then size concurrency

A rate target says nothing about how many requests must be in flight. That comes from Little's law: concurrency = rate × latency. If a search takes 5 seconds and you need 39 a second, roughly 195 requests are in the air at any moment — a very different worker pool than the 39 people usually guess.

Don't guess latency; probe it. This is the complete planning script — we ran exactly this code against our production API before publishing (swap in your own key):

"""Capacity probe: measure real latency, then size the overnight batch."""
import json
import statistics
import time
import urllib.parse
import urllib.request

API_KEY = "YOUR_API_KEY"
BASE = "https://apiserpent.com/api/search/quick"
KEYWORDS = ["best crm software", "car insurance quotes", "standing desk reviews"]

latencies = []
for kw in KEYWORDS:
    url = f"{BASE}?q={urllib.parse.quote(kw)}&engine=google&country=us"
    req = urllib.request.Request(url, headers={"X-API-Key": API_KEY})
    t0 = time.monotonic()
    with urllib.request.urlopen(req, timeout=60) as resp:
        body = json.load(resp)
    dt = time.monotonic() - t0
    latencies.append(dt)
    print(f"{kw!r}: {dt:.2f}s, {len(body.get('results', []))} results")

median = statistics.median(latencies)

N, WINDOW_HOURS, HEADROOM = 1_000_000, 8, 0.90   # use 90% of the window
rate = N / (WINDOW_HOURS * 3600 * HEADROOM)       # required requests/sec
workers = rate * median                           # Little's law

print(f"\nmedian latency : {median:.2f}s")
print(f"required rate  : {rate:.0f}/s  ({rate*60:.0f}/min, {rate*3600:.0f}/hr)")
print(f"workers needed : {workers:.0f} concurrent requests in flight")

Output from our run (July 18, 2026):

'best crm software': 6.18s, 11 results
'car insurance quotes': 4.72s, 11 results
'standing desk reviews': 4.39s, 11 results

median latency : 4.72s
required rate  : 39/s  (2315/min, 138889/hr)
workers needed : 182 concurrent requests in flight

Three real seconds of median latency either way moves the worker count by more than a hundred — which is why the probe belongs at the start of planning, run in the same window you'll run the batch. Keeping 182 requests healthily in flight (pooling, backoff, resumable state) is exactly the engineering our at-scale client guide covers; this post's job ends at the number.

Step 3: budget the retries you will definitely have

No first pass finishes clean at this volume. At a 97% first-pass success rate, a million-keyword batch produces ~30,000 retries; at 95%, it's 50,000 — a whole extra hour at 39/s if you didn't reserve for it. That's what the 0.9 in the window formula is: a 10% retry budget, planned as capacity rather than discovered as overrun.

Practical shape: run the main pass at 90% of your permitted rate, collect failures in a straggler list, and sweep it in the window's final hour. On a 429, honor the wait the provider tells you — our docs publish the 429 contract (Retry-After header plus retryAfter, window and limit fields, windows resetting on fixed UTC boundaries), so backoff can be exact instead of guessed. A 429 at 2 a.m. is the window telling you your automation is drawing faster than the account's allocation — it's feedback for the scheduler, not an error to hammer through. And know your provider's overage behavior before the sweep: some subscriptions re-buy themselves when a runaway loop drains the quota, a mechanic we document in the hidden costs of SERP APIs.

Step 4: pick the window for data quality, not just capacity

“Overnight” isn't only about server room; it's about comparability. Google's results genuinely differ run to run — we measured position churn across back-to-back identical queries in why Google results differ every time — so a rank series is only as comparable as its sampling is consistent. Three scheduling rules follow directly:

Same window every night. A keyword checked at 2 a.m. Monday and 6 p.m. Tuesday isn't a trend line; it's two different experiments. Steady beats bursty. Front-loading the night burns your per-minute cap early, then idles — a flat 90%-of-cap rate finishes the same batch with fewer 429s and cleaner timestamps. Shard and checkpoint. Split the million into resumable chunks (by client, by country, by priority tier) so a 3 a.m. failure costs one shard, not the night — the architecture guide owns the how. For rank tracking specifically, the same shard boundaries map naturally onto per-market runs — the pattern behind our rank tracking API workflows.

Step 5: forecast the bill before you run it

One million page-one checks, at prices read from each provider's own pricing page on July 17, 2026 (the same verified dataset behind our interactive cost calculator, which will run these numbers for any volume):

Depth changes this table more than any discount. Since Google removed num=100 in September 2025, one request returns ~10 results, so top-100 tracking costs per-page billers roughly 10× the page-one figure — the mechanics are in our num=100 breakdown. On a $10+ balance, Serpent API's flat per-call pricing includes up to 10 pages (the top ~100) in one charge, so the $30 row is the same $30 at depth 100. That per-1,000 spread is also why capacity and cost have to be planned together: the provider that fits your window and the one that fits your budget are frequently not the same provider, and the table above is where the trade gets made explicit.

FAQ

How many keywords can a SERP API check overnight?

Whatever the documented rate limit allows — not the monthly quota. Published ceilings in July 2026 run from 6,000/hour (SerpApi's largest plan: ~48,000 per 8-hour night) to 5,000/minute (Value SERP's top tiers: 1M in ~3.3 hours). Providers that publish no numbers can't be planned against at all.

How much does it cost to check 1 million keywords?

From $30 (Serpent API Scale, $0.03/1K) through $600 (DataForSEO Standard) and ~$1,000 (Value SERP) to ~$2,000 (DataForSEO Live) at verified July 2026 prices — while SerpApi's published plans stop at 30,000/month, so 1M there means a custom contract.

How many concurrent requests do I need?

Rate × median latency (Little's law). Our measured example: 39/s required × 4.72s median = ~182 requests continuously in flight. Probe your own latency in your own window before sizing anything.

What is a retry budget?

Reserved capacity for re-runs inside the same window. At 97% first-pass success, 1M checks produce ~30,000 retries; planning at 90% of your permitted rate absorbs them plus an end-of-window sweep.

Does checking the top 100 cost 10× more than the top 10?

On per-page billers, roughly yes since Google removed num=100. Serpent API's flat call includes up to 10 pages in one charge on a $10+ balance, so depth 100 doesn't multiply the bill.

References

• SerpApi pricing (plans, hourly throughput caps) — as seen July 17, 2026
• SearchApi pricing (20%-of-plan hourly rule) — as seen July 17, 2026
• DataForSEO docs (2,000 calls/min, 100 tasks per POST, Standard vs Live)
• Value SERP (plan-tied per-minute limits, plan pricing)
• Zenserp pricing (concurrency guidance)
• Serper (public site; pricing and limits gated behind signup)