How to Scrape Bing in 2026 (for Free!)

For years, the Bing Search API was the developer's quiet escape hatch: cheaper than the alternatives, sanctioned by Microsoft, and a non-Google source of results. Then on August 11, 2025, Microsoft turned it off. Keys went dark, and a whole layer of tools had to scramble.

The upside is that scraping Bing directly is genuinely easy — it has the lightest anti-bot of the major engines, and it often still returns clean HTML to a plain HTTP request. And because Bing's index powers Yahoo and DuckDuckGo too, learning to parse it teaches you all three.

This guide gives you working Python and Node scrapers for Bing, the result selectors, pagination, locale handling, and the honest line on where free scraping runs out.

Why everyone's here: the API retirement

Microsoft retired the Bing Search APIs in August 2025, closing a service that countless apps, SEO tools, and early AI products depended on for affordable, sanctioned search. There was no drop-in Microsoft replacement at the same price, so teams split two ways: pay more for another provider, or scrape the public results page.

If you came here from a dead Bing key, you have company. The migration options — including paid replacements — are laid out in our guide to Bing Search API alternatives and the broader Google Search API alternatives. This post is the free, do-it-yourself half of that decision.

Bing is the index behind Yahoo and DuckDuckGo

Here is the leverage. Bing does not just serve Bing — it is the engine behind Yahoo's web results and DuckDuckGo's too. Scrape Bing well and you understand the entire non-Google, non-Brave family of Western search.

That also means you can pick your access point. If Bing throws a challenge for a query, the same results are usually reachable via scraping Yahoo (with its redirect-wrapped links) or scraping DuckDuckGo (with its rate-limit quirks). Three doors, one index — useful redundancy when one door sticks.

A free Bing scraper in Python

Unlike Google, Bing frequently serves real, parseable HTML to a plain request. A believable User-Agent and an Accept-Language header are usually enough:

import requests
from bs4 import BeautifulSoup

HEADERS = {
    "User-Agent": (
        "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
        "(KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36"
    ),
    "Accept-Language": "en-US,en;q=0.9",
}

def scrape_bing(query, cc="us", count=20, first=1):
    resp = requests.get(
        "https://www.bing.com/search",
        params={"q": query, "count": count, "first": first, "cc": cc},
        headers=HEADERS, timeout=20,
    )
    soup = BeautifulSoup(resp.text, "html.parser")

    results = []
    for offset, li in enumerate(soup.select("li.b_algo")):
        link = li.select_one("h2 a")
        caption = li.select_one(".b_caption p")
        if not link:
            continue
        results.append({
            "position": first + offset,
            "title": link.get_text(strip=True),
            "url": link.get("href"),
            "snippet": caption.get_text(strip=True) if caption else None,
        })
    return results

for r in scrape_bing("best running shoes 2026")[:10]:
    print(r["position"], r["title"])

One pleasant difference from Yahoo: Bing's organic links are usually the real destination URLs, not redirect wrappers, so there is no decoding step. What you parse is what you store.

The b_algo selectors explained

Bing's markup is mercifully stable compared to Google's. Each organic result lives in an <li class="b_algo">. Inside it, the title and link sit in an h2 a, and the snippet text is in a .b_caption p. Those three anchors have survived many redesigns.

Still, treat them as likely-to-drift rather than permanent. Anchor on li.b_algo as the container and reach inward for the heading and caption, so a class tweak on a nested element does not wipe your whole parse. The discipline of resilient selectors is the same across every engine and worth building in from day one.

Pagination and locale

Bing paginates with first, the index of the first result to return — first=1 for page one, then roughly first=11, first=21, and so on. The count parameter requests more results per page, and locale is controlled with cc (country) and setlang (language).

import time, random

def scrape_bing_paged(query, pages=3, cc="us"):
    seen, out = set(), []
    for page in range(pages):
        first = page * 10 + 1
        batch = scrape_bing(query, cc=cc, first=first)
        if not batch:
            break                       # block or end of results
        for r in batch:
            if r["url"] and r["url"] not in seen:
                seen.add(r["url"])
                out.append(r)
        time.sleep(3 + random.uniform(0, 3))   # be polite between pages
    return out

De-duplicate on the URL, because consecutive Bing pages occasionally repeat a result near the boundary. And keep the randomized delay — the request rhythm is what trips anti-bot, not the parsing.

The headless Node fallback

When a query trips a captcha or returns a thin page, a real browser gets you through. The Puppeteer version mirrors the others in this series: stealth plugin on, heavy resources blocked, parse the same li.b_algo blocks.

const puppeteer = require('puppeteer-extra');
const Stealth = require('puppeteer-extra-plugin-stealth');
puppeteer.use(Stealth());

const BLOCK = new Set(['image', 'font', 'media', 'stylesheet']);

(async () => {
  const browser = await puppeteer.launch({
    headless: 'new',
    args: ['--no-sandbox', '--disable-blink-features=AutomationControlled'],
  });
  const page = await browser.newPage();
  await page.setRequestInterception(true);
  page.on('request', (req) =>
    BLOCK.has(req.resourceType()) ? req.abort() : req.continue()
  );

  const q = 'best running shoes 2026';
  await page.goto(
    `https://www.bing.com/search?q=${encodeURIComponent(q)}&count=20`,
    { waitUntil: 'domcontentloaded', timeout: 30000 }
  );

  const results = await page.$$eval('li.b_algo', (nodes) =>
    nodes
      .map((li, i) => {
        const a = li.querySelector('h2 a');
        const cap = li.querySelector('.b_caption p');
        return a
          ? { position: i + 1, title: a.innerText, url: a.href,
              snippet: cap ? cap.innerText : null }
          : null;
      })
      .filter(Boolean)
  );

  console.log(results);
  await browser.close();
})();

Because Bing's defenses are lighter, you will reach for this fallback far less than you would with Google. For most queries the plain Python request is enough, which is the whole reason Bing is a good first scraping target.

Proxies, bandwidth, and where free ends

One IP carries you further on Bing than on Google, but the same wall arrives eventually: push volume and Bing answers with captchas and thin pages. The fix is rotating residential proxies so requests come from many addresses.

Those are billed per gigabyte, so block images, fonts, and stylesheets at the request layer to keep the bill down — the technique in cutting scraping bandwidth by blocking resources. Any provider works the same: Bright Data, Oxylabs, Decodo, IPRoyal, DataImpulse, and SOAX all give you a gateway plus credentials you wire into the browser with --proxy-server and page.authenticate().

The reliable free defenses remain pacing and patience — a clean residential IP does not save a robotic rhythm, the lesson of why proxies get banned. For the full build-vs-buy maths across engines, see web scraping vs a SERP API.

The one-call alternative

If the Bing API's retirement is what brought you here and you just want stable results, a SERP API is the like-for-like replacement — same idea, no key revocations, clean JSON:

import requests

resp = requests.get(
    "https://api.apiserpent.com/api/search",
    headers={"X-API-Key": "sk_live_your_key"},
    params={"q": "best running shoes 2026", "engine": "bing", "country": "us"},
)

for r in resp.json()["results"]["organic"]:
    print(r["position"], r["title"], r["url"])

The same JSON shape works for every engine — switch engine to google, yahoo, or ddg, or hit the dedicated news and image endpoints. Try a live query in the playground before you write any code.

FAQ

Is the Bing Search API really gone?

Yes. Microsoft retired its Bing Search APIs on August 11, 2025, ending a long-standing, relatively cheap source of sanctioned search results. Existing keys stopped returning data, which is why so many tools and AI products had to migrate to alternatives or to scraping the public Bing results page directly.

Is Bing easier to scrape than Google?

Generally yes. Bing's anti-bot layer is lighter than Google's, and for many queries it still returns parseable HTML to a plain HTTP request with realistic headers, no JavaScript rendering required. You still need pacing and, at volume, residential proxies, but the barrier to a first working scraper is much lower.

Does Bing block scrapers?

It can. Push too many requests from one IP and Bing starts returning captchas, thin pages, or empty results. The defenses are softer than Google's, so polite pacing from a clean IP goes a long way, and residential proxy rotation handles steady volume. Always check that a common query returns results rather than a block page.

Why scrape Bing instead of Yahoo or DuckDuckGo?

They share the same index — Yahoo and DuckDuckGo both serve Bing's results — so you scrape whichever is easiest for your case. Bing gives you the source directly with the richest result set and clean li.b_algo markup; Yahoo and DuckDuckGo are convenient mirrors with their own quirks like redirect wrappers and rate limits.