News API for Developers: How to Aggregate News from Search Engines

Why News Aggregation Matters

Staying informed is no longer optional for businesses. Whether you are tracking competitor announcements, monitoring industry trends, or building a media intelligence platform, access to timely news data is critical. Manual news monitoring does not scale, and traditional RSS feeds cover only a fraction of the news published online each day.

News aggregation through search engine APIs solves this problem by providing real-time access to the same news results that appear on Google News and Yahoo News. These search engines crawl thousands of news sources worldwide and rank articles by relevance and recency, giving developers access to a curated, up-to-date news feed for any topic.

For developers building media monitoring tools, content discovery platforms, or competitive intelligence dashboards, a news search API is the foundation that makes everything else possible.

The Landscape of Existing Solutions

The news API space has changed significantly over the years. Google deprecated its dedicated News API years ago, leaving developers scrambling for alternatives. While services like NewsAPI.org filled some of the gap, they come with limitations: restricted free tiers, delayed results, and coverage gaps for non-English sources.

Traditional RSS-based approaches require you to manually subscribe to individual sources. This works for a handful of publications but falls apart when you need comprehensive coverage of a topic across hundreds of outlets.

Search engine news results offer a fundamentally different approach. Instead of subscribing to sources, you search by topic and get results from every indexed news source. This means you automatically discover new publications covering your topics without maintaining a source list.

Why Search-Based News Is Better

• Automatic source discovery -- New publications are included as they get indexed
• Relevance ranking -- Results are sorted by relevance, not just publication time
• Global coverage -- Access to news from any country using the country parameter
• No source management -- No need to maintain RSS feed lists or publication databases

How Search Engine News Works

Search engines maintain a dedicated news index separate from their main web index. This index prioritizes content from recognized news publishers and updates much more frequently than the regular web index. When you search for news, the engine returns articles from this specialized index, ranked by a combination of recency, relevance, source authority, and topical match.

The Serpent API exposes this news index through a simple parameter. By setting type=news in your API request, you get results from the news index instead of the regular web index. The results include article titles, URLs, publication sources, publish dates, and descriptions.

Using the Serpent API News Endpoint

The news endpoint follows the same pattern as the standard search endpoint, with the type parameter set to news:

GET https://apiserpent.com/api/search?q=artificial+intelligence&type=news
Headers:
  X-API-Key: sk_live_your_api_key

Available parameters for news search:

• q (required) -- Your search query
• type (required) -- Set to news
• num (optional) -- Number of results (default: 10, max: 100)
• engine (optional) -- google (default) or yahoo
• country (optional) -- Country code for localized news (e.g., us, uk, de)

Code Examples: Node.js and Python

Node.js Example

const API_KEY = 'sk_live_your_api_key';

async function fetchNews(query, options = {}) {
  const params = new URLSearchParams({
    q: query,
    type: 'news',
    num: options.num || 20,
    engine: options.engine || 'google',
    ...(options.country && { country: options.country })
  });

  const response = await fetch(
    `https://apiserpent.com/api/search?${params}`,
    { headers: { 'X-API-Key': API_KEY } }
  );

  if (!response.ok) {
    throw new Error(`API error: ${response.status}`);
  }

  return response.json();
}

// Fetch latest AI news
async function main() {
  const data = await fetchNews('artificial intelligence breakthroughs', {
    num: 20,
    country: 'us'
  });

  if (data.success && data.results.news) {
    data.results.news.forEach((article, i) => {
      console.log(`${i + 1}. ${article.title}`);
      console.log(`   Source: ${article.source}`);
      console.log(`   Date: ${article.date}`);
      console.log(`   URL: ${article.url}`);
      console.log('');
    });
  }
}

main();

Python Example

import requests
from datetime import datetime

API_KEY = 'sk_live_your_api_key'

def fetch_news(query, num=20, engine='google', country=None):
    """Fetch news articles from Serpent API."""
    params = {
        'q': query,
        'type': 'news',
        'num': num,
        'engine': engine
    }
    if country:
        params['country'] = country

    response = requests.get(
        'https://apiserpent.com/api/search',
        params=params,
        headers={'X-API-Key': API_KEY}
    )
    response.raise_for_status()
    return response.json()

# Fetch and display tech news
results = fetch_news('cloud computing trends', num=15, country='us')

if results.get('success'):
    articles = results.get('results', {}).get('news', [])
    for i, article in enumerate(articles, 1):
        print(f"{i}. {article.get('title')}")
        print(f"   Source: {article.get('source')}")
        print(f"   Date: {article.get('date')}")
        print()

Building a News Aggregator

Let us build a practical news aggregator that monitors multiple topics, deduplicates results, and stores them for later analysis.

const fs = require('fs');

class NewsAggregator {
  constructor(apiKey) {
    this.apiKey = apiKey;
    this.articles = new Map(); // URL -> article for dedup
  }

  async fetchTopic(topic, options = {}) {
    const params = new URLSearchParams({
      q: topic,
      type: 'news',
      num: options.num || 20,
      engine: options.engine || 'google'
    });

    const response = await fetch(
      `https://apiserpent.com/api/search?${params}`,
      { headers: { 'X-API-Key': this.apiKey } }
    );

    const data = await response.json();

    if (data.success && data.results.news) {
      for (const article of data.results.news) {
        if (!this.articles.has(article.url)) {
          this.articles.set(article.url, {
            ...article,
            topic,
            fetchedAt: new Date().toISOString()
          });
        }
      }
    }

    return data.results.news?.length || 0;
  }

  async monitorTopics(topics, options = {}) {
    const results = {};
    for (const topic of topics) {
      const count = await this.fetchTopic(topic, options);
      results[topic] = count;
      // Rate limiting: wait between requests
      await new Promise(r => setTimeout(r, 1500));
    }
    return results;
  }

  getArticles() {
    return Array.from(this.articles.values())
      .sort((a, b) => new Date(b.date) - new Date(a.date));
  }

  save(filepath) {
    const articles = this.getArticles();
    fs.writeFileSync(filepath, JSON.stringify(articles, null, 2));
    return articles.length;
  }
}

// Usage
const aggregator = new NewsAggregator('sk_live_your_api_key');

const topics = [
  'startup funding',
  'tech layoffs',
  'artificial intelligence regulation',
  'cybersecurity breach'
];

const counts = await aggregator.monitorTopics(topics);
console.log('Articles found per topic:', counts);

const total = aggregator.save('news-digest.json');
console.log(`Saved ${total} unique articles`);

Filtering by Freshness

News relevance decays quickly. An article from three days ago is less valuable than one published an hour ago. Here is how to filter results by publication time:

function filterByFreshness(articles, maxAgeHours = 24) {
  const cutoff = new Date();
  cutoff.setHours(cutoff.getHours() - maxAgeHours);

  return articles.filter(article => {
    if (!article.date) return false;
    const pubDate = new Date(article.date);
    return pubDate >= cutoff;
  });
}

// Get only articles from the last 6 hours
const freshNews = filterByFreshness(articles, 6);
console.log(`${freshNews.length} articles from the last 6 hours`);

Multi-Keyword Monitoring

For comprehensive monitoring, you often need to track multiple keywords and combine the results. This approach is useful for PR monitoring, competitive intelligence, and industry tracking.

async function monitorKeywords(keywords, apiKey) {
  const allArticles = [];

  for (const keyword of keywords) {
    const params = new URLSearchParams({
      q: keyword,
      type: 'news',
      num: 10
    });

    const response = await fetch(
      `https://apiserpent.com/api/search?${params}`,
      { headers: { 'X-API-Key': apiKey } }
    );

    const data = await response.json();
    if (data.success && data.results.news) {
      allArticles.push(...data.results.news.map(a => ({
        ...a,
        matchedKeyword: keyword
      })));
    }

    await new Promise(r => setTimeout(r, 1000));
  }

  // Deduplicate by URL
  const unique = new Map();
  for (const article of allArticles) {
    if (!unique.has(article.url)) {
      unique.set(article.url, article);
    }
  }

  return Array.from(unique.values());
}

// Monitor your brand and competitors
const articles = await monitorKeywords([
  '"Acme Corp" announcement',
  '"Acme Corp" partnership',
  'competitor name funding'
], 'sk_live_your_api_key');

Real-World Use Cases

Media Monitoring Dashboard

PR teams use news APIs to build internal dashboards that track brand mentions in real-time. By running scheduled searches every hour, they can react quickly to press coverage and measure the impact of PR campaigns. The Serpent API's affordable pricing makes it practical to run frequent checks without budget concerns.

Content Discovery for Publishers

News publishers and content teams use news aggregation to identify trending stories early. By monitoring broad industry keywords and analyzing which topics are generating the most coverage, editorial teams can prioritize their own reporting to match reader interest.

Financial News Monitoring

Investment firms and fintech applications monitor news for specific companies, sectors, and economic events. Real-time news data feeds into sentiment analysis models and trading algorithms. The ability to search by country makes it possible to track market-moving news in specific regions.

Crisis Management

When a crisis hits, organizations need to monitor news coverage in real-time. A news API allows crisis management teams to track how a story evolves, identify which outlets are covering it, and measure the volume of coverage over time. Automated alerts based on search results ensure nothing is missed.