How to Read RSS & Atom Feeds in Bulk to JSON in 2026

RSS is not dead — it’s the last open, structured content feed the web still ships by default, and every news site, blog, changelog and podcast still publishes one. The problem is that “RSS” isn’t one format: you get RSS 2.0 from one source, Atom from another, RSS 1.0 / RDF from a third, dates in five different formats, and duplicate articles across overlapping feeds. Parsing all of that reliably, in bulk, is more work than it looks. This guide covers how to read hundreds of feeds in a single run, normalize every dialect into one clean row shape, de-duplicate across sources, and even auto-discover a feed from a plain website URL — no API key, no login.

What’s worth extracting

The reader returns 12 fields per item, normalized regardless of the source dialect:

• Source — feedUrl (the feed the item came from) and feedTitle (the channel title).
• Content — title, contentSnippet (plain-text summary/description) and full HTML content when the feed provides it.
• Links and identity — link (canonical item URL) and guid (the unique item identifier, used for de-duplication).
• Metadata — pubDate (normalized to ISO 8601), author (when present) and categories (tags as an array).
• Media — enclosureUrl, the attached media URL. For podcast feeds this is the per-episode audio link; for others it can be an image or other enclosure.
• Provenance — scrapedAt, the ISO 8601 timestamp the item was read.

The guid-plus-pubDate combination is what makes this useful as a monitoring feed: dedupe on guid, sort on pubDate, and you have a clean, chronological, deduplicated stream across every source.

How the source actually works

There’s no single “RSS API.” RSS, Atom and RDF are open XML formats that each site serves at its own URL. The reader fetches those feeds over direct HTTP — no browser, high concurrency — parses each dialect, and merges everything into one dataset.

The minimum input is one or more feed URLs. A bulk news-and-tech run:

{
  "feedUrls": [
    "https://hnrss.org/frontpage",
    "http://feeds.bbci.co.uk/news/rss.xml",
    "https://www.theverge.com/rss/index.xml",
    "https://techcrunch.com/feed/"
  ],
  "discoverFromWebsites": false,
  "maxItemsPerFeed": 0,
  "maxResults": 0,
  "proxyConfiguration": { "useApifyProxy": true }
}

The clever part is feed auto-discovery. If you don’t know a site’s feed URL, turn on discoverFromWebsites and paste plain homepage URLs. The reader inspects the page’s <link rel="alternate"> hints and probes common feed paths — /feed, /rss, /rss.xml, /atom.xml, /index.xml — to locate the feed, then parses it:

{
  "feedUrls": [
    "https://www.theverge.com",
    "https://techcrunch.com",
    "https://arstechnica.com"
  ],
  "discoverFromWebsites": true,
  "maxItemsPerFeed": 25
}

For podcasts, point it at the feed and cap maxItemsPerFeed to grab only the latest episodes — each row carries the enclosureUrl audio link:

{
  "feedUrls": [
    "https://feeds.megaphone.fm/thedailyshow",
    "https://feeds.simplecast.com/54nAGcIl"
  ],
  "discoverFromWebsites": false,
  "maxItemsPerFeed": 10,
  "maxConcurrency": 5
}

▶ Open the Bulk RSS Feed Reader on Apify — paste feeds (or plain site URLs with discovery on) and get one clean, deduplicated row per item. No API key, no login. Export to JSON, CSV, Excel or XML.

The access reality

Feeds are published specifically to be read by feed readers, so there’s no anti-bot wall. The real friction is the messiness of the format and doing it at scale:

• Dialect chaos. RSS 2.0, RSS 1.0 / RDF and Atom differ in element names and structure. The reader normalizes all three into the same item shape so you never branch on the source format.
• Date formats vary wildly. Feeds emit RFC 822, ISO, and half-broken variants. Every pubDate is parsed into a sortable, filterable ISO 8601 timestamp.
• Broken and slow feeds. In a batch of hundreds, some feeds will be empty, timing out, or malformed. Per-feed error isolation means one bad feed is skipped without killing the run — every other feed still produces rows.
• IP rate limits at volume. Reading hundreds of feeds from one IP invites throttling. Proxy support (Apify Proxy) spreads the requests, and maxConcurrency (1–100, default 10) controls how many feeds fetch in parallel — higher is faster but uses more proxies.

The naive approach — a feedparser loop in a script — works for five feeds and falls over at five hundred: one hung request stalls the batch, one malformed feed throws, and you have no dedup across overlapping sources. The reader handles all three.

Example output

One trimmed item row:

{
  "feedUrl": "https://techcrunch.com/feed/",
  "feedTitle": "TechCrunch",
  "title": "A new startup wants to reinvent the RSS reader",
  "link": "https://techcrunch.com/2026/06/14/rss-reader-startup/",
  "guid": "https://techcrunch.com/?p=2847193",
  "pubDate": "2026-06-14T16:32:00.000Z",
  "author": "Jane Doe",
  "categories": ["Startups", "Apps"],
  "contentSnippet": "The team behind the project says RSS is overdue for a comeback...",
  "content": "<p>The team behind the project says RSS is overdue for a comeback.</p>",
  "enclosureUrl": "",
  "scrapedAt": "2026-07-06T09:00:00.000Z"
}

Empty enclosureUrl and author fields are normal — not every feed provides media or bylines.

Typical use cases

What people build with a bulk feed reader:

• News and brand monitoring — track topics, mentions and outlets across hundreds of news feeds in near real time.
• Content aggregation — power a reader app, dashboard or homepage from dozens of sources with a single deduplicated stream.
• AI / RAG / LLM pipelines — turn fresh feed content (title, link, pubDate, author, full content) into structured rows for retrieval, summarization or classification. Because dedup is by guid, each scheduled run surfaces only the newly published items, keeping a knowledge base fresh without re-ingesting duplicates.
• Competitive intelligence — watch competitor blogs, changelogs and press feeds and catch every new post.
• Newsletter and podcast tracking — follow Substack/blog feeds and podcast RSS, capturing the per-episode enclosureUrl audio link.
• Research and archiving — snapshot what a set of sources published, with ISO timestamps, for analysis or compliance.

Cost and effort: build vs. managed

You can absolutely write your own reader — the parsing libraries exist. What you’re really signing up for is the operational layer: concurrency control, per-feed timeout and error isolation so one bad feed doesn’t stall the batch, cross-feed de-duplication by guid/link, date normalization across every dialect, proxy rotation to dodge rate limits, and a scheduler if you want a recurring feed. That’s a small system, not a script, and it needs maintenance as feeds change.

The managed actor is pay-per-result: a small per-run start fee plus a per-item rate, no subscription. Use maxItemsPerFeed to keep only the latest posts per source and maxResults as a hard ceiling on total rows to cap a run’s size and cost; leave both at 0 to backfill everything the feeds expose. For a scheduled hourly news poll across a few hundred feeds, the GUID de-dup means you’re mostly paying for genuinely new items.

Common pitfalls

Specific to feed reading:

• De-dup is by guid/link, not content. If two feeds carry the same story under different GUIDs, both survive. Overlapping aggregators can still produce near-duplicates — collapse on link downstream if you need stricter dedup.
• content is often empty or truncated. Many feeds only ship a summary. When content is blank, fall back to contentSnippet, and expect to fetch the full article separately if you need the body.
• Discovery isn’t guaranteed. Auto-discovery relies on <link rel="alternate"> tags or the common feed paths existing. Some sites hide or omit their feed; if discovery finds nothing, supply the feed URL directly.
• Set a maxResults ceiling on big backfills. Leaving both caps at 0 across hundreds of feeds can pull a very large dataset in one run. Cap it while you’re testing.
• categories is an array. Unlike most fields, categories is a list. Flatten or join it when exporting to a flat CSV consumer that expects scalars.
• Feeds only expose recent items. Most feeds carry the latest N entries, not full history. For an archive, you need to poll on a schedule over time — a single run only captures the current window.

Wrapping up

RSS remains the cheapest, most reliable way to pull structured content off the open web, and the only hard parts are dialect normalization, dedup and running it at scale without one broken feed taking down the batch. If you’d rather not build that operational layer — or you just want to point it at a homepage and let it find the feed — the managed reader does it over pure HTTP, keyless.

▶ Run the Bulk RSS Feed Reader on Apify — one run reads hundreds of RSS, Atom and RDF feeds (or auto-discovers them from plain URLs) into thousands of deduplicated, ISO-dated item rows. No API key, no login. Export to JSON, CSV or Excel. Start on Apify’s free monthly credit.