RSS, Atom and JSON Feed to clean JSON, with feed discovery and only-new-items mode

#api #automation #javascript #data

Feeds are the most reliable structured data on the web, and every parser you write handles the first three feeds fine and breaks on the fourth. Here is what breaks and a way to stop caring.

Why feed parsing is messier than it looks

Four formats: RSS 2.0, RSS 1.0 (RDF), Atom, and JSON Feed.
Content lives in description, content:encoded, Atom content, or content_html, often wrapped in CDATA with HTML entities escaped once or twice.
Authors are author, dc:creator, or Atom author/name, sometimes only at feed level.
Dates come in RFC 822, ISO 8601, and things that are neither.
Images hide in media:content, itunes:image, an enclosure, or the first <img> in the body.
Many people give you a website URL, not the feed URL, so you need <link rel="alternate"> discovery and a fallback probe of /feed, /rss.xml, /atom.xml.

The two-minute version

RSS Feed to JSON takes feed URLs or plain website URLs, discovers the feed, and returns one normalised record per item with HTML and plain-text content, categories, enclosures and a lead image.

Paste feed or website URLs into Feed or website URLs.
Optionally set Max items per feed and Published after.
Click Start. Items appear within seconds; send them to Google Sheets, Slack, Make, Zapier or a webhook from the Integrations tab.

For a scheduled digest, add a Schedule and switch on Only new items since the last run: the Actor remembers delivered items in a state store and marks each record isNew, and skipped items are not billed.

From code

import { ApifyClient } from "apify-client";

const client = new ApifyClient({ token: "<YOUR_API_TOKEN>" });
const run = await client.actor("josh99smith/rss-feed-to-json").call({
    feedUrls: ["https://blog.apify.com/rss/", "https://hnrss.org/frontpage", "https://www.theverge.com"],
    maxItemsPerFeed: 50,
    plainText: true,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items.map((i) => `${i.feedTitle}: ${i.title}`));

What you get back

{
    "feedUrl": "https://blog.apify.com/rss/",
    "feedTitle": "Apify Blog",
    "feedType": "rss",
    "id": "64884d212b8e8a0001df7576",
    "title": "12 LangChain alternatives for building AI agents in 2026",
    "url": "https://blog.apify.com/langchain-alternatives/",
    "author": "Theo Vasilis",
    "publishedAt": "2026-09-15T12:00:00.000Z",
    "summary": "LangChain is a powerful framework for developing LLM apps, but ...",
    "contentText": "LangChain is still the most widely used framework ...",
    "categories": ["AI", "Tool comparisons"],
    "imageUrl": "https://storage.ghost.io/.../langchain-alternatives-1.png",
    "isNew": true
}

A page with no discoverable feed returns one free { "success": false, "errorType": "not-found" } record.

Cost and limits

$0.0005 per delivered item (2,000 items = $1), no start fee, failed feeds and monitor-mode skips free, cost cap per run. Feeds behind logins or bot protection are reported as blocked, not bypassed.

Use it from an AI agent

Add https://mcp.apify.com?tools=josh99smith/rss-feed-to-json to your MCP client and ask "what are the latest 10 posts on blog.apify.com?".

Disclosure: I built this Actor. Source: github.com/josh99smith/rss-feed-to-json.

DEV Community