Redis RSS Feed Cache: TTL, Locks and a Stale Fallback

Short answer: Redis is a good place to keep the finished XML of an RSS feed for a few minutes. Build the feed once, store the whole document under one key with an expiry, return the stored copy to every reader and keep a longer-lived stale copy for the moments when the database or the application is slow. Use a short lock so that only one request rebuilds an expired feed, delete the key when you publish if you want instant updates, send ETag and Cache-Control headers from the cached text and connect the address to an auto-poster.

A feed address is one of the most requested URLs on a busy site. Readers, search engines, aggregators and auto-posters all fetch it, often every few minutes, and each request can trigger a database query, a template and an XML build. When traffic grows, or when a crawler with a bad schedule arrives, the feed becomes a quiet source of load, and the page that suffers is often the home page, because it shares the same database. A cache in front of the feed removes most of that work. Redis fits well because it is fast, it expires keys by itself and it offers an atomic way to take a lock. This guide shows a pattern in Node.js with Express, but the same ideas apply to any language that has a Redis client.

What to cache and what to leave alone

Cache the finished document, not the rows from the database. The XML is the same for every reader, it is small, usually a few dozen kilobytes, and building it is the expensive part. A single string under a single key is easy to store, easy to delete and easy to inspect with a command line tool.

• The whole feed for the main address, such as /feed.xml.
• One key per category or language feed, named after the path, so that each can expire separately.
• A validator such as a hash of the text, stored beside the document, so that conditional requests are cheap.

Do not cache answers that depend on the reader, such as feeds for logged-in members or private feeds that include a token. If the content differs by user, include the user or the token in the key, or skip the cache for those feeds. A shared key for private content is a data leak, and no gain in speed justifies it.

The basic read-through pattern

The simplest pattern reads from Redis first and builds the feed only when the key is missing. The code below uses the node-redis client and an Express route.

import express from 'express';
import { createClient } from 'redis';
import { createHash } from 'node:crypto';
import { buildFeedXml } from './feed.js';

const redis = createClient({ url: process.env.REDIS_URL });
await redis.connect();

const app = express();
const KEY = 'feed:main';
const TTL = 300;

app.get('/feed.xml', async (req, res) => {
  let xml = await redis.get(KEY);
  if (!xml) {
    xml = await buildFeedXml();
    await redis.set(KEY, xml, { EX: TTL });
  }
  const etag = '"' + createHash('sha1').update(xml).digest('hex') + '"';
  if (req.headers['if-none-match'] === etag) return res.status(304).end();
  res.set({
    'Content-Type': 'application/rss+xml; charset=utf-8',
    'Cache-Control': 'public, max-age=300',
    ETag: etag,
  });
  res.send(xml);
});

The key expires after five minutes, so the feed is rebuilt at most once in that time. The ETag is computed from the text that is returned, which means that it stays the same until the content changes and tools that poll often receive a short 304 answer. Details of the validator mechanism are in the guide to ETags and conditional requests.

The stampede problem

The basic pattern has a weakness. When the key expires, every request that arrives before the first rebuild finishes also finds the key missing, and each of them builds the feed. If the build takes two seconds and the site receives fifty requests per second, a hundred builds start at once, the database slows down, the builds take longer and more requests pile up. This is called a cache stampede, and it usually happens at the worst time, because a busy moment is also when a popular post has just been published.

There are three defences, and they work well together:

• A lock so that only one request rebuilds the feed.
• A stale copy that other requests can return while the rebuild runs.
• A little random time added to the expiry, so that many keys do not expire at the same second.

A lock with a stale fallback

Redis can create a key only when it does not exist, with an expiry, in a single command. That command is the foundation of a simple lock. Keep a second copy of the feed with a much longer expiry, and return it when the lock is taken by someone else.

const STALE_KEY = 'feed:main:stale';
const LOCK_KEY = 'feed:main:lock';

async function getFeed() {
  const fresh = await redis.get(KEY);
  if (fresh) return fresh;

  const gotLock = await redis.set(LOCK_KEY, '1', { NX: true, EX: 30 });
  if (!gotLock) {
    const stale = await redis.get(STALE_KEY);
    if (stale) return stale;
    await new Promise((r) => setTimeout(r, 200));
    return (await redis.get(KEY)) ?? (await redis.get(STALE_KEY)) ?? buildFeedXml();
  }

  try {
    const xml = await buildFeedXml();
    const jitter = Math.floor(Math.random() * 30);
    await redis.set(KEY, xml, { EX: 300 + jitter });
    await redis.set(STALE_KEY, xml, { EX: 86400 });
    return xml;
  } finally {
    await redis.del(LOCK_KEY);
  }
}

The lock expires after thirty seconds, so a crashed process cannot block the feed for ever. The request that gets the lock builds the document and writes both copies. The other requests return the stale copy at once, which is at most a few minutes old, or wait a moment and read the new one. If the build fails with an error, the stale copy keeps the feed alive for a day, and readers see an old but valid document and not an error page. This matters for feeds, because tools treat repeated errors as a reason to slow down or to stop.

Invalidate when you publish

A lifetime of five minutes is acceptable for most sites, but if you want a new article to appear in the feed immediately, delete the key when you publish.

async function onArticlePublished() {
  await redis.del('feed:main', 'feed:category:news');
}

The next request rebuilds the feed with the new item. Keep the stale copy, because it is only a fallback. If your publishing system runs outside the Node application, a small command that deletes the key after publication is enough. Be careful to delete the keys of every feed that contains the article, such as the category feeds and the tag feeds, or one of them will show the old state until its lifetime ends. The reasons why a post can appear late are collected in the guide to feed caching.

Keys, memory and eviction

A feed with fifteen to thirty recent items is plenty for most tools, and a smaller document is cheaper to store and to send. The recommended size is explained in the guide to the number of items in a feed. If Redis shares an instance with sessions or a job queue, set a memory limit with an eviction policy for least recently used keys, and never use an eviction policy that can remove the queue. A separate instance for cache data is the safest arrangement.

What if Redis is down?

A cache must never become a single point of failure. Wrap the Redis calls so that a connection error is logged and the application builds the feed directly, perhaps with a small in-memory copy for a few seconds. Set short timeouts on the client, because a hanging connection is worse than a missing key. After the cache returns, the next request fills it again and the system recovers without any action from you. Monitor the hit rate and the number of rebuilds, since a sudden fall in hits tells you that keys are evicted too early or that the expiry is wrong.

Test the result

Check three things with curl: the content type and the validators on the first request, the 304 answer on a conditional request and the time of the answer when the cache is warm.

curl -sI https://example.com/feed.xml
curl -s -o /dev/null -w '%{http_code} %{time_total}\n' https://example.com/feed.xml
curl -sI -H 'If-None-Match: "paste-the-etag-here"' https://example.com/feed.xml
redis-cli ttl feed:main

• The first request returns 200 and the RSS content type with a UTF-8 charset.
• The second shows a total time of a few milliseconds when the key is warm.
• The third returns 304 when the validator matches.
• The last command shows how many seconds are left before the key expires.
• The address passes the W3C Feed Validation Service without errors.

A longer routine is in the feed validation guide, and a shorter one in debugging a feed with curl.

Common mistakes to avoid

• No lock. An expired key sends a crowd of requests to the database.
• No stale copy. A failed rebuild turns into an error for every reader.
• A very long expiry. New posts reach social channels late.
• Caching private feeds under a shared key. Content leaks between users.
• Forgetting category feeds on invalidation. One feed shows the old state.
• Trusting the cache to be always there. Handle connection errors.

How PostRSS works with a cached feed

PostRSS has been publishing RSS and Atom feeds to social networks since 2014. Paste the address of your feed, connect your accounts and choose how each post is built from the title, the description or the page title. PostRSS checks the feed as often as every minute and shares each new entry, with its image and link, to the channels you choose among 66 supported networks, messengers, team chats and blogs, including LinkedIn, X, Bluesky, Mastodon and Telegram. A cached feed with validators makes frequent checks cheap for your server. Keyword filters, UTM parameters and posting windows let you tune each channel. See the PostRSS features page and the plan limits on the pricing page. More developer guides are under the Technical Guide tag.

Related reading

• Express RSS feed in Node.js
• Fastify RSS feed in Node.js
• Feed timeouts and slow servers

The bottom line

Redis turns a feed from a repeated database job into a cheap read. Store the finished XML under one key with an expiry of a few minutes, take a short lock so that a single request rebuilds it, keep a stale copy for the bad moments, add a little random time to the expiry and delete the keys when you publish if you need instant updates. Send validators from the cached text, handle a missing cache gracefully and then connect the address to an auto-poster, so that every article you publish reaches your social channels quickly and your server stays calm.

FAQ

How long should an RSS feed be cached in Redis?

Five minutes is a good start. Shorter lifetimes cost more rebuilds, and lifetimes of hours make new posts reach your social channels late.

What is a cache stampede?

It is a burst of identical rebuilds that starts when a popular key expires and many requests find it missing at the same time. A lock and a stale copy prevent it.

Should I delete the key when I publish?

Yes, if you want the new article in the feed at once. Delete every key that contains the article, including category and tag feeds.

Is it safe to cache private feeds?

Only if the key includes the user or the token. A shared key for private content leaks it, so skip the cache for such feeds if unsure.

Can PostRSS read a feed that is cached in Redis?

Yes. PostRSS sees only the feed address and reads any valid RSS or Atom document that is reachable without a login. Paste the address, connect your channels and new articles are shared as soon as they appear.