Elasticsearch RSS Feed: Build It From a Search Query

Short answer: If your articles, products or announcements live in Elasticsearch, you can build an RSS feed from a single search request. Query the index for published documents, sort by the date field in descending order, return only the fields that a feed needs, convert the dates to the RFC 822 form, escape the text for XML and send the result with the RSS content type. Use a read-only API key, keep the cluster private, cache the finished document for a few minutes and connect the feed address to an auto-poster so that every new item reaches your social channels.

Many sites use Elasticsearch as the engine behind their search box, and some use it as the main store for catalogues, news items or documentation. When the data already sits in an index, it is natural to ask whether the feed can come from there as well. It can, and the work is small, because a feed is just a short list of the newest documents with a title, a link, a date and a summary. The difficulty is not the query but the details around it: the right date field, documents that must not be public, the way dates are written in the XML, the time it takes for a new document to become searchable and the fact that a search engine must never be exposed directly to the web. This guide uses the official JavaScript client with Express, but the same query works with any client or with a plain HTTP request.

Check the index before you write code

A good feed needs a small set of fields in every document. Look at your mapping and confirm that you have them, and add them to the indexing process if one is missing.

The date field is the most important one. It must hold the real publication date of the item, not the time of the last update or the time when the document was indexed, because tools use the date to decide what is new. If you re-index everything after a change in the mapping and the date is replaced by the time of the import, every item looks new and auto-posters may publish old articles again. The same trap exists in other systems, and it is explained in the guide to duplicate posts and GUID fixes.

The search request

The request asks for the newest published documents and nothing else. A filter does not calculate relevance scores, which makes it faster and cacheable, and the sort order does the work of ranking.

GET /articles/_search
{
  "size": 20,
  "track_total_hits": false,
  "_source": ["title", "url", "summary", "published_at", "image_url"],
  "sort": [{ "published_at": { "order": "desc" } }],
  "query": {
    "bool": {
      "filter": [
        { "term": { "status": "published" } },
        { "range": { "published_at": { "lte": "now" } } }
      ]
    }
  }
}

The size limits the answer to twenty items, which is plenty for a feed, and the explanation of why a small feed is better is in the guide to the number of items. Turning off the total hit count saves work, because a feed does not need it. The source list returns only the fields that the XML uses, so that long bodies and private fields never leave the cluster. The status filter keeps drafts out, and the range filter keeps scheduled items out until their time comes. If your documents have no status field, add one, because a feed that leaks a draft is a much bigger problem than a missing field.

Build the feed in Node.js

The route below runs the query, converts each hit to an item and sends the document. The helper that escapes text is essential, because titles and summaries often contain ampersands and angle brackets. Here it comes from a small package that handles the special characters, but any tested function will do.

import express from 'express';
import { Client } from '@elastic/elasticsearch';
import escapeXml from 'xml-escape';

const es = new Client({
  node: process.env.ES_URL,
  auth: { apiKey: process.env.ES_FEED_KEY },
});

const esc = (s = '') => escapeXml(String(s));

const app = express();

app.get('/feed.xml', async (req, res) => {
  const result = await es.search({
    index: 'articles',
    size: 20,
    track_total_hits: false,
    _source: ['title', 'url', 'summary', 'published_at', 'image_url'],
    sort: [{ published_at: { order: 'desc' } }],
    query: {
      bool: {
        filter: [
          { term: { status: 'published' } },
          { range: { published_at: { lte: 'now' } } },
        ],
      },
    },
  });

  const items = result.hits.hits.map(({ _id, _source: d }) => `
    <item>
      <title>${esc(d.title)}</title>
      <link>${esc(d.url)}</link>
      <guid isPermaLink="false">${esc(_id)}</guid>
      <pubDate>${new Date(d.published_at).toUTCString()}</pubDate>
      <description>${esc(d.summary)}</description>
      ${d.image_url ? `<enclosure url="${esc(d.image_url)}" type="image/jpeg" length="0"/>` : ''}
    </item>`).join('');

  res.type('application/rss+xml; charset=utf-8').send(
    `<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"><channel>
  <title>Example articles</title>
  <link>https://example.com/</link>
  <description>Newest articles from Example</description>${items}
</channel></rss>`);
});

The guid is the document identifier, which never changes, so a tool can recognise an item even if you edit its title. The date uses the standard method that writes the time in the form that RSS expects, with the Greenwich zone. Writing dates in any other way is a frequent reason for feeds that validators reject, and the rules are in the guide to RFC 822 date errors. The enclosure line is optional and shows how to attach an image, but its length and type should match the real file when you know them. Another way to choose the picture that appears in a post is described in the image guide.

Security: the feed is public, the cluster is not

Never point a public address at the search cluster, and never place the cluster’s own address in front-end code. The feed route belongs to a small server that talks to Elasticsearch over a private network or a tunnel. Create a dedicated API key for it that can only read the one index, and ideally only the fields that the feed uses, and store the key in an environment variable. If the key leaks, the damage is limited to reading public content. Also check that the index does not contain fields that must stay private, such as internal notes, author emails or unpublished text, and keep them out of the source list even if the key could read them. Security rules for the cluster itself, such as authentication and network access, belong to the team that runs it.

Freshness and the refresh interval

Elasticsearch is a near real-time engine. A document becomes searchable after the next refresh of the index, which by default happens about once a second, and sometimes the interval is changed to reduce load during heavy indexing. For a feed this delay is harmless, but it can cause confusion in a test: you publish an article and request the feed immediately, and the new item is not yet there. Wait a moment before you decide that something is broken. If your pipeline indexes documents in batches, check that the batch finishes and that the refresh happens before you announce the article anywhere. Do not force a refresh on every document in production, because that can slow down the cluster.

Cache the finished feed

Search is cheaper than a database join, but it still costs time and CPU on a cluster that serves your visitors, and a feed is requested all day by tools. Store the finished XML in memory or in a cache for a few minutes and send validators with it, so that tools receive a short 304 answer when nothing changed. A pattern with a lock and a stale copy is shown in the guide to feed caching, and the validator mechanism is described in the guide to ETags. If the cluster is slow or unavailable, return the last good copy instead of an error, because repeated errors make tools slow down. The guide to feed timeouts explains the effect.

Feeds for categories and tags

The same query can produce a feed for each section of the site. Add one more filter clause, such as a term on a category field, and give the feed its own path, for example one path per category. The auto-poster then sends each section to the right channel, and the idea is covered in the category filtering guide. Generate the list of allowed categories from a fixed list in your code, not from the request, so that a visitor cannot ask for a filter that you did not plan. An unchecked parameter in a query is how many search-based sites expose more than they intended.

Test the feed

Test the request in the console of your cluster first, then the feed route from outside.

curl -s https://example.com/feed.xml | head -20
curl -sI https://example.com/feed.xml
curl -sI -H 'If-None-Match: "paste-the-etag-here"' https://example.com/feed.xml

• The answer is valid XML and starts with the XML declaration.
• The headers show the RSS content type and a UTF-8 charset.
• Dates are in the RFC 822 form and in descending order.
• Drafts and scheduled items do not appear.
• The address passes the W3C Feed Validation Service without errors.

A longer routine is in the feed validation guide, and a shorter one in debugging a feed with curl.

Common mistakes to avoid

• Using the indexing time as the date. A re-index makes old items look new.
• Returning the whole document. Private fields can leak into the feed.
• No status filter. Drafts and scheduled items become public.
• An admin key in the feed service. Use a read-only key.
• Forgetting to escape text. One ampersand makes the XML invalid.
• No cache. Every poll becomes a search on your cluster.

How PostRSS works with a feed built from Elasticsearch

PostRSS has been publishing RSS and Atom feeds to social networks since 2014. Paste the address of your feed, connect your accounts and choose how each post is built from the title, the description or the page title. PostRSS checks the feed as often as every minute and shares each new entry, with its image and link, to the channels you choose among 66 supported networks, messengers, team chats and blogs, including LinkedIn, X, Bluesky, Mastodon and Telegram. Keyword filters, UTM parameters and posting windows let you tune each channel. See the PostRSS features page and the plan limits on the pricing page. More developer guides are under the Technical Guide tag.

Lectura relacionada

• Express RSS feed in Node.js
• Supabase RSS feed from an edge function
• RSS feed encoding issues

The bottom line

An Elasticsearch index can serve a feed with one short query. Filter for published items, sort by the real publication date, return only the fields that the feed needs, write dates in the RFC 822 form, escape every text value and use the document identifier as the guid. Keep the cluster private with a read-only key, remember the refresh delay, cache the finished document for a few minutes and connect the address to an auto-poster, so that every new item you index reaches your social channels without a manual step.

FAQ

Can I build an RSS feed from Elasticsearch?

Yes. Run a search for published documents sorted by date, map each hit to an item and return the XML with the RSS content type. Keep the cluster behind a small server.

Which date field should the feed use?

The real publication date of the item. Do not use the indexing time or the last update, or a re-index will make old items look new.

Why is a new document missing from the feed?

The index may not have refreshed yet, or a cache is holding an older copy. Wait a moment, then check the cache lifetime and the filters.

Is it safe to expose the search cluster for the feed?

No. Put a small server in front, use a read-only API key and return only public fields.

Can PostRSS read a feed built from Elasticsearch?

Yes. PostRSS sees only the feed address and reads any valid RSS or Atom document that is reachable without a login. Paste the address, connect your channels and new items are shared as soon as they appear.