HAProxy RSS Feed: Health Checks, Rate Limits and Caching

Short answer: HAProxy can deliver an RSS feed reliably with a few rules. Send the feed path to its own backend with a health check, set the RSS content type and a cache lifetime of a few minutes, compress the XML, redirect old addresses in one step, apply a generous rate limit that never blocks feed readers and keep a small in-memory cache for the feed if the application is slow. Validate the configuration, test the headers with curl and connect the address to an auto-poster so that every new article reaches your social channels.

HAProxy is a load balancer and reverse proxy that many teams use in front of web applications, often with several copies of the same service behind it. It is very good at keeping a site available, and because it sits in front of everything, its rules affect every tool that fetches your feed. A health check that is too strict can remove a working server, a rate limit tuned for browsers can answer a feed reader with a 429 status and a timeout that is too short can turn a slow database query into an error. This guide shows how to configure HAProxy for an RSS or Atom address: routing, headers, compression, health checks, rate limits, a small cache and redirects. The examples use the standard configuration language, so check the documentation of your version before you copy a line.

Route the feed to its own backend

The frontend receives requests and decides where to send them. A named condition, called an ACL, recognises the feed path, and a rule sends matching requests to a separate backend. That gives the feed its own timeouts, checks and limits without touching the rest of the site.

frontend www
    bind :80
    bind :443 ssl crt /etc/haproxy/certs/example.com.pem
    mode http
    option httplog
    http-request redirect scheme https code 301 unless { ssl_fc }
    http-request set-header X-Forwarded-Proto https if { ssl_fc }

    acl is_feed path -i /feed.xml
    use_backend be_feed if is_feed
    default_backend be_app

backend be_app
    mode http
    server app1 10.0.0.11:3000 check

backend be_feed
    mode http
    timeout server 15s
    option httpchk
    http-check send meth GET uri /health ver HTTP/1.1 hdr Host example.com
    http-check expect status 200
    server app1 10.0.0.11:3000 check inter 10s fall 3 rise 2
    server app2 10.0.0.12:3000 check inter 10s fall 3 rise 2

The feed backend has a shorter server timeout than the rest of the site. A feed that takes thirty seconds to build is a problem you want to see as an error, not a connection that hangs and makes every tool retry. The health check asks a lightweight address, and a server is removed after three failures and returned after two successes, which prevents flapping. Slow answers are discussed in the guide to feed timeouts.

Set the headers the feed needs

Response rules can add or replace headers for the requests that match the feed condition. Two headers matter most: the content type and the cache lifetime.

    http-response set-header Content-Type "application/rss+xml; charset=utf-8" if is_feed
    http-response set-header Cache-Control "public, max-age=300" if is_feed

Use the first line only when the application sends a vague type such as text/html or text/plain. If the application already sends the correct type, leave it alone, because it knows whether the document is RSS or Atom. A short lifetime of a few minutes lets caches and readers reuse an answer without hiding new posts for hours; the trade-off is explained in why auto-posted content sometimes lags, and the type in the guide to the feed content type.

Compress the XML

An XML feed is repetitive text and compresses to a fraction of its size. HAProxy compresses responses when you name an algorithm and the content types that should be compressed, and when the client announces that it accepts compression. The default list of types does not always include feed types, so name them.

backend be_feed
    compression algo gzip
    compression type application/rss+xml application/atom+xml text/xml application/xml

If the application already compresses its answers, do not compress twice. After a change, request the feed with an accept-encoding header and look for a content encoding line in the answer. More background is in the feed compression guide.

Apply a rate limit that does not block readers

A rate limit protects the application from a client that asks too often, but a limit tuned for browsers can answer a polling tool with a 429 status. HAProxy tracks request rates in a stick table. The example below allows a generous number of requests for each client address in a short window and refuses only clear abuse.

backend be_feed
    stick-table type ip size 100k expire 60s store http_req_rate(60s)
    http-request track-sc0 src
    http-request deny deny_status 429 if { sc_http_req_rate(0) gt 120 }

Choose a number that is well above what a feed reader or an auto-poster needs, because several tools can share one network address. If HAProxy sits behind another proxy or a content delivery network, the source address is that of the proxy, and all visitors look like one client. In that case, take the real address from a trusted forwarded header, or apply the limit at the outer layer. If a tool reports a 429 status, this rule is the first place to look.

Keep a small cache for a slow feed

When the feed is built from a database on each request, a short cache saves the application from bursts. HAProxy has a built-in cache that keeps small objects in memory. A feed of a few hundred kilobytes fits easily.

cache feedcache
    total-max-size 4
    max-object-size 1048576
    max-age 300

backend be_feed
    http-request cache-use feedcache
    http-response cache-store feedcache

The size is in megabytes and the age in seconds. The cache stores only responses that are allowed to be cached, and it follows the headers that the application sends, so make sure that the response is not marked as private or as not to be stored. If you combine the cache with compression, the order of the filters matters, so test the result and read the documentation for your version. Another option is to let the application keep its own short cache and answer conditional requests with a 304 status, as described in the guide to ETags and conditional requests. Whatever you choose, keep the lifetime short, so that new articles are not hidden.

Redirect old feed addresses once

Keep one canonical address for the feed and use it in the link tags of your pages, in your notes and in the auto-poster. If older addresses must keep working, redirect them directly to the final one.

    http-request redirect location /feed.xml code 301 if { path -i /rss /feed /feed/ }

Tools follow redirects, but every extra hop costs a request, and a chain of two or three is a common reason for a feed that works in a browser and fails in a tool. The redirect from plain HTTP to HTTPS in the frontend counts as a hop, so always publish the secure address. The meaning of each status code is explained in the guide to feed status codes.

Keep links correct behind the proxy

Feed entries must contain absolute links that begin with the public address of the site. An application behind a proxy sometimes builds these links from the internal address or the wrong scheme, so every item points to a page that readers cannot open. Make sure that HAProxy sends the original host and the protocol, with the forwarded protocol header shown above and the forwarded address option, and that the application is set to trust them. After any change, fetch the feed and look at the link of the first item.

Do not block the readers

Besides rate limits, two other protections cause quiet outages. Rules that deny requests without a user agent, or with an unusual one, also block feed readers and auto-posters. A firewall or a bot filter that sits in front of HAProxy may challenge requests from data centres. Give the feed path an exception in every layer. When a tool reports that the feed is forbidden or unreachable, check these settings before you look anywhere else. A related case is described in how a content delivery network blocks a feed.

Validate, reload and test

HAProxy can check a file before it applies it, and a reload keeps existing connections alive.

haproxy -c -f /etc/haproxy/haproxy.cfg
sudo systemctl reload haproxy
curl -sI https://example.com/feed.xml
curl -sI -H 'Accept-Encoding: gzip' https://example.com/feed.xml
curl -sI https://example.com/rss

• The first request returns 200 and the RSS content type with a UTF-8 charset.
• The second shows a content encoding header, which means that compression works.
• The third shows one permanent redirect to the final address.
• Stopping one server leaves the feed available, which shows that the health check works.
• The address passes the W3C Feed Validation Service without errors.

A longer routine is in the feed validation guide, and a shorter one in debugging a feed with curl.

Common mistakes to avoid

• One backend for everything. The feed cannot get its own timeouts and limits.
• A health check that is too strict. Working servers leave the rotation.
• Rate limits without headroom. Auto-posters receive 429 errors.
• Double compression. Choose one layer and check the answer headers.
• Internal addresses in feed links. Pass the original host and protocol to the application.
• Skipping validation. A typo stops the reload.

How PostRSS works with a feed served by HAProxy

PostRSS has been publishing RSS and Atom feeds to social networks since 2014. Paste the address of your feed, connect your accounts and choose how each post is built from the title, the description or the page title. PostRSS checks the feed as often as every minute and shares each new entry, with its image and link, to the channels you choose among 66 supported networks, messengers, team chats and blogs, including LinkedIn, X, Bluesky, Mastodon and Telegram. Keyword filters, UTM parameters and posting windows let you tune each channel. See the PostRSS features page and the plan limits on the pricing page. More developer guides are under the Technical Guide tag.

Související čtení

• Static RSS feed with a cron script
• RSS feed XML declaration error
• Express RSS feed in Node.js

The bottom line

HAProxy keeps a site available, and a few rules keep a feed dependable too. Give the feed its own backend with a light health check and a short timeout, set the RSS type and a lifetime of a few minutes, compress the XML once, apply a rate limit with plenty of headroom, add a small cache if the application is slow, redirect old addresses straight to the final one and pass the original host and protocol so that links are correct. Validate the file, test from outside and then connect the address to an auto-poster, so that every article you publish reaches your social channels without surprises.

FAQ

Does HAProxy compress RSS by default?

No. Name a compression algorithm and the feed content types, then check the answer headers for a content encoding line.

Can HAProxy cache a feed?

Yes. Its built-in cache keeps small objects in memory for a limited time. Keep the lifetime short, and test it together with compression.

Why do my tools receive a 429 status?

A rate limit is probably too strict, or all visitors share the address of a proxy. Raise the limit for the feed backend or use the real client address.

How do I redirect an old feed address?

Use a redirect rule with a permanent code and point it directly at the final address, so that tools meet only one redirect.

Can PostRSS read a feed served by HAProxy?

Yes. PostRSS reads any valid RSS or Atom address that is reachable without a login. Paste the address, connect your channels and new articles are shared as soon as they appear.