Lighttpd RSS Feed: Config for Headers, Gzip and Redirects

Short answer: Lighttpd can serve an RSS feed correctly with a few lines of configuration. Load the setenv, deflate, expire and redirect modules, give the feed path the RSS content type with a UTF-8 charset, compress the feed types, let the static file handler send its ETag and Last-Modified validators, add a Cache-Control header with a lifetime of a few minutes and redirect old feed addresses straight to the final one. Test the configuration before you restart, check the headers with curl and then connect the address to an auto-poster so that every new article reaches your social channels.

Lighttpd is a small, fast web server that still runs on many virtual servers, home servers, routers and embedded devices, and on sites where someone chose it years ago because it used little memory. It is easy to forget until something goes wrong. A feed that works on a developer laptop can behave differently behind lighttpd: the type may be a generic one, the compression module may not include XML, the cache lifetime may be missing or the feed path may be caught by a rule written for something else. This guide shows a working configuration for a static feed and for a feed that comes from an application, explains what each setting does in plain language and lists the traps that appear when a feed is moved to a new server. Module names follow the 1.4 series, so check your version with the version command of the server before you copy anything.

Check what you are running

Lighttpd changed over the years. Recent releases handle compression with the deflate module, while very old ones used a different module, and some defaults, such as the way validators are produced, depend on the version. Run the command below and note the number. If the version is very old, plan an upgrade, because old releases also lack security fixes, and a feed that goes through an outdated server is a risk for the whole site.

lighttpd -v
lighttpd -tt -f /etc/lighttpd/lighttpd.conf

The second command loads the configuration and its modules without starting the server and prints an error with a line number when something is wrong. Get into the habit of running it before every restart. A typo in a configuration file can stop the server from starting, and the feed goes down with the rest of the site.

A minimal configuration for a static feed

If a script writes a file such as feed.xml into the document root, lighttpd can serve it with its static file handler and nothing else. The block below enables the modules that the feed needs and applies headers only to the feed path.

server.modules += ( "mod_setenv", "mod_deflate", "mod_expire", "mod_redirect" )

$HTTP["url"] == "/feed.xml" {
    mimetype.assign = ( ".xml" => "application/rss+xml; charset=utf-8" )
    setenv.add-response-header = ( "Cache-Control" => "public, max-age=300" )
}

deflate.mimetypes = ( "application/rss+xml", "application/atom+xml", "text/html", "text/css" )

The condition on the URL limits the changes to the feed. Inside it, the type for the xml suffix is replaced by the RSS type with a UTF-8 charset, and the response gets a cache lifetime of five minutes. The deflate list names the feed types explicitly, because a default list is written with web pages in mind. The meaning of the header and the types that tools accept are explained in the guide to the content type header.

Make sure the feed is compressed

An XML feed is repetitive text and shrinks to a fraction of its size when compressed. That matters when many tools fetch the same address every few minutes, and when a feed has grown to hundreds of items. After you add the types to the deflate list, request the feed with a header that accepts compressed answers and look at the response headers.

curl -sI -H 'Accept-Encoding: gzip' https://example.com/feed.xml

A content encoding line in the answer means that compression works. If it is missing, check that the module is loaded, that the type in the answer matches an entry in the list exactly and that the file is not too small to be worth compressing. Some versions have a minimum size or a cache folder setting, so read the documentation for the number you found in the first step. The background on size and speed is covered in the feed compression guide.

Validators and conditional requests

For static files, lighttpd builds an ETag from properties of the file, and it sends a Last-Modified header as well. When a tool sends the validator back, the server answers with a 304 status and no body, which costs almost nothing. This is the best reason to publish a static feed if your content does not change every minute. Two details deserve attention. First, if a script rewrites the file, write the new content to a temporary name and rename it into place, so that no request reads a half-written document. Second, if you run several servers behind a balancer, make sure the ETag does not depend on something that differs between machines, such as the inode number, otherwise a tool will see a new validator on every request and download the whole feed each time. Lighttpd has settings that choose which file properties go into the ETag, and the modification time and size are the safest pair. The mechanism is described in the guide to ETags and conditional requests.

Proxy a feed from an application

When a CMS or an application produces the feed, lighttpd sits in front and forwards the request. The proxy module sends matching paths to a backend, and the same URL condition can add a cache header.

server.modules += ( "mod_proxy" )

$HTTP["url"] =~ "^/feed(/|\.xml)?$" {
    proxy.server = ( "" => ( ( "host" => "127.0.0.1", "port" => 3000 ) ) )
    setenv.add-response-header = ( "Cache-Control" => "public, max-age=300" )
}

The pattern matches the feed path with or without a trailing slash or an extension. Everything else on the site keeps its own handler. Lighttpd does not store proxied responses in its core, so a burst of requests all reach the application. If the feed is expensive to build, let the application keep a short cache of its own, or put a caching layer in front, as described in why auto-posted content sometimes lags. A backend that answers slowly is a separate risk, because tools give up after a time limit, which is explained in the guide to feed timeouts.

Redirects and one canonical address

Keep one address for the feed and use it everywhere: in the link tags of your pages, in your notes and in the auto-poster. If older addresses must keep working, redirect each one directly to the canonical address.

url.redirect = (
    "^/rss$"  => "/feed.xml",
    "^/feed/$" => "/feed.xml"
)
url.redirect-code = 301

Tools follow redirects, but every redirect costs a request, and a chain of two or three is a common reason for a feed that works in a browser and fails elsewhere. If you moved the site from another server, keep the old paths alive for as long as any tool uses them. Make sure the redirect also covers the plain HTTP address if you serve both, and send visitors to HTTPS once. The meaning of each status code is explained in the guide to feed status codes.

Several feeds with one pattern

Sites that offer a feed for each category or language need one rule that covers all of them. A regular expression on the URL that matches every xml file under a feeds folder can apply the same type, compression and cache settings to the whole group. Keep the list of feed paths in one place and review it when a section is added or removed, so that no feed is left with default settings. If feeds are generated dynamically from category pages, make sure the pattern does not also catch ordinary pages that happen to end with the same word.

Do not block the readers

Lighttpd is often paired with protective tools. A limit on connections per address, a rule that blocks empty user agents or a firewall in front can answer a polling tool with an error when several tools share an address. Auto-posters and feed readers do not behave like browsers: they send a plain request on a regular schedule and do not run scripts. Give the feed path an exception in every layer, including a content delivery network in front of the server. When a tool reports that the feed is forbidden or unreachable, check these settings before you look anywhere else. A related case, with its own fixes, is described in the guide to blocked feeds.

Read your own logs

Give the feed its own access log so that the numbers are easy to read.

server.modules += ( "mod_accesslog" )
$HTTP["url"] == "/feed.xml" {
    accesslog.filename = "/var/log/lighttpd/feed-access.log"
}

Look at three things: the share of 304 answers, which should be high when validators work, the share of errors, which should be close to zero, and the interval between requests from the same client. A client that asks every few seconds is misconfigured, and a client that never appears is probably blocked. Compare the log with the posts in your social accounts when something seems late, because the log shows whether the tool really fetched the feed.

Test, restart and verify

lighttpd -tt -f /etc/lighttpd/lighttpd.conf
sudo systemctl restart lighttpd
curl -sI https://example.com/feed.xml
curl -sI -H 'Accept-Encoding: gzip' https://example.com/feed.xml
curl -sI -H 'If-None-Match: paste-the-etag-here' https://example.com/feed.xml

• The first request returns 200 and the RSS content type with a UTF-8 charset.
• The second shows a content encoding header, which means that compression works.
• The third returns 304 when the validator matches.
• The address passes the W3C Feed Validation Service without errors.
• A request from another network gives the same answer as one from your office.

A longer routine is in the feed validation guide, and a shorter one in debugging a feed with curl.

Common mistakes to avoid

• Leaving the generic xml type. Set the RSS type for the feed path.
• Assuming compression is on. Check the answer headers for a content encoding line.
• Writing the feed file in place. Write to a temporary name and rename it.
• Caching for hours. New posts reach social channels late.
• Redirect chains. Point every old address directly at the final one.
• Restarting without a test. One typo can take the whole site down.

How PostRSS works with a feed served by lighttpd

PostRSS has been publishing RSS and Atom feeds to social networks since 2014. Paste the address of your feed, connect your accounts and choose how each post is built from the title, the description or the page title. PostRSS checks the feed as often as every minute and shares each new entry, with its image and link, to the channels you choose among 66 supported networks, messengers, team chats and blogs, including LinkedIn, X, Bluesky, Mastodon and Telegram. Keyword filters, UTM parameters and posting windows let you tune each channel. See the PostRSS features page and the plan limits on the pricing page. More developer guides are under the Technical Guide tag.

قراءة ذات صلة

• Static RSS feed with a cron script
• Does robots.txt block your RSS feed from auto-posting?
• RSS feed security: can someone spoof or hijack your feed?

The bottom line

Lighttpd keeps feed serving small and predictable. Load the modules you need, give the feed path the RSS type with a UTF-8 charset, add the feed types to the compression list, let the static handler provide validators, give the feed a lifetime of a few minutes, redirect old addresses straight to the final one and keep the path outside every limit and bot rule. Test the configuration before every restart, check the headers from outside and then connect the address to an auto-poster, so that every article you publish reaches your social channels without surprises.

FAQ

Which content type should lighttpd send for an RSS feed?

Use application/rss+xml with a UTF-8 charset. Set it for the feed path with a URL condition and the mimetype setting, so that the rest of the site is not affected.

How do I compress an RSS feed in lighttpd?

Load the deflate module and add the RSS and Atom types to its list of types. Then request the feed with an accept encoding header and look for a content encoding line in the answer.

Does lighttpd support conditional requests for a static feed?

Yes. The static file handler sends an ETag and a Last-Modified header and answers a matching conditional request with a 304 status and no body.

How do I redirect an old feed address?

Use the redirect setting with a pattern for the old path and point it directly at the final address, so that tools meet only one redirect.

Can PostRSS read a feed served by lighttpd?

Yes. PostRSS reads any valid RSS or Atom address that is reachable without a login. Paste the address, connect your channels and new articles are shared as soon as they appear.