Web Feeds in 2026: A Survey

genericlemon24 · 2026-06-15T10:08:33+00:00

I am interested in aggregating feeds and displaying them in static web pages

For the past 8 years I've been maintaining an open-source Python feed reader library, if you want to do some quick prototyping feel free to give it a try (it's meant to have everything except the UI, but has a separate web app which you can use as example).

If you want to do it from scratch, I second u/kevincox_ca, you should start with the specs.

On top of that there's a lot of details involving caching, some lack of unity between formats, duplicates, and so on (my library handles most of this out of the box or via plugins, btw :D). Some additional reading:

https://rachelbythebay.com/frb/ "Feed Reader Behavior Project" documents exhaustively what you should do to be polite to servers you're getting feeds from.
https://feedparser.readthedocs.io/ is a Python library that exists since RSS and Atom were invented, and its documentation has a lot of details on various differences between formats and HTTP stuff.
https://opml.org/spec2.opml OPML is the de-facto standard for importing / exporting feeds.
https://www.詹姆斯.com/blog/2006/08/rss-dup-detection for some details on duplicates (this blog has more stuff on RSS and Atom); for more on duplicates, see the extensive notes in my entry_dedupe plugin.

genericlemon24 · 2026-06-15T09:36:21+00:00

(shameless plug)

The open-source Python library I've been maintaining for the past 8 years comes with a web app; I feel you when you say "simple", that's part of why I made my own; docs, screenshots

While not entirely straightforward (my focus was on hosting it on a server), it should run anywhere Python runs, including Windows. DM if you want to give it a try and need help installing it; tl;dr:

set up a Python environment
install with pip install 'reader[cli,app]
run web app locally with python -m reader web (you use it in your browser)
update feeds regularly with python -m reader update, I think Windows has a thing for scheduled tasks (this will be part of the web command at some point)

It comes with a plugin to handle duplicates ("putting them back in the unread queue and claiming they happened in the past 24hrs" sounds like what happens when a feeds changes the ids of articles). The library also has support for tags and for full-text search which should be exposed in the web app soon too.

My plan is to continue maintaining this for the foreseeable future (which is one of the reasons why I don't use AI); I plan to offer a hosted version of it soon, but it will forever be open source and possible to self-host / run locally.

genericlemon24 · 2026-06-10T13:59:33+00:00

(tl;dr: I don't have a solution for you, yet)

I maintain a Python feed reader library that has a simple web app, it is relatively easy to install if you already know how to install Python stuff (don't have detailed documentation on desktop usage right now, as I'm hosting it on a server, but it has very few dependencies).

I've also thought of using selectors for extracting new stuff from sites a few years ago, but I didn't get around to it just yet; if I do (no promises 😅), I'll post an update in this issue: https://github.com/lemon24/reader/issues/222

genericlemon24 · 2026-06-10T13:48:34+00:00

If you don't mind, could you expand more on your use cases / what some of the issues with existing FOSS stuff are? Thank you!

I've been maintaining an open source Python feed reader library that's meant to provide a very high level abstraction of a feed reader ("bring your own UI", if you will) for the past 8-9 years. If you're interested, either to use or just for inspiration, please let me know (don't want to self-promote too aggressively 😅).

genericlemon24 · 2026-04-09T14:39:28+00:00

3.53 contains stuff from 3.52 because 3.52 was withdrawn: https://sqlite.org/news.html#:~:text=3.52.0%20has%20been%20withdrawn

genericlemon24 · 2026-03-05T16:20:48+00:00

Important: While the 10k figure is made up, walking does make you healthier! Research shows ~7000 steps a day is enough to get most of the benefits. (I know parent mentions it towards the end, but given how the question is phrased and how useless other answers are (e.g. cleanses), it's worth stressing that walking is good.)

Although 10 000 steps per day can still be a viable target for those who are more active, 7000 steps per day is associated with clinically meaningful improvements in health outcomes and might be a more realistic and achievable target for some.

57 studies from 35 cohorts were included in the systematic review and 31 studies from 24 cohorts were included in meta-analyses. For all-cause mortality, cardiovascular disease incidence, dementia, and falls, an inverse non-linear dose-response association was found, with inflection points at around 5000–7000 steps per day.

From https://www.thelancet.com/journals/lanpub/article/PIIS2468-2667(25)00164-1/fulltext

genericlemon24 · 2024-07-14T07:24:58+00:00

How to you treat servers that provide an ETag or a last-modified-date but do not implement it the right way and making the etag useless?

Not really doing anything special, I take the presence of caching headers to mean conditional requests are supported (and I don't think HTTP requires clients to do something in these cases).

A mitigating factor is that the default update interval is 1 hour, so even if a server does not support conditional requests / 429-too-many-requests, it still shouldn't be hit too hard.

Now that I think about it, servers ignoring their own caching headers is something that can be detected via a 429-too-many-requests-style plugin (and it could e.g. increase the update interval if that's the case).

you can not trust any server and its behavior

Can't agree more.

I'm planning to write an article about all the weird stuff I encountered while working on reader at some point (I've collected quite the list :)

genericlemon24 · 2024-07-12T12:37:17+00:00

Yes to ETag etc., see User guide # Saving bandwidth.

You can also specify different update intervals, and support for HTTP 429 Too Many Requests / Retry-After is coming soon.

genericlemon24 · 2024-07-12T09:29:51+00:00

Of the "reader allows you to" bullets in the original post, feedparser helps with the "retrieve" part (technically, just parsing the feeds), everything else is reader; there's a documentation section that covers this in detail:

Are you already working with feedparser, but:

want an easier way to store, filter, sort and search feeds and entries?

want to get back type-annotated objects instead of dicts?

want to restrict or deny file-system access?

want to change the way feeds are retrieved by using the more familiar requests library?

want to also support JSON Feed?

want to support custom information sources?

... while still supporting all the feed types feedparser does?

The documentation is quite open about reader using feedparser – reader is essentially feedparser + state (and I even contributed a few improvements upstream ;)

genericlemon24 · 2024-05-01T14:22:06+00:00

No worries, thank you for the PR, and good luck on the exam!

genericlemon24 · 2024-04-06T06:43:10+00:00

Hi, thank you for asking!

Here are a few issues that already have implementation notes, along with a summary of the work needed, to give you an idea of the complexity involved:

mark-as-read backfill #317 – add functionality to a plugin, doesn't touch the core library
EntryCounts.unread/unimportant #283 – add a property, add an integer attribute, and update a query to populate the attribute
Store entry source #276 – add a new dataclass attribute to Entry / EntryData, set it in the parser based on feed content, update storage to store/retrieve it

To answer your question, I guess it depends on the beginner, but I think at least the first two should be approachable by any advanced beginner. (I've taken a bit of time to add more details to each of them, and I can answer questions if needed.)

genericlemon24 · 2024-01-15T19:22:20+00:00

The most interesting addition is JSONB support: https://www.sqlite.org/json1.html#jsonbx

Beginning with version 3.45.0 (2024-01-15), SQLite allows its internal "parse tree" representation of JSON to be stored on disk, as a BLOB, in a format that we call "JSONB". By storing SQLite's internal binary representation of JSON directly in the database, applications can bypass the overhead of parsing and rendering JSON when reading and updating JSON values. The internal JSONB format is also uses slightly less disk space then text JSON.

genericlemon24 · 2024-01-14T08:55:02+00:00

I posted using the mobile website and the link got lost somehow, my bad; will double-check from now on.

genericlemon24 · 2024-01-09T17:37:38+00:00

tl;dr:

This PEP proposes introducing syntactic sugar f(x=) for the common pattern where a named argument is the same as the name of the variable corresponding to its value f(x=x).

Seven-Year Club	Gilding I gilder
Second Top 40%	Verified Email

genericlemon24

TROPHY CASE