Open Sourcing My Eternal Pet Project

😅 I finally did it

✅ “Change repository visibility” → “Change to public”

🔗 Here it is: https://github.com/irq0/llar

🖖 Live Long And Read - LLAR

❓ A new aggregator focused on programmability(1) (1)The capability of customizing how news content is fetched, processed, and presented to the user through the use of code.

2016/2017 – after completing my Master’s degree – I took a sabbatical. I dedicated some time to learn Clojure and started the development of what has now evolved into LLAR.

Initially conceived as a backend-only application, its focus was to fetch feeds, process items, store them in a document database, and serve them via a Fever-compatible API(2) (2)There will be a follow up on the essence of news aggregators shortly . CouchDB’s data model was the perfect fit for the application. It even offered features like sync and offline operation, that appealed to me. My primary itch and biggest discontent with other news aggregators was data processing. The ability refine data with code before it reaches my eyes(3) (3)Newsbeuter was a big exception and the filter language and execurl feature a huge influence . Despite many changes over time, this is still on of the core concepts of LLAR.

As the project matured, PostgreSQL replaced CouchDB. The previously valued sync and offline capabilities were underused. The MapReduce query approach increasingly inconvenient. My growing data store led to performance issues and I found myself missing SQL. PostgreSQL was the ideal replacement, with its JSON column type accommodating the key-value feed entry data perfectly(4) (4)This might sound scary denormalized, but isn’t: init.sql .

The introduction of the web-based reader interface slowly replaced the Fever API. Originally, my intention was to integrate with an existing feed reader app(5) (5)If I remember correctly my preferred app was Mr. Reader, that was unfortunately discontinued end of 2016 . This worked temporarily, but certain other itches appeared:

• Discontent with page scrolling dropping me in the middle of a paragraph without any visual cue. Very distracting to reading flow.
• Interfaces focused on reading, but not skimming, tagging, and saving for later.
• The realization that description are often missing or lack value. Out came the idea of a tag cloud of nouns, places/people, and links.
• An appreciation for reading time estimates.
• Strong font and typography preferences.

What took me so long?

Time and pragmatism. Catering only to one user allowed me to work with and around many imperfections. For a significant duration, running an unscheduled update required connecting Emacs via nREPL. Time was a scarce resource creating creative tension between new features, experiments or refining.

Being a stranger in a strange land. I’m not a Clojure professional. Although my proficiency grew, I’m still concerned that my code is not as idiomatic as it should be. Tools like linters – especially clj-kondo – and later AI assistants were invaluable.

The configuration engine was a significant hurdle. How to properly configure software when software is part of the configuration? Until recently, feed sources and processing functions were hardcoded into the application 🤷. Surprisingly, the solution was simpler that expected.

It was hard to let go.

What Helped Me Moving Forward?

Reading Uncurled by Daniel Stenberg on the way back from FOSDEM 2024 and actually working in public for an Open Source company.

Sometimes all you need is a map.

What’s next?

LLAR has been my daily driver for years with my instance having well over 100K items 🙃. My commitment to work on LLAR remains. I’m still as excited to work on it as I was in 2016.

Recently my focus was cleaning up, refining and generally making it useful to others. The immediate focus will continue to be on bug fixes, testing and gathering feedback. Next milestone? Releasing a version 1.0

Beyond 1.0 – the journey continues. Possible directions include replacing OpenNLP with something more contemporary and perhaps integrating an AI API for summarizing or digesting content. Additionally, adding back an API to use apps like ReadKit on a mobile device.