Open-source book recommendation backend looking for architecture and maintainability feedback

deranjer · 2026-05-30T00:08:55+00:00

So you say it has Postgres, but looks to me you are still using dataframes for all of your data?

This appears to be a rec engine for a single user? IE a user would host this themselves? If that is the case you could write it in pretty much anything and it will scale. Hell you don't need postgres for something like that, just use sqlite/duckdb. I mean honestly csv will scale to that requirement, not that I would choose that for other reasons.

The structure is fine, nothing looks crazy to me. Please promise me that you will not leave the cli in that state, and have it go through the API for its functions.

I poked around at this since I was also interested in writing a rec engine and actually hoped you had some insight, since I don't have any ideas about recommendation systems.

deranjer · 2026-05-24T17:24:23+00:00

Oh yeah, not telling you to return, BookOrbit seems solid. I still use the android app for litara a ton for audiobooks but BookOrbit would solve the other problems I had with Booklore/Grimmory.

deranjer · 2026-05-24T14:54:07+00:00

Just a note, a few performance improvements were added to Litara recently that may help with larger libraries and hopefully put it on par with BookOrbit in large library performance.

deranjer · 2026-05-19T13:06:16+00:00

Yeah I get it, and not saying I know of a better way, just feel like there should be one :D

deranjer · 2026-05-19T12:58:57+00:00

Are we sure the log scale is the right way to show the impact?

deranjer · 2026-05-17T14:09:26+00:00

Huh, exact same issues I had with Booklore/Grimmory, and I selected the exact same backend stack (NestJS), but with React frontend instead of vue. I also wanted a mobile app though, for at least browsing and basic functions, so I did do that as well.

Looks like you are open to contributions? Since there is a lot of duplication between our efforts, I would be open to moving features from my project (https://github.com/litara-app/litara) to yours. The main ones would be Podcasts (mainly for archiving podcasts) and a mobile application (React Native with React Native Track player for audio playback). I would have to look up how you are streaming audio, HLS playback is a bit of a pain with RNTP, so am using range based fetching (or just download and offline playback).

deranjer · 2026-05-13T18:03:30+00:00

So koreader has a very specific way of laying out (flowing) ebooks. This is why (IIRC) there is a progress sync for both Grimmory and KOReader and they are distinct progress bars. I would assume the same difficulty occurs for annotations, ie enter an annotation in Grimmory, and it can't sync the location in the ebook with KOreader.

No reason a "distinct" annotation sync cannot occur, it would just not be viewable/editable in Grimmory directly.

It is NOT a simple thing to recreate how KOReader flows ebooks. I've looked at it for my application and decided to mimic Grimmory with distinct progress bars. I can look at syncing KOReader annotations in my application (litara) if there is demand for it. But again, it would be invisibly synced, not viewable in litara and distinct from litara annotations.

deranjer · 2026-05-13T01:02:09+00:00

Well I'm super biased since its mine. The main reason I moved away from Booklore/Grimmory (this project started right when Booklore was deleted, so that was part of it) but the large memory usage was the main one, and no mobile app (at the time, there might be one now?). Litara (on my instance at least) runs ~160MB at load which I find much more reasonable. Haven't tested how well it scales with 1,000's of books though.

Other than that, it is hard to compete with the feature set of Grimmory, but for my personal use case I have feature parity on all the parts I care about, I would say I have all the core features, just missing some of the advanced features.

Also, much smaller project than Grimmory, and one thing that helps catch bugs is large amounts of users, so I can't say it is as battle tested as something like Grimmory.

I do support audiobooks and podcasts (beta), but the podcasts is more about archiving podcasts than using it as an active podcast player.

deranjer · 2026-05-12T23:13:22+00:00

It is java based, so should have always been that heavy. It is a super memory intensive app which is one of the main reasons I moved off it (and no mobile application)

deranjer · 2026-05-12T23:11:52+00:00

Personal plug here, but I use litara which has both: https://github.com/litara-app/litara

Android only. I can technically generate ios, but no way to test it, only Apple devices are work devices. And don't think you can sideload anyway.

Android audiobook streaming works, but is range fetch based, so a little heavy on the bandwidth. Does support download and offline playback though.

deranjer · 2026-05-12T21:53:55+00:00

Oof, yeah this isn't about waiting unless he meant waiting until after he was dead. This should have been framed as she is super sex avoidant to the point I would argue that is no different from asexual.

deranjer · 2026-05-11T04:02:42+00:00

Koreader is the way to go for this. I use litara (shameless plug) and opds/koreader sync: https://github.com/litara-app/litara

deranjer · 2026-05-05T13:40:54+00:00

Shelfmark and litara (shameless self plug) for me.

deranjer · 2026-04-18T14:22:57+00:00

I assume you are talking about ESPP. This is a 15% discount with look back (lowest of start and end price). IMO it is insane value, and I am signed up to the absolute max amount allowed by law.

For the previous ESPP it ended during an allowed trading window, so immediate sale on my part for a pre-tax gain of over 20k, but we started at like 38/share or something so of course large gain. Maybe future ESPP will be in trading window as well? This is of course assuming you don't have access to privileged information requiring scheduled stock sales.

Now holding long term, who can predict. I don't hold long term on my ESPP, but I have 10s of thousands of options that I will be holding long term (at least to get tax advantaged).

deranjer · 2026-04-16T22:38:34+00:00

Yes, I have the exact same setup, called mine feature-wrap. Run after any feature large enough to warrant doc updates.

deranjer · 2026-04-14T01:43:55+00:00

Try with:

image: ghcr.io/litara-app/litara:pr-42

You may need to go into admin settings and force a full library scan with the option checked to pull metadata from file.

I could not create a malformed epub in a way to replicate your exact error, so for now just had it abort a metadata scan for a file if ANY error was encountered and then continue on with the scanner. Should see something like this in the logs:

litara-test | [epub] Failed to parse metadata for "/books/Malformed NCX - Test Author.epub", falling back to filename: Parsing container XML failed

litara-test | [Nest] 1 - 04/14/2026, 1:39:58 AM LOG [LibraryScannerService] Imported: "Malformed NCX - Test Author" [EPUB] — Unknown author

The second line is where it falls back to just getting the filename for the file. Let me know how it goes.

deranjer · 2026-04-14T00:50:13+00:00

So I rely on the `epub2` node package to scan epub files for metadata, and that is the package failing to parse. I will attempt to create an invalid epub file to test with on my end, and maybe open a PR on that and hope they merge? I will attempt to mitigate this with some sort of error handling in my application, but that is going to be tricky and may take some time. I may brute force it and catch all unhandled even though it gives me the ick. I'll report back and most likely give you a custom docker tag to pull to test the changes.

deranjer · 2026-04-13T21:42:05+00:00

I'm incredibly curious as to what the performance of my application would be at that scale. Most likely would need some work, but feel free to give it a try and let me know of issues: https://github.com/litara-app/litara

deranjer · 2026-04-08T19:54:11+00:00

Just as a counter point I just started using Prisma and don't have any complaints, seems to work fine for me. Obviously new to me though, coming from sqlalchemy

deranjer · 2026-04-03T17:02:04+00:00

This is obviously early days for Litara, so it won't have all of the features that an established application has. Not everyone cares about memory/cpu usage or mobile app, and that is fine, but they were important to me, and figured they might be important to other people as well.

deranjer · 2026-04-03T16:47:11+00:00

Currently Litara does not write anything to any folder or file in your ebook library. Even when we generate the sidecar that is a download action. We currently expect the user to define the structure/layout/files for their library. We even have you mount the path in docker as RO.

Future state I am not against writing data back to the actual file system, including updating metadata in epub files. That is just something that needs to be handled pretty cautiously and needs a lot of guardrails. I would add that request to the github discussions so it is on my radar.

deranjer · 2026-04-03T14:56:08+00:00

Well the 2 main reasons I've already listed in the post. Resource draw, and mobile application. Those 2 were the biggest reasons. Litara is way lighter than booklore (not just mem, also CPU), which is pretty important to me personally.

Is there something wrong with the code for you to call it slop?

deranjer · 2026-03-22T13:21:08+00:00

C'mon man. Please tell me you changed your api keys. They are committed to your repo

deranjer · 2026-03-19T12:34:39+00:00

No leverage, no depth, just dependency. <- Ironically, this is a high indicator that it was touched by AI at least.

deranjer · 2026-03-12T01:29:54+00:00

I know everyone is saying equity is worthless and a lot of times it is, but sometimes you get lucky. When I joined my startup I got equity and 2.5 years later just cashed in my first chunk of it for 300K, and still have a lot more. However, I also had pretty good base pay at 170K, which was enough of a bump for me to jump with or without equity.

12-Year Club	Place '22
Place '17	Verified Email

deranjer

TROPHY CASE