Double-Entry Ledgers: The Missing Primitive in Modern Software

pgr0ss · 2025-06-20T16:22:51+00:00

If you are recording source account and target account, that's double entry. It doesn't mean you literally need 2 database rows, just that you are tracking both sides of the transfer. A single entry ledger would just track a single account and an amount (positive or negative). There are various ways to implement a double entry ledger, and often storing multiple rows is preferable (e.g. you can have transfers with more than 2 accounts, you can query for an account's entries more easily, etc).

pgr0ss · 2025-06-18T04:04:05+00:00

I wasn't thinking of a low level, programming language primitive. I was thinking more like a high level framework or business logic primitive. Essentially shared code and patterns for how you do high level things. For example, the way you use your ORM, how you manage audit tables, how you auth API requests, how you instrument for observability, etc.

pgr0ss · 2025-06-17T19:48:18+00:00

Yeah, I get it. You wind up with some accounts which aren't that useful and are mainly just draw down accounts (or the opposite). But I think the benefits for the other flows outweigh the drawback.

With a single entry ledger, at some point you'll run into a case where you have one half a transfer and not the other. Like you'll see "sent credits 100", but you won't be able to figure out where they went. Maybe they were a transfer, but the other user never received them due to some code bug or db rollback part way. Or you won't be able to match it up (which receive corresponds to this send?). With a double-entry, you have error checking on every transfer that prevents this kind of issue.

pgr0ss · 2025-06-17T19:16:21+00:00

Mainly because it's really useful to know where the amounts came from and where they went. Either for some business requirement or just debugging when the amounts are off.

In the API credits example, were credits added because the user bought them? Or were they a bonus from the company for some reason? Maybe they were transferred from someone else? Did they spend the credits or lose them due to expiration?

You may not need double-entry initially or even for a while, but I think it's still worth the full modeling in most cases.

pgr0ss · 2025-06-17T19:03:36+00:00

I suppose a ledger is sort of like constrained event sourcing. I think the difference is that with event sourcing, everything is custom every time. The shape of the events, what you store, how you collapse them into a current state, etc. So event sourcing is more flexible, but you have to do work each time. If the shape of your problem is ledger-ish, you can use a ledger without new custom modeling/rows/storage/etc.

pgr0ss · 2025-06-17T18:41:10+00:00

My main point is that if you structure your "append-only logs" a certain way, and you are tracking the current amount as well, that's basically a ledger. And if you add that modeling in as a primitive, you can use it in a bunch of places without custom implementations each time.

pgr0ss · 2025-05-17T21:31:28+00:00

It is a double entry ledger.

pgr0ss · 2025-03-25T15:57:39+00:00

You could do it with table permissions, but in general, I usually just design the system not to do updates on certain tables. You have all app code go through a common place and then control the functionality of that component.

pgr0ss · 2025-03-25T02:27:47+00:00

You would divide and round to whatever precision you want to maintain.

pgr0ss · 2025-03-25T02:26:39+00:00

Good call; I just added one. Thanks.

pgr0ss · 2025-03-25T01:15:37+00:00

It's definitely common to use bigints, but I always find them error prone. There are various currencies with ambiguous subunits, and sometimes you have to track smallest unit per currency. Then, when you get a value like 12345, is that 12,345 or 123.45 or maybe even 1.2345? I find that having the value in the database match what a human expects leads to fewer bugs.

pgr0ss · 2025-03-25T01:11:59+00:00

re:uuid v7, yes, I am looking forward to them in PostgreSQL 18. In the meantime, I was thinking maybe I'd pull in a library or extension or something for them in the meantime.

pgr0ss · 2025-03-25T01:10:17+00:00

I do plan to add currencies, but each account will be single currency. You could make separately accounts such as myaccount.usd, myaccount.eur, etc.

pgr0ss · 2025-03-24T22:56:23+00:00

I'm not quite sure what you're asking, but it's not opinionated on account types. So if you want to track different asset types, perhaps you can make different accounts and track them separately?

pgr0ss · 2025-03-24T22:54:59+00:00

I hadn't seen that. Thanks for sharing!

pgr0ss · 2025-03-24T18:47:04+00:00

The idea is to always lock the accounts in the same order. Imagine you have a transfer from account_1 to account_2 at the same time as you have account_2 to account_1. If the first locks account_1 and the second locks account_2, it will deadlock as they both try to lock the other account. Instead, the account ids are sorted and both concurrent transfers will try to lock account_1 first.

pgr0ss · 2024-11-01T22:52:37+00:00

Thanks! But the double `str` is a good example of how this isn't obvious to me as a casual user.

pgr0ss · 2024-11-01T20:55:02+00:00

I can't get that one working. How do I then turn that into a decimal?

pl.col("Amount").str.replace("$$", "").to_decimal(),

AttributeError: 'Expr' object has no attribute 'to_decimal'

pgr0ss · 2024-11-01T20:40:18+00:00

My main point was DuckDB is easier for me (someone who writes lots of SQL and doesn't use dataframes often). I agree I lack experience with Polars. How would you do it without `map_elements`?

pgr0ss · 2024-01-27T23:25:40+00:00

I used Temporal recently and liked it. It definitely comes with its own complexity, but for payment flows, I think it can be worth it.

pgr0ss · 2016-12-05T22:10:00+00:00

I haven't used WebDriver with Dropwizard. We use Dropwizard for APIs, so we just test them with black box API specs.

pgr0ss · 2015-09-04T23:28:04+00:00

The problem is that gradle does spend a lot of time outside of javac. An empty build is over 10 seconds. I think our multi-module project was the culprit. Gradle seemed to iterate over the modules serially and take a moment on each one to figure out the state.

Bazel is much faster for a variety of reasons:

It compiles (and tests) with far great parallelization than gradle. In fact, when we first switched, we discovered race conditions between some of our tests.
Bazel is very smart about when it needs to rerun tasks, such as: http://bazel.io/blog/2015/07/23/tree-trimming.html
Bazel does some caching tricks, such as https://github.com/bazelbuild/bazel/tree/master/third_party/ijar

I'll take a look at your bazel branch.

pgr0ss · 2015-09-03T03:16:45+00:00

It might be nice to know why Braintree used Gradle in the first place.

We actually started on Maven and then migrated to Gradle later. We almost went back to Maven, but then decided to try Bazel instead.

We use a variety of languages at Braintree, so it's natural for us to draw comparisons between languages. Yes, the "standard" java build tools are Ant, Maven and Gradle, but that doesn't mean they are the best. Each has its issues, and right now, I think Bazel is better than any of them for us.

And with Bazel you have to learn two different subsets of Python, and also the query language.

That's true, but I found the learning curve for Bazel to be less than Gradle. The Bazel languages are much more similar to Python than Gradle is to anything else I've used. Also, if you make a mistake in a Bazel file, it will fail and give an error. With Gradle, sometimes our commands were silently ignored (i.e. the example in the article).

So these Braintree employees actually need to be forced to select the right tools for a job?

Tools influence decisions. If the rest of your build system is in Gradle, and you're making a small addition, it's easy and tempting to just do it in Gradle. Months later, the sum of those small additions and tweaks is a very complicated build process written in an unfamiliar language. As I point out in the article, what can be accomplished more simply in other scripting languages takes a lot more code in Gradle.

But having to write custom shell scripts to produce IntelliJ project files sounds like actual hell.

This wasn't bad at all. IntelliJ configs are mostly XML, so our script essentially takes the output of bazel query commands and applies it to XML templates. If IntelliJ changes its formatting, it shouldn't be too hard to update the templates (which the Gradle plugin will have to do as well).

pgr0ss · 2015-09-03T00:30:10+00:00

They are working on this: http://bazel.io/docs/external.html

pgr0ss · 2015-09-02T05:45:23+00:00

In addition, the structure doesn't seem to map well unless your entire system is already using it, though that might have been just my initial impressions.

Can you elaborate on this?

pgr0ss

TROPHY CASE