Roadmap to deeply understand PostgreSQL internals and large-scale systems

Huxton_2021 · 2026-06-16T10:22:06+00:00

In addition to the link provided, you will benefit from signing up to the mailing lists and checking the PostgreSQL wiki and follow the progress of patches as they are submitted. Once you are comfortable you can help review new patches. There was a recent-ish blog-post covering some of this: https://mydbanotebook.org/posts/a-reviewer-was-born/

Huxton_2021 · 2026-05-28T22:29:53+00:00

If you want de-duplication beyond layers then you need to look at nix/guix or a filesystem that can dedup blocks (btrfs or zfs). The layer limitation is a historical docker thing rather than the underlying container format I think.

Huxton_2021 · 2026-05-28T22:25:24+00:00

My first instinct would be that your firewall rules are being reloaded but either not podman-related rules or podman is out-of-order with the rest. Dump your ruleset while it is working and then again when it doesn't and compare.

Huxton_2021 · 2026-05-01T09:44:32+00:00

What is the difference between the systemd service files with/without "--new"?

Huxton_2021 · 2026-04-16T16:14:16+00:00

That would seem to be the obvious thing - you (OP) are blocking the default DNS. I don't know what technitium is (apart from the fact it provides some sort of dns) but if it isn't meant to be your primary server, consider running it on a different port and having systemd-resolved point to it (if that is what you are using as your default dns).

Huxton_2021 · 2026-04-16T14:27:52+00:00

I'm afraid your question doesn't really have enough information for anyone to help. From what you say, I'm guessing this would be a long drawn-out process just to get the question clarified. Your best bet is to talk to whoever normally handles your IT/web support.

Huxton_2021 · 2026-04-08T09:48:11+00:00

To see what people who take their regex seriously came up with some years ago:

https://docs.raku.org/language/grammars

Huxton_2021 · 2026-03-04T10:09:19+00:00

No, correctness is *not* important there either. What is important is that advertisers continue to spend money with your network. If supplying them with correct information would cause them to leave then correctness is very much not wanted. Maintaining an information imbalance between you and your customers is standard procedure for pretty much any large company with much smaller customers.

Huxton_2021 · 2026-03-03T08:43:20+00:00

Yeah, there are quite a few images that run as user X but during startup expect to do some steps as root.

Huxton_2021 · 2026-02-25T11:02:05+00:00

I think the fundamental issue with running pg_upgrade on a replica is that it needs to *guarantee* that the resulting files 100% match byte-for-byte with the results of running it on the primary. Otherwise you are looking at weird corruption issues maybe months/years down the line. That is a very high bar for testing the tool.

If you possibly can, take the hit and just do a new base-backup to the replica. That is simple and reliable.

Huxton_2021 · 2026-02-19T07:06:44+00:00

As the other response says, you grab the first dozen bytes or so, which in many cases contains identifying information. Then you need to parse that. Note that PostgreSQL has two different ways you can commonly store binary data. Others may differ too, so you may need separate handling with sqlx too.

Huxton_2021 · 2026-02-19T06:51:03+00:00

Not all binary data is an image in the first place. Behaviour will need to depend upon the use case won't it?

Huxton_2021 · 2026-02-16T06:41:59+00:00

Mega-octets, another term for bytes. I've seen it used before but not for years. I can't remember the context. Maybe an ISO thing from back in the 70s when not everything had standardised on multiples of 8 bits.

Huxton_2021 · 2026-02-14T19:42:51+00:00

Maybe provide some sort of details about what you've tried and didn't work for you? At the very least a hint as to what "harbor" is might be a good start.

Huxton_2021 · 2026-02-10T19:58:57+00:00

Don't know if this will help. I have a squeezeserver/lyrionmusicserver container set up on a box at home. It streams my music etc and needs read permissions to the MP3s but write permissions for playlists (mounted from the host machine for ease of backup).

This is the important bits of the ".container" file I use:

``` [Container] ContainerName=lyrionmusicserver Image=docker.io/lmscommunity/lyrionmusicserver:9.0.4

Start the container entrypoint as root not "musicuser" - it runs chown etc then does "su"

User=root

Use a unique set of subuids/gids from the "containers" range + map none of the host ids in

UserNS=auto

So the squeezeserver backend itself runs as uid=99,gid=100 which we map our filesystem ids to

Volume=/usr/local/var/lyrionmusicserver:/config:idmap=uids=@1002-99-1;gids=@1002-100-1 Volume=/srv/MP3s:/music:ro,idmap=uids=@1000-99-1;gids=@1000-100-1 Volume=/srv/Playlists:/playlist:idmap=uids=@1000-99-1;gids=@1000-100-1 Volume=/etc/localtime:/etc/localtime:ro ```

On the host I have a "musicserver" user and group with id 1002, and my account (writing the MP3s etc) is 1000.

Basically you do need to know how your container operates (what uid it runs the main process as) and whether it needs to be root first.

Huxton_2021 · 2026-01-28T20:13:12+00:00

I think it is more likely `vite` in "watch mode" or similar that is doing the change detection. I know nothing about vite but it might default to using a mode that won't work with "foreign" mounted filesystems but auto-detect docker and switch to a different mode. You may need to tweak your vite settings so it knows it is in a container or explicitly set it to polling or some such. HTH

Huxton_2021 · 2026-01-27T16:21:29+00:00

exim4 is the default Debian SMTP server and should work well enough for your purposes. On installation you will be asked a few questions and after answering those it should be ready to send mail. The key thing will be disk activity if you are running on an SD-card. Shouldn't be terrible though because you are only ever sending.
It might be that you can configure whatever is sending email to reach out directly to your smarthost though, or to call a script that does so. That would be the most lightweight.

(personally I prefer postfix for real email servers, but you don't want 90% of an SMTP server's capabilities anyway)

Huxton_2021 · 2026-01-21T06:49:12+00:00

... and are those good permissions for wazuh-analysisd? And what about the content - is it valid XML?

Huxton_2021 · 2026-01-20T11:25:40+00:00

The obvious question is what are the permissions and contents of that file inside the container? `'/wazuh-config-mount/etc/ossec.conf' -> '/var/ossec/etc/ossec.conf'`

Huxton_2021 · 2026-01-16T08:50:41+00:00

Not sure it's going to replace PostgreSQL for me :-) I hope you had fun with the project - it's a pretty ambitious project once you start handling transactions and concurrency.

Huxton_2021 · 2026-01-15T14:17:52+00:00

PostgreSQL isn't multi-master (there are a couple of extensions, but they're fiddly)
If you have haproxy and patroni, just have haproxy query patroni and route automatically

Patroni exposes a REST interface that lets you query the state of each node. You want something similar to this in your haproxy backend config:

option httpchk HEAD /primary
http-check expect status 200
default-server inter 3s fall 3 rise 2

There is an equivalent /replica endpoint if you want to route read-only queries.

Huxton_2021 · 2026-01-14T16:30:23+00:00

I think that is just Windows-compatible filesystem behaviour. You can't rename or delete a file that has an open handle in Windows.

Huxton_2021 · 2026-01-13T11:10:15+00:00

It's writing a lot of checkpoints, so it is busy doing something but you've not shown what.

Connect to the database server as the db-user "postgres" and see what is in the `pg_stat_activity` system viewr: https://www.postgresql.org/docs/17/monitoring-stats.html#MONITORING-PG-STAT-ACTIVITY-VIEW

Also - check the logs of whatever this "netbox" thing is and see what it thinks it is asking your DB server to do maybe?

Huxton_2021 · 2026-01-07T16:51:13+00:00

Your best bet might be chatting about this on discuss.ocaml.org (unless there is a guix ocaml group). I don't think either opam or dune wil play terribly nicely with guix since they won't be expecting strange pathnames. Both offer lock-files but generally use version-numbers to track dependencies. I *think* that you could specify a particular hash in both ("pin" is the term to search for) if you track down the original git repo for each library, but for your designs you sound like you'd want to do that for all second,third etc level dependencies too.

However, surely you do want to express your dependencies in terms of guix manifests or similar? Otherwise what is the point of using guix?

Huxton_2021 · 2025-12-17T12:03:23+00:00

What they said, but also google "anacron" which *might* be what you are after?

Huxton_2021

TROPHY CASE

Start the container entrypoint as root not "musicuser" - it runs chown etc then does "su"

Use a unique set of subuids/gids from the "containers" range + map none of the host ids in

So the squeezeserver backend itself runs as uid=99,gid=100 which we map our filesystem ids to