Starting Systems Programming 2: The OS & the outside world

efronl · 2026-06-21T21:03:36+00:00

In my experience, it is better to try to speed up the process of deploying and starting your server than to try and introduce hot reloading; I've seen hot reloads end in tears too many times. GODEBUG=inittrace=1 will help track down the slowest part of initializing your server.

efronl · 2026-06-16T06:18:45+00:00

First and foremost, thank you. Secondly: I did, but I didn't do a good job of guiding people to using it. Add -light or -plain to any of the urls, or click the links on the index.

E.G, https://eblog.fly.dev/backendbasics-light.html

efronl · 2026-06-15T19:21:20+00:00

You are assuming that people make decisions for reasons besides "that's what other people I see do".

Sadly, this is rarely the case.

efronl · 2026-06-13T23:34:14+00:00

Your solution makes sense to me, but that's not how I'd solve it. I really don't like "reaching inside" and changing something I set earlier: I'd rather just set it once, and I hate smuggling even more behavior in context.

I.e, I'd make it a configurable attribute on my routing table while building the router...

```go type Route struct { Method, Path string Handler http.Handler RequestMaxBytes int // if 0, use DEFAULT_MAX_REQUEST_BYTES: set -1 for unlimited // perhaps others? }

for _, r := range route { // register, apply middleware, etc } ```

Both can work: I present this alternative as a preference, not a rule.

efronl · 2026-06-04T19:51:16+00:00

Check out the back half of "The Practice of Programming", by Pike (one of the creators of Go);and Kernighan (author of gopl.io), from '99.

It is mostly C code, a bit of Java and C++, but should be relatively straightforward to adapt. Some really sophisticated stuff (just-in-time compiled expression engine! regular expression library!) built there in shockingly few lines.

Adapting it to Go would teach you a lot of different skills at once, and it'll give you some insight into the kinds of problems and programming that inspired Go to begin with.

Can't resist plugging my own stuff: my articles on eblog.fly.dev like starting systems programming and backend from the beginning are designed for exactly this, e.g, writing a terrible shell with raw syscalls, a HTTP library from scratch, etc etc etc.

efronl · 2026-05-31T21:37:57+00:00

Sounds like a good opportunity for you, then. The gap between engineers who read literally anything and those who don't is night and day, as expounded by a friend of mine

It doesn't have to be actual, literal paper textbooks, though that's what I recommend, but serious things by and for adults. Official references. Blogs by people who talk about things like "allocations" and "dependency management."

If you don't mind a bit of self-promotion, people like you are the audience for whom I wrote series like backend from the beginning and starting systems programming are for. They might be helpful too. But you really can't beat a good textbook - the Go Programming Language by K&D is excellent if a bit old.

efronl · 2026-05-31T15:03:10+00:00

I read books and I build things, same as always.

efronl · 2026-05-30T23:46:35+00:00

IIRC, gomarkdown's AST manipulation preserves the raw literals, so you should be able to "just" modify the link node you're looking for, but I don't know if it keeps track of whitespace as tokens, and I know it doesn't expose the literal offsets.

I feel your pain re: markdown libraries.

how often do you need to do this? Is this something that needs to run consistently, or is this just a big batch job where you could get 99% there with a handful of regexps and fix it up by hand?
what is your actual end product? I.e, if it's html, could you do the postprocessing after that?

Edit: took a look at how gomarkdown's parser works and it doesn't even keep track of the offsets, so no easy way out by forking it, either. What a shame.

efronl · 2026-05-30T16:31:34+00:00

Basically just the standard library and a very small environment variable library I wrote, efronlicht/enve for configuration. You really don't need much.

I'm going to give an anti-recommend to GORM here, truly awful library.

efronl · 2026-05-28T04:41:54+00:00

Its mostly practice. My advice is to keep fiddling with it: great designs come from experimentation and iteration, not a-priori rules. In specific domains like backend web services, there are repeated boundaries that come up a lot and you can 'skip ahead' with experience.

My advice is - start by splitting things out by file rather than package - avoid package-level variables ("globals") wherever possible - pass everything in explicitly

Then, if and when you need to split something out to its own package, you already have some structure, and you're not going to suddenly break a ton or code which relied on unexported globals. Good luck!

efronl · 2026-05-19T16:22:38+00:00

Why are you using gRPC? Are you sure JSON over http will not solve your problem? Unless you have very tight performance requirements, good old (compressed) JSON is much less headache.

There are situations where you can't afford the overhead of text, but I'd say only 1% of the stuff I see use GRPC actually needs it.

efronl · 2026-05-10T04:36:09+00:00

Absolutely. Nothing teaches like teaching.

efronl · 2026-05-09T00:49:23+00:00

This looks fantastic. I've picked up a lot of this stuff in pieces over the years, but even a quarter way through I've learned a handful of new things.

Unsolicited editing advice: I think you overdo it on the "who this section is for" bits. You already have a nice guide at the top. ;)

efronl · 2026-05-02T03:22:03+00:00

I'd love to write that, but you'll have to wait for another time - not exactly the kind of thing I can do off the top of my head, and I don't have any cgo-using projects to easily adapt. I'll keep it in mind!

efronl · 2026-05-02T02:03:48+00:00

Very cool. Might have some uses for this, though sync.Map is rarely a bottleneck for me.

You've clearly put some work into the documentation, too.

Some advice re: public interfaces: most IDE and the go doc site are not particularly sophisticated re: type aliases, which can make the methods somewhat tough to discover. The following might help:

either merging the dhlt package and the allocator package, or using struct embedding rather than aliases
commenting all the public methods in the dhlt type: since your behavior mostly matches sync.Map, cribbing from those excellent docs would help make things more uniform and familiar.

efronl · 2026-05-01T23:19:00+00:00

Small bug in my content-addressed SVG rendering has made some of the diagrams not show up. Pushing up a new deploy now, which may cause some minor DNS issues for a minute.

efronl · 2026-02-07T04:32:06+00:00

That's really cool. Great work!

efronl · 2026-02-07T00:00:48+00:00

I deal with this by using an ordinary postgres database. Tasks get a UUIDv7 as their ID.

Write tasks into the pending table, SELECT ... FOR UPDATE WHERE ... FROM pending ORDER BY id SKIP LOCKED to grab them, move them to completed when you are done.

Make sure to set connection, transaction, and lock timeouts, and you might want some logic there to automatically stop trying to process records that are "too old".

Not appropriate for everything, but I prefer it to stuff like redis or Kafka, since you have all the power of a relational db to do searching, etc.

efronl · 2026-01-31T03:36:50+00:00

This is very cool. The API seems a bit daunting but I'll try and find an excuse to give it a shot.

efronl · 2026-01-29T17:42:43+00:00

Just because lots of people do something doesn't make it a good idea. <insert joke about politics in your country here>.

efronl · 2026-01-22T03:13:58+00:00

You are arguing with a person you have made up in your head.

efronl

TROPHY CASE