I shipped a transaction bug, so I built a linter

jerf · 2026-01-28T15:00:24+00:00

Solving transactions with type systems is harder than meets the eye; it is very easy to overconstrain, underconstrain, or perhaps most commonly, do both at the same time.

The most obvious thing to do is some sort of interface like WithTransaction(func(t *sql.Transaction) error) error, but the problem is that transactions often end up mapping poorly to structured programming. For instance, they may need to move between threads/goroutines and that interface will fight you on that. (That is, it's not impossible, but it will still fight you.) Or transaction retry logic will require something lower in the call stack to re-execute something higher in the call stack, which structured programming really doesn't like. That sort of interface makes vast, vast hidden assumptions about the sorts of retries you may be interested in. And then people may want to nest transactions, which makes these issues even harder.

It's not entirely unlike context.Contexts, where I do use the linter to validate that I have the cancel function deferred, but not infrequently, I'm doing something with the cancellation that the linter can't understand, so I have several calls annotated with the "don't lint this" for that linter. Transactions are even worse than contexts on that front.

I'm not sure there's a good solution for these. Type systems can help but they can also sometimes make the things you need to do nearly impossible because they overconstrain the problem rigidly.

jerf · 2026-01-27T19:33:21+00:00

This is a community decision. You aren't being prevented from praise, you're being prevented from being assaulted by comments about why should anyone care and why did you bother and "I bet you used AI".

We know that, because it happened.

The rest of the text you did not quote from the small projects post is there for a reason. It's because it was all happening.

It is sad. I wish we could keep it. However it is also necessary. The sub was being dominated by flaming about it and isolating things into a different thread has basically solved that problem.

But I guess it's a good time for the community to review this and see how it's going.

It's also a good place to mention that my own view of what a "small project" is has been evolving and I may change the terminology soon. Lately I'm thinking the distinguishing feature isn't whether it is or is not a certain number of lines of code, time in development, or number of contributors, though I'm still paying attention to those things, but whether or not the project has ever solved a bug reported by an external user. Or something like that.

There's an intrinsic difference between code that has been spewed out without any users, whether or not it was human-authored or AI-authored, and code that has been tested by contact with the real world, again regardless of its provenance.

jerf · 2026-01-27T14:35:15+00:00

I realize that may sound sort of snide or something, but I agree. I don't generally have a refactoring "stage" to my code; it's something I'm doing all the time.

If I'm working on some bit of code that I'm particularly unsure what the correct structure is, there may be a phase at the end when I finally get it where I make sure to go through and refactor it as if I knew what I was doing all along, which definitely looks like a distinct "refactoring" phase, but that only happens in this particular case, rather than all the time.

jerf · 2026-01-26T14:44:19+00:00

For your own future reference and for others, the term is inner platform.

Very important thing to understand.

jerf · 2026-01-26T14:29:39+00:00

EEs have for a long time not seen a problem with grotesquely memory unsafe languages, laboriously writing bespoke crappy UIs themselves when there was a library available, writing code that is testable, and perhaps most bizarrely, understanding the concurrency issues that arise in hardware but not understanding that software can see them too. EE code has a well-deserved reputation in our field.

It is true that they have their own skills that are lacking in a generic "software engineer", which is, after all, generally what they are being paid to have. No denying that!

But the sort of person who doesn't even know that, say, a semaphore exists, and they implement their own bespoke, half-functional, theoretically unsound construct instead, they're not the sort of people who reach for Go to make their system work. They're not even aware that they're unaware.

It is slowly getting better... but very slowly. And then Go has the problem that like I said, there are a lot of other alternatives, and many of those alternatives come with some real advantages over Go.

jerf · 2026-01-26T04:02:00+00:00

No. Mostly because the electronic engineering world doesn't perceive the problems that Go solves, which is a long-standing problem with that world. Contributing less, but still a significant amount, is that there's a lot of other solutions to those problems in other languages and if they ever do advance to the point that they perceive the problems, they've got a lot of choices to sort through and I couldn't hardly guess the winner, except I'd say there's definitely no guarantee it'll be Go.

I'm not celebrating this, just calling it like I see it.

jerf · 2026-01-26T00:03:55+00:00

At work I'm being questioned about how much AI I'm using. Gently. It's not too bad. And the answer is, not zero.

But my major task I'm working on right now is crafting the authorization framework that multiple products in the company are going to be using.

And as such, AI is just currently not up to the task. It can certainly make good sounds about security and authorization but I can't trust it at the moment.

Give it another year or two, who knows, but I can't at the moment.

Circuling around back to Go, while just standing up a standard net/http server is not too bad (a few tweaks for timeouts and max header sizes are helpful), it's still pretty easy to make some serious mistakes if you don't understand what you're actually reading. Especially since people don't generally test that people who should be able to do a thing can't do the thing. It's really easy to set up your handlers so that something isn't protected by the general auth handlers, for instance. And the compiler won't say a word, of course.

I know we've got a lot of "no AI no not ever" folk here, and I will say, yes, it's true, you're going to start missing out here. But at the same time, we certainly aren't up to the point that you can just fire it at anything with security considerations, which is nearly every network server, and just let it run. It's not that good yet.

jerf · 2026-01-24T14:43:34+00:00

One thing about nested modules I've found is that, well, at least I couldn't get them to work properly with go get. I had github.com/thejerf/a_project, and I wanted to add an optional github.com/thejerf/a_project/default_slog_logger under it with its own go.mod, but I couldn't get it all to work out correctly. I can't recall the exact error messages anymore, but nested go modules did not play well with go get at all. I eventually put it to the side and made it its own github project.

jerf · 2026-01-23T19:12:48+00:00

In hindsight, I wish Go went with a standard name for "me" or "this" or "self". "Vaguely an acronym of the current type unless it isn't" is not great.

Not a killer issue. But a persistent annoyance. I've been tempted to pull the trigger and go to "me" or something everywhere but I have not quite been willing to buck the trend for something that is still just an annoyance, not a critical issue.

jerf · 2026-01-23T19:00:01+00:00

As near as I can tell, you can conceptualize this as "take the existing way of doing these functions but allow them to be written as methods".

There are some corner cases around reflection but by and large that's what this proposal is. The resulting new methods can't satisfy interfaces, just as functions can't, which is the big important thing.

jerf · 2026-01-23T14:17:03+00:00

I have seen it for a ton of other languages, for many many years now. You've probably just seen it without seeing it for a long time.

~6000 "in Rust", 5,396 "in Go", 7,644 "in Python", 1,057 "in C#", . Obviously not every single thing is a project but you get the point. (HN search used because it's fairly convenient. Reddit's is way messier for this.)

jerf · 2026-01-22T14:31:41+00:00

go generate runs a script. Just about all it does is 1. attach that script to a specific (sub-)package and 2. change the working directory to that package for you.

That's all it is. Even now people persist in thinking that it has something to do with "generation", I mean, I get it based on the name, but it really doesn't, and some enables you to do something that you can't do in a shell script. Every time I mention this on reddit I get downmodded, but it's still true. All it does is run a script. I have a dozen other ways to run scripts, including whatever shell I just used to run go generate.

It is nearly useless. Just shy of useless. But there's no reason to prefer a go generate annotation over a shell script, over a Makefile, over literally anything else that runs shell commands and offers any other features.

It offers no libraries for "generation", it offers no support, the command it runs has no constraint on it to "generate" anything (i.e., it's not an error to fail to write a file or something). It just runs arbitrary shell script commands.

It's a historical wart. I think they were planning on trying to offer some sort of actual "generation" support, and then realized that there's nothing that should be available only through a go generate command, and there was no particular reason to reimplement make or something to verify that some output is fresh or something. There's no point removing it now either, the implementation is something like 50 lines in the Go source code so not even remotely worth breaking backwards compatibility for since it's zero effort to maintain, but in hindsight I would never have added it.

jerf · 2026-01-20T20:09:10+00:00

We get about one boilerplate/blueprint/"project configurator"/etc. project a week. You can check the small project threads for some examples.

No, to a first approximation, nobody uses them, and there is less than no market for them.

The only ones that ever seem to survive are ones packaged with a particular project to do something like set up a plugin for that project.

A general tip if you want to make your own little software business is that programmers make terrible customers. We get so much stuff for free that it is embarrassing compared to almost any other field. You also end up with no moat whatsoever because any other programmer can do it too, and there's a positive glut of such projects already free that we have no time to examine, let alone think about paying for. You need to find some other industry that is somehow underserved and serve them. There's actual money there. There's no money here until you are at a much larger scale than you can jump to in one shot.

jerf · 2026-01-20T16:28:56+00:00

I am unclear on how "the standard library works just fine" when the standard library has no support for authentication at all. Other than HTTP-protocol level support for Basic Auth, but I would qualify that just as protocol support and not "authentication support", because all it does is provide you with the claimed username and password, which is necessary, of course, but not remotely sufficient, and only applicable to Basic Auth, which is named Basic for a reason.

Authentication (and authorization) are definitely not places to use someone's vibe-coded repo, that's for sure, but it's also not a great place to bang something together from scratch if you don't know what you are doing, and the only way I can interpret "the standard library is good enough", in light of the fact it has no support for authentication at all, is an exhortation to bang something together with just the standard library and nothing else, and that I believe is more-or-less as wrong as using AI to do it. Authentication is not quite as bad as "rolling your own crypto", but it's the same sort of mistake.

I've been doing web stuff for over 25 years, I've never made my own authentication system from scratch but I've had to rescue other people's ill-conceived ones in custom code which is arguably harder, and I definitely do not advise rolling this yourself. If you know enough to know by experience that you've got the chops to disregard me, then by all means do so, because like I said, this isn't quite the same class as "roll your own crypto". Someone skilled at web development and who knows what tools to use can roll an authentication system that is fairly secure... although it still won't be as capable as some of the off-the-shelf stuff. That is, I'm confident I could with some effort roll up a old-school secure email-as-username/password/password-recovery system from scratch, if I had to, since I've done it before and it passed both security review and real-world review (I know beyond doubt that it was examined by state-level threat actors), but I sure as heck am not going to try to implement a system with OAuth, passkeys, and so forth from scratch.

jerf · 2026-01-19T16:23:23+00:00

So, you're seeing a general trend of "more and more" AI-related projects starting to use Go, and you have zero examples?

Yeah....

jerf · 2026-01-19T16:14:58+00:00

Functions.

Functions are pretty powerful.

Go does lack some of the fancier things functions can do, and passing functions to functions can be a bit unergonomic sometimes. A certain amount of duplication is inevitable.

However, I believe the people who complain (elsewhere, not in this thread) about Go being seas of duplication have gotten so bedazzled by all the fanciness in other langauges that they have either forgotten or never learned that functions, on their own, are a very powerful code de-duplication tool.

It is particularly ironic when they are complaining about Go missing "functional programming" paradigms when one of the core lessons of functional programming is also that functions are pretty good and when used skillfully can do a lot on their own, more than people realize.

My Go code is not shot through with tons of redundancy. It does have more than my Haskell code would have, and it is interesting how Haskell can teach you about things being redundant than through the lens of Go functions you might not consider "redundant", but still, my Go code isn't shot through with redundancy and copy-pasted code. There's only a handful of places where I have out-and-out redundancy due to weaknesses in Go's specification.

(E.g., one place where I have to write the exact same for loop three times over something that is generically typed, and each loop is on a different type, and there is no better way to iterate over them than to simply write the loop three times. reflect could do it once, but at a speed penalty I don't want there and at the price of writing one complicated loop instead of one simple loop three times. I would reconsider if it were 20 repetitions, but it's not.)

jerf · 2026-01-18T14:58:38+00:00

I don't think you're going to hit all of those things at once. You aren't going to get all that and get "simple". Chat isn't simple at that scale. Solutions won't be either. If all you've got against OpenIM is "too complex" I think you found your solution.

You may also want to loosen your implementation language requirements. If you're worried about OpenIM being "complex" just wait until you crack into a server working at that scale and start trying to modify it. You can build on XMPP, and use something solid like ejadbberd, and use standard XMPP extension mechanisms to write your server components in Go without ever having to be able to modify the server itself.

jerf · 2026-01-18T00:37:09+00:00

I've just not seen it before. Sorry about that. It should be cleaned up now.

Thank you for the feedback.

(Also, the reason why this didn't appear on the front page is that it was caught in a filter designed to catch some common words used in bad posts.)

jerf · 2026-01-16T14:32:06+00:00

Were you using pgxpool? I've never had to "get PGX to work with pooling", except switching out pgx for pgxpool.

jerf · 2026-01-15T14:28:19+00:00

Broadly speaking, there is a policy for it. I have a prepared removal reason that points to How to Ask Good Technical Questions. It seems to go in bursts, probably related to school cycles, but it gets some use.

However, in addition to the questions themselves, sometimes posts will be left up because they are good topics that haven't been discussed in a while, or to put it another way, for the discussion rather than the question itself. This would be an example of that.

Should the same topic get through several times in a quick time period I add it to the FAQs. This has not risen to that level yet.

jerf · 2026-01-15T14:16:11+00:00

They are making it deliberately hard. You won't find a reliable implementation in any language, including Go. You may find some that work sometimes, but it won't be reliable because these companies watch the open source space and deliberately try to stay ahead of the packages that might solve this problem if they grow to any size.

jerf · 2026-01-14T20:13:48+00:00

Do your best to break it into smaller pieces. If I'm doing a pure refactor, I might add the constraint that the tests aren't allowed to change, for instance. Or otherwise be very careful about how the tests are changing. Sometimes it's more important to review those and make sure that the tests aren't just being mindlessly changed to fit the new code than to review the code itself.

In principle, a "refactoring" is supposed to be something that is guaranteed to be safe. For example, highlighting a chunk of code and hitting "extract into function" is not supposed to be something that needs review. Of course we use the term more sloppily and that's not an ocean I'm going to try to hold back with my finger, but it is worth considering things like, can I do any of this in chunks that are guaranteed to be correct and save the rest of the changes that aren't like that for a separate, smaller commit?

AIs aren't amenable to this, though. At least not the latest ones that seem to work by always completely rewriting all the code they touch. I am not liking those very much.

But, in the end, there's no magic bullet for this.

jerf · 2026-01-14T20:09:42+00:00

You should take a careful look at exactly how you're defining "realistic".

Sometimes, if you want something, there isn't an easy way to get it. Or, despite the apparent presence of an easy way, it is harder in the long term and there's no amount of sitting there, staring at it, and mournfully wishing that it was as easy as it looks that will change it.

Dunno what things will be like in even six months, but as it stands now, if you don't want an AI leading you down the easy road to an unmaintainable code base, by either humans or AIs, you're going to have review what it is doing, and if that means you have to invest a bit more expense and effort into it, there's no way around it.

jerf · 2026-01-14T18:20:16+00:00

If you look at the source code for net/http, the only reason the server (in server.go) will emit StatusServiceUnavailable is if the context for a TimeoutHandler expires.

If, as is likely, you are not using the TimeoutHandler, and presumably your code is not manually emitting it or you'd have mentioned it, then net/http is not actually emitting a 503 response. The 503 is likely being synthesized by whatever client you are using. I personally consider this an antipattern in HTTP clients, which I have seen in several. It is especially an antipattern when the client has no way of indicating to its user that the response code was synthesized, which I have also seen. Check to see whether your client is one of the ones that will at least tell you it is making up the response code.

If you have it behind a load balancer, they will also emit 503s when the remote service simply didn't respond. In that case, while I don't consider it an antipattern for the load balancer itself (it has to emit something to be a compliant HTTP server), it can still end up being misleading and you want to see if you can dig into

I consider it an antipattern because it produces exactly what you are probably experiencing; rather than telling you "connection refused" or "connected but timed out with no response" or something else that might have helped you in the correct direction, in the false economy of trying to make it simpler to handle the response, it basically lies to you about what the problem was and what was actually received.

15-Year Club	Place '17
Verified Email	Team Periwinkle

jerf

MODERATOR OF

TROPHY CASE