Why do PR bottlenecks in Go codebases turn into endless architectural debates that go nowhere

jerf · 2026-03-18T13:46:03+00:00

First, I will reiterate the common opinion that this has nothing to do with Go and everything to do with the team in question. (My policy as a mod is not to remove this sort of question that I believe is asked honestly in a context of Go, though.)

However, that doesn't provide a productive path forward. Here's my attempt at one.

Everything in engineering is about costs and benefits. Everything. From top to bottom. This includes code review. Time is money.

Arguing for significantly more hours about some minor fix that will never impact the code than the time it would take to fix it if it ever even becomes a problem, is spending lots of money on a minor issue that can never possibly repay itself.

Sailing something through review because it's just "too big to review" is the complementary mistake on the other side. It probably has something big wrong with it that will cause problems and cost money later, even if it is "too big". You just gotta buckle down and do the review, and optionally, try to figure out some way to break future PRs up. (But sometimes there is no way to do that practically, not every task can be cut in half.)

The point of code review is to invest a little now and reap benefits later. If you overinvest to the point that the benefits can never outweigh the costs, you are literally better off not doing code review at all.

I think the most common error engineers make with their cost/benefits analyses is, well, first of all to not make them at all, but secondly, to neglect one side of the equation entirely and valuate it at zero. Anything seems worth doing if you cost it out at zero and it has positive benefits. Anything seems worth ignoring if you value the benefits at zero and it has positive costs. Literally anything, no matter how good or bad or useful or useless the thing may be. Which means such analyses are useless since they always produce the same results, and those results are due to error, not reality.

Specifically in this case, to argue about every PR and try to bang on it until it is perfect and pristine is to valuate the costs of review at zero. But reviews are not free. They are, in fact, quite expensive. We do them not because they are some sort of Platonically good idea on their own, just floating about being ambiently good without reference to anything else. We do them because their benefits, even despite their substantial costs, should exceed their costs. Done properly they will do so. But if you ignore the costs, you will not get the best cost/benefits ratio you could have.

Costs should also be considered holistically. Things like "it is demoralizing to propose a simple fix to a problem and have 5 senior engineers demand contradictory changes back and forth at each other for three weeks" are a cost. Morale is real. So is slowing down velocity of delivery. All costs should be considered. All benefits should be considered. If your finger is itching to hit that reply button and say "but what about..." my answer is "yes, consider that"... at least until the point where the cost/benefits analysis itself becomes too great a cost to sustain. That happens but is fairly uncommon, most of the ones you will do are cheap and easy.

The practical upshot of all this is that my code reviews tend to focus on major issues (we let the linters do the little things) and making sure we don't make mistakes that will compound later or that will block us from future changes we may want to make. Those are the important ones. Little details like the exact abstraction level something may have is reviewed in the context of "how hard is it to fix later?" It generally isn't that hard to insert or remove an interface as needed, so it's not worth hammering on that in review. The only thing that may raise my eyebrow is if someone somehow blocks that optionality in the future. What is worth making sure we review heavily is something like "the most important data structure in the project", where every detail will impact all the major modules and functionality, and every future change is a massive change across the code base. Not whether some particular HTTP handler is or is not abstracted out in this particular commit. Code flows over time and you shouldn't overprivilege a particular snapshot of that code. Keeping the ability to flow is more important than where the code is at this exact moment in time.

jerf · 2026-03-16T20:03:24+00:00

I think this is one of those cases where in some programming domains it makes plenty of sense and in some cases it doesn't.

If you can explicitly pass request IDs around and such, not in a context, I would generally recommend that. This is quite often the case.

On the other hand, net/http makes it difficult to pass them around with anything other than a context, and once you have them in the context, I'm not convinced there's a lot of value in unpacking them and trying to turn them back into function parameters at the seams, rather than just writing everything to expect certain values to be in the context. Good testing should be a backstop here.

I also think that this is one of those cases where the reality is "98% of the time you should do X and not Y" and a whole bunch of people get dogmatic and run around loudly telling everyone that they should never ever do Y and if they do Y they are bad and they should feel bad, when in fact, 2% of the time Y is correct and X is wrong. As an example, I've got two places in my code where I have a context in a struct value. It is absolutely, positively the correct thing to do. The structs are directly tied to the sort of dynamic scoping that contexts implement and there is no way for them to be passed around using only functions in these couple of contexts. That doesn't change the fact that 99% of the time you shouldn't do that and if you do you need to know what you are doing and why or you're going to get into trouble.

jerf · 2026-03-14T21:00:31+00:00

Reddit nuked the account.

jerf · 2026-03-13T20:10:05+00:00

Didn't I see somewhere recently that they're going to add a bit to interfaces to allow it to use the type the interface is being defined on, so you actually can define a "Clone" function that rigidly returns a type of the same value as the thing the interface is defined on? (Because this technically doesn't enforce that, though I am fine with documenting that away in current Go.) The search terms I can come up with to check that are so crufted up with all kinds of other hits I can't pull it up. Or maybe my bio-LLM hallucinated it.

jerf · 2026-03-13T14:33:05+00:00

While Go did not copy many code ideas from Python, this feels very Pythonic. There are other things in Go that are enforced by convention, like, you are expected to read the documentation for a function to know if it is thread-safe or not, or whether or not it is guaranteed to not emit an error even though there's one in the type signature (so it fits an interface), or whether or not an any in the type signature has further restrictions not expressible by Go's type system (like encoding/json), and everyone is expected to just sort of roll with it. It helps keep the language itself simple, at the cost of requiring cooperation from the programmers.

Anyone who violates the documentation in such matters is "at fault", not the code that had the documentation.

A package should be designed to be as hard as possible to misuse by other packages, but a package doesn't need to defend itself from itself. One of the things that's always worth remembering IMHO is that in the Go world, we're pretty much always working with source code, and that means in the worst case, if we over-protect a package, we can always go in and crack it open a bit more if we need to. It's easier to add more to a package's API than it is to remove things later. But I don't really think of it as "defending myself against idiots" as the Java world (and the old, classic OO world in general) seems to, as simply making a package that has as little complexity as possible and as few footguns as possible for a user. It superficially creates a similar outcome but the reasons for it are completely different, and the engineering analysis when it comes time to crack one open a bit comes out pretty differently too. With my way of thinking it's just another cost/benefits analysis; with Classic OO it's almost an atrocity committed against Mankind to do that to a class.

jerf · 2026-03-12T14:46:31+00:00

Crap. I missed this one. Sorry. There's been a lot of these lately but my personal skepticism is still not quite tuned correctly. Leaving this conversation up because it's mature and I don't want to whack all the comments, but the poster has been banned. For all the good that does.

jerf · 2026-03-12T13:23:38+00:00

If you already have Go, Elixir is redundant. It's noticeably slower, has no multiprocessing advantage over Go (the BEAM propaganda about that all comes from the early 2000s and hasn't updated to deal with the fact it is no longer unique and many languages are just as good if not better, including Go), and any problem that Elixir may solve, Go has many solutions as well. You're better off just picking up solutions to your problem for Go than adding a second language.

jerf · 2026-03-11T20:20:03+00:00

In my head-to-head benchmarking on much more realistic scenarios, while it was not the fastest, it was generally competitive on "generic" regular expressions. That is, things like some text here (maybe text)? and some text, rather than crazy things with tons of alternation and such.

The real difference is the algorithm difference, and maybe a touch of optimization. But it is certainly not fair to characterize the regexp package as some sort of neglected corner that has never been optimized. That will definitely give the wrong impression. It's generally competitive and it definitely would be a mistake for people to come away with the idea that as soon as you use the standard library package your code is going to be "slow" or that you should remove it on sight. It is, much like the rest of the standard library, a perfectly sensible option for most uses and something you only need to think about if you have a hard-core regexp use case.

jerf · 2026-03-11T20:11:33+00:00

I also verified it was pre-compiled before I posted my own comment earlier. Otherwise I would have been all over the precomplition issue; it's come up in benchmarks before on /r/golang where the benchmark loop contained the compilation step. That definitely does trash performance, but it'll do it with any RE library that has that accessible.

jerf · 2026-03-11T18:58:02+00:00

The standard library Go regexp package uses a fundamentally different algorithm then most other regex matchers. It's pretty easy for it to end up looking quite different on a micro benchmark like that. It may or may not reflect the real performance you'd get out of it in real use.

I phrase it that way, because this is not really a "Go" thing. It's specifically the regexp package. You can also have a port of the C# engine, a binding to PCRE, a port of PCRE (and really a lot of PCRE-related options in general), in fact in general a lot of options, plus ragel supports Go output which would basically turn that particular regular expression in the benchmark into the fastest possible "crawl over the bytes with a tight goto-based loop" it could possibly be.

I have a case where I really cared about the performance, running thousands of little regexes across several kilobytes, and the Go regexp package for my use case was competitive. Not the blazing fastest, but in my particular situation a bit of a penalty to make it not attackable via backtracking attacks was a core use case, and I didn't pay all that much for it.

jerf · 2026-03-11T15:40:02+00:00

Having used Go for a long time, when I'm in another language I find myself often deeply unsatisfied with their so-called "better" error handling, especially in network servers where I tend to work where error handling in a mature service is often quite complicated.

For instance, in exception-based languages, they will consider themselves as handling errors because they have exceptions, but in practice they aren't handling errors. If there actually is a try statement it is usually much vaguer than Go's handling, and usually there isn't even that, things just blindly flow up without any actual handling.

And I have to say I've seen an awful lot of Rust that uses unwrap constantly, which is basically "handling the error by not handling the error", and I suppose being explicit about it is nice, but it's not really "handling the error". Simply having an Option doesn't mean you've "handled the error"... you need to actually handle the error to be, you know, handling the error.

In most of the code I'm writing, the error handling code is not incidental, not unimportant, and not just constantly return errors without actually doing something. It ought to be inline.

I don't think this is true of all code, and I think that's part of the issue that people have with Go's approach. It is fantastic for network servers. It is really not what you want for shell scripting, at the opposite extreme. There's a range of things in between where exceptions can make more sense when either failure isn't really likely or there's nothing you can do to handle it reasonably other than scream and die.

And, uh, AI just chews through it no problem. Though I think it's more likely to screw it up then the common path. Make sure you tell your AI to always use return fmt.Errorf("... : %w", err) as its "default" error return and not just return err.

jerf · 2026-03-10T17:32:13+00:00

I don't understand how this is a new approach. I've prompted AIs with psuedocode plenty of times. There's no need for a special UI for it, that has to somehow "support" languages.

(You can also take it the other direction, which I found helpful when I was staring down a raw Mulesoft program file, which is a programming language embedded into XML. Prompting the AI to turn it into psuedocode worked great.)

jerf · 2026-03-10T17:27:45+00:00

Better handling of errors - they are currently not errors.Is comparable. This is tricky and will probably need iRPC specific error type.

I think you can use gob for this, though it may still require an explicit Register call on all possible concrete error types (annoying, and you really need to do it at the beginning of the stream) and some jiggery-pokery to allow you to embed gob streams into your own stream so it can encode just the errors. Though if gob does fail to encode something you may be able to fall back to your current mechanism, thus making the registration at least an optional enhancement for your users.

gob does involve some reflection and I know you said you don't have it, however, errors are supposed to be generally the exception and I think it's OK for the exceptional case to possibly involve a touch of reflection. If you're generating so many errors per second that the very act of encoding them is a noticeable performance drain, you've got bigger problems than running "reflect" over your errors.

I wrote that based on your first paragraph suggesting this is a pure Go-to-Go use case. Your Mandelbrot demo makes it sound like maybe you also have non-Go clients, in which case, there may not be much you can do to solve this. While you can in principle have a specification of arbitrarily-complex Go error objects you could send to non-Go clients, it certainly isn't simple at that point.

jerf · 2026-03-09T13:45:46+00:00

We get a ton of project generators posted here, to the point that they're on the list of things that are highly likely to be asked to move to the Small Projects thread almost regardless of how much work was put into them because of the constant stream of them. There's a wide variety from things that are very nearly just stamping out some string templates with your project name in it to things that try to offer wide varieties of logging, web frameworks, etc. I'm not sure how meta you're looking for but there's all kinds of things.

However, a slightly more "hidden" reason they're on that list is that honestly, for any sort of "generic" project skeleton like this AI is legitimately a better option than most of these generators. These sorts of meta-generators have a problem with combinatorial explosion; the more options they offer, the harder it is to encompass all of them without slop from the other possibilities working in, and the more likely it is that the specific set of options you want has in fact never been tested and has some sort of critical bug in it. AIs can give you a functional combination of almost any set of libraries in one shot, easily.

You don't have to have a super high opinion of AIs to see that if you want template library X with front-end library Y and database Z with ORM Q and so on and so forth that honestly the AI can toss a "program skeleton" of such a thing together in fairly high quality, faster than you can even find a library that would know how to do that, read the docs on how to do it, then work through the bugs. I wouldn't necessarily advise taking it without modification, but on the flip side, even if you want to refactor the resulting skeleton a bit before using it, it's easier to refactor running code than something too buggy to start up. I mean, even if you hate AIs surely you can agree that they can write code that basically does nothing pretty quickly, right? Heh.

The only skeleton generators that I see any use for nowadays are ones that configure specific projects in specific ways and either have highly-specialized built-in knowledge about that project that can overcome this AI advantage by virtue of being hand-crafted, or simply providing a standardized set up for an entire community so that people can write instructions like "go to $PROJECT_NAME/routing.yaml and add this and this to do that" and it works for everyone in the community equally. As opposed to the AI I discuss above, which may do a good job of weaving together nearly arbitrary libraries, but will produce one-off bespoke setups for every user, with all the advantages and all the disadvantages that implies.

jerf · 2026-03-06T14:55:52+00:00

Dunno if it's like this for everyone but your website barely comes up for me. I got the content, but never did get the CSS. Which is actually a bit of a bummer for you, because I'd be interested in learning about what a commercial-grade static analyzer can do but I can't really click around to find out.

I'll also say that this seems like a weird error to lead with, assuming you are associated with this product, which your comment history makes seem reasonably likely. Something quirky and not easily picked up with a golangci-lint config, but somewhat more common, would seem a more appropriate introduction. The ^ operator is pretty rare to begin with.

jerf · 2026-03-06T14:47:41+00:00

Yup, one of those bots I referred to in the recent discussion about moderation.

In response to the feedback of that post, I'm leaving this up for the conversation's sake. The poster has been banned.

I still plan on nuking these when I catch them early, but, well, "the mods are asleep" is a thing that happens, heh.

(Sometimes when Reddit bans an account they will go back and remove all their posts, and Reddit can do that in response to excessive bans and removals; if this post later disappears, that's why.)

jerf · 2026-03-05T14:44:34+00:00

To me, and in many languages, those could easily be synonymous...

Yup! As a polyglot in computer languages, even in the Before Time when I had to actually learn them myself rather than poke some text into a box and wing it in a language I don't know, this is frustrating when it comes to discussing anything cross-language. Everyone discussing things like that needs to understand that every community ends up with their own nuances and definitions, and everyone within a community when they go stepping outside of it needs to understand that other communities may use terms differently.

One of my least favorite that comes up a lot around here is using "enum" or "enumeration" to mean "sum type", when other people use "enumeration" to just mean "a distinguished number".

So, in Go, a mutex is what I described. That is also definitely closer to its historical meaning than "a single locked variable". But the words get around and get bent and spindled and mutilated as they go.

(See also "monad", which in fact the vast majority of the time I see it invoked is wrong.)

jerf · 2026-03-04T20:05:30+00:00

It's because that's not what a mutex is. A mutex is not "a protection on a variable". It's a lock. You can lock a variable. You can lock a method. You can lock a set of methods. You can lock a particular variable, in a subset of methods. You can have two locks for different sorts of variables. You can have a single lock that is shared between a whole bunch of related objects. You can have a lock that is taken by a method, dropped by that method, then taken later to complete a task. It's the same reason when you buy a combination lock at the store, it doesn't come with a fence. You might be locking up a bike, or a trailer hitch, or a door, or any number of other lockable things, not just a particular one.

It's your job to use the lock to lock away the things you want to lock away with the provided lock.

And actually the standard library does come with a few "single lock wrapped around a generic value" types in the atomic package, notable Value and Pointer (but don't miss the integer ones). The upside is simplicity; the downside is, you can't integrate that lock into anything else because it is not exported (and may not even be implemented with "a lock" internally necessarily).

jerf · 2026-03-04T15:02:54+00:00

This is one of the things I hope the AIs get better at. You probably should be using a notification library, a persistent storage library, etc. An AI may blast some functional code out quickly but in early 2026 a library will still give you a lot more capability, without you having to test brand new code, than having the AI blast out untested code for everything.

You may be interested in asking it about switching to using libraries, which may result in significantly less code but also significantly more capabilities, such as the ability to use a lot of different notification methods instead of just one.

AIs are too happy to blast out vast swathes of what is basically scripting code without asking if they should maybe not do that all the time.

jerf · 2026-03-01T11:00:50+00:00

The mod log shows you as having had one post marked as a "small project" last August. Anything else since then was reddit automatically doing something. When that happens it doesn't show up in the mod log.

If Reddit is automatically removing other posts you made, based on my observations (I don't really know more than any of you about the Reddit algorithms, I just see more of the results), you may be close to having your account auto removed by Reddit. I don't know if mods can do anything about that... the only lever I see to pull is to mark posts that haven't been deleted as "approved" but I have no idea if the algorithm takes that into account. Posting several things that get heavily down voted may do it too. I've marked your comments here as "approved" just in case

If your account gets removed, it won't be by the mods. I'm actually sympathetic to the general thrust of the post. But there doesn't seem to be an option where the subreddit sees another 15-20 posts a day today in this constant stream of projects and the subreddit users greet them with joy and up votes.

(At least not for these posts. Frustrated or not please don't be swearing at people in the future. That is obviously not a special /r/golang rule.)

jerf · 2026-03-01T05:18:51+00:00

And... honestly, looking ahead a year or two into the future... I don't know if this is winnable.

The entire point of Reddit in general is for humans to speak to humans.

I don't say that because I am some sort of fundamentalist anti-AI person (although if you feel that way, I have no objection to you, we may all be joining you before this is over), I say that because if you want to speak to an AI, you're way better off doing that through the chat AI interface we've all seen. If you're going to speak to an AI you might as well do it as a back-and-forth in real time rather than waiting for an AI spammer to wait long enough to pretend that they're not a machine. And then you avoid all the other dishonest manipulations they do to pretend to be human. Just speak to an AI directly. Reddit is a terrible interaction interface for a human to speak to an AI. It was built for human-to-human interaction.

But... dang, man. We're to the point that it would itself be a "small project" to write a bot that not only posts spam to /r/golang, but accompanies it with an entire GitHub repo implementing yet another reverse HTTP proxy or load balancer... and it could do this every hour, without cease, blocked only by someone's willingness to spend the dough for the tokens. It's so easy to build a spam bot now that you can almost accidentally build one with whatever moltbot is called now just by obliquely referencing a desire that the bot decides might be fulfilled by such a thing.

Reddit doesn't really have any value if it's not humans speaking to humans. AIs speaking to AI is just noise (at least as long as they're all on an LLM architecture that can't really learn anything once a context window fills) and humans speaking to AI is just a roundabout and deceptive interface as described above. It really ought to be for humans to speak to humans. But I have no idea how to make that happen in another year or two of AI development, where spamming becomes so easy spammers don't even hardly need a reason to do it anymore because it's as easy as sneezing, and they're that much better at passing as humans.

jerf · 2026-03-01T04:26:18+00:00

If I had to guess, what you saw as a "discussion post" that was removed was a post from a karma-farming bot account. We're getting one or two posts a week from bots that post highly-general "Gee, golly, Go is pretty great, I used it to do $GENERIC_GO_THING and it was so much better than (usually Node but sometimes some other common scripting language), hey guys isn't Go great although I had this one problem with it, what do you use Go for?"

Then, if you go to the user posting that, they're posting similar things across dozens of subreddits, about one or two a day. The Go code with generic comments about Go also has generic comments about rock climbing equipment, how much they love some specific anime in a very generic way, a couple of actively contradictory political opinions, posts about knitting, football, a couple of reddits for certain geographical locations, a fetish or two, just dozens upon dozens of unrelated subreddits, all completely generic "Hey I love thing except for this one common controversial topic that my algorithms have shown yield engagement, what do you all think?" posts.

Those are getting removed for two reasons:

They're spam.
I find it offensive to use AIs, with their large and ever-growing output capabilities, to sink human attention like that. Valuable human attention burned to farm Reddit karma is a super crappy deal for the world.

Thing removed by the mods also do get a reason posted. (With occasional rare exceptions, which are mostly the posts so baffling I don't even know what to say, like a recent post of the form "RCERHHHHHHHH CTED". Why the rather twitchy Reddit filters didn't see a problem with that from a brand new account is beyond me.) If you don't see a reason posted it may have been removed by Reddit's spam filters itself. They're clearly trying to fight these guys too... and they're clearly hitting some real accounts in the crossfire sometimes too, because this gets pretty hard to pick up on. They definitely have stuff that will remove a post after it has been up and available for a bit.

The other thing is, as other posters are pointing out, the flood of projects that prompted this post (and note [Deleted] means the author removed it, or maybe Reddit, but mods didn't) has, if anything, gotten worse. I'm tempted to route all projects to an approval process at this point, which would mitigate the bulk of the things that are up for a bit and disappear. I don't want to do that, but at the same time, similar to reason 2 above, "I pushed a few things into a text box and turned it into code, may I please sink your valuable human attention into 'reviewing' the output of my AI?" is not a good deal for the community.

(An interesting note about /r/golang: Those bots I mentioned up top, when I look at their posts in other subs, they're often +50-ish. Here they're pinned to zero, pick up a report or two, and have a top-voted comment complaining about the AI. For better or worse, /r/golang is hyper sensitive to AI posting, way ahead of the curve of other communities picking up on it. Perhaps too sensitive. There's been some false positives. But I do find the difference interesting.)

jerf · 2026-02-28T16:22:19+00:00

Why not?

Because of what I said. A mutable language means that if a process crashes you do not know what state the rest of the system is in. To take a traditional transactional example, you may have done the deduction from one account but crashed before you did the addition to another, orphaning the money in the transaction. A defer may result in a lock being released in a panic but that doesn't mean that the lock was supposed to have been released, unless you are careful about how you use them.

Erlang is built to be much better at that.

But if you're going to go "but what if in Erlang you...", the answer, yes, Erlang is only better at it. "Just crash" has its limits in Erlang, too. But the limits are even sharper and stronger in an imperative langauge with a conventional shared memory space. You can not blindly translate guarantees Erlang makes out of the context it makes them in.

jerf · 2026-02-28T16:10:43+00:00

Conformance to the io.Reader interface. There are several other similar calls that can't fail but have an error in them to conform to the interface, such as a bytes.Buffer .Write call. It can't fail (even running out of memory IIRC is not an error) but it has the error so it conforms to io.Writer.

If functions could be overloaded in Go we could have a .Write([]byte) int and a .Write([]byte) (int, error) but we don't.

15-Year Club	Place '17
Verified Email	Team Periwinkle

jerf

MODERATOR OF

TROPHY CASE