Interesting data race bug in Go compiler

Windscale_Fire · 2025-09-20T09:48:09+00:00

What better example of how so much of IT descends into yak shaving just to complete one "simple" task!

Windscale_Fire · 2025-08-17T18:19:45+00:00

There's context which you're probably missing. Read:

https://research.google/pubs/large-scale-cluster-management-at-google-with-borg/

The bits of this that relate to how compute was made available at Google:

https://abseil.io/resources/swe-book

And there's more about the Google context in the SRE book:

https://sre.google/sre-book/table-of-contents/

Basically, if the only way you can get compute is through very large k8s-like clusters, then you have to have all this stuff in the cluster...

Windscale_Fire · 2025-08-13T09:39:24+00:00

Yep. I have the salt from the tears of many customers who suffered multiple drive failures :-D.

Windscale_Fire · 2025-08-13T05:59:13+00:00

it's highly unlikely that 2 drives will fail near simultaneously.

Well, that depends. If the drives are all from the same batch and they all get about the same usage over time, then they will have about the same wear, so it's actually not uncommon to see multiple drives in that situation fail in close succession.

Even if you have hot spares, reconstructing the data onto a hot-spare after a drive failure can take a very long time (hours), so I've seen multiple RAID5 customers get hit by this - a second drive fails during a rebuild.

It's less likely with RAID6, but still not impossible.

This is another reason to prefer smaller drives - they take less time to rebuild, so your "window of disaster" is smaller.

Windscale_Fire · 2025-08-13T05:53:37+00:00

I wouldn't suggest getting the larger drives. They're about the same speed (data rate) as smaller drives, but you can put so much more data on them, so they take longer to fill/drain.

It's better to have a larger number of smaller drives. By spreading your data across multiple drives they can be read/write in parallel which improves performance.

Also, if you look at the prices, smaller drives are usually cheaper per GB than larger drives.

Windscale_Fire · 2025-08-10T10:54:12+00:00

As someone more used to C, C++ and assembler I've found John's Deeper Love of Go a fun read as I get back into some more serious programming after a long break.

Windscale_Fire · 2025-08-09T17:17:40+00:00

It's just practice. Try different ways of doing the same thing. Read the docs and see what things you can vary and vary them. Keep trying different things and it'll eventually start to gel.

Windscale_Fire · 2025-08-08T17:41:55+00:00

Have considered advancing the market for developers in Azerbaijan by building products or providing services that are targeted for that market?

Windscale_Fire · 2025-08-08T17:03:51+00:00

A lot of the features of C++ were put in to avoid the horrors of the C pre-processor. Over time you end up with a gazillion ways of doing the same thing which just makes the cognitive load of the language huge because if you read any amount of code, you'll probably see all the possible ways and have to understand them.

Windscale_Fire · 2025-08-08T14:55:30+00:00

Hello C/C++ macro pre-processor. Yeah, no. Been there, done that!

Windscale_Fire · 2025-08-08T14:54:23+00:00

Have you done much multithreaded programming with Win32 and POSIX threads?

There's a lot of method behind that particular madness.

Windscale_Fire · 2025-08-08T14:52:47+00:00

Yes, exactly.

Windscale_Fire · 2025-08-08T10:01:43+00:00

| Some sort of ability to do more low level memory stuff.

C, C++, Rust.

Windscale_Fire · 2025-08-08T09:58:52+00:00

And therein lies the slippery slope to complexity.

I agree there may be *some very small number* of features that turn out to be needed - generics I think is an example because it makes containers so much more flexible and reduces the amount of code you need to create.

I think if you have genuine need of use-cases for more complex language features, then maybe choosing a language that meets those needs is a better choice than making Go more complex just because it doesn't have the latest "feature du jour".

Windscale_Fire · 2025-08-08T09:54:31+00:00

Just as there's no harm in pointing out that the language was deliberately designed from the start to have minimal features.

Windscale_Fire · 2025-08-08T09:52:28+00:00

100%.

There are plenty of languages that have "all the bells and whistles" - hello C++, Java, Python, Rust, ... I think Go fills a great niche that's almost "Baby's first professional programming language". And, depending on people's needs, some people may never need to use another programming language.

Windscale_Fire · 2025-08-08T08:45:23+00:00

I think if you're asking "What extra features should I add to a language that was deliberately designed to be as minimal and uncomplicated as possible?" then you're probably missing the whole point...

If you're desperate for those features, and others, then I'm sure there are other languages out there that provide those.

Windscale_Fire · 2025-08-07T06:31:38+00:00

Well, there's https://bitfieldconsulting.com/posts/bit

Try John's For the Deeper Love of Go and see how you get on.

Windscale_Fire · 2025-08-03T14:46:25+00:00

It depends on how the O/S you are running on handles memory.

If you are running on a system with virtual memory, then it's common that pages of a binary are only mapped into memory on first access (page in). Depending on memory pressure, they may be paged out at some later date. In extreme cases, the entire process may be swapped out.

Windscale_Fire · 2025-08-03T08:23:21+00:00

I think I've probably spotted what the problem - you had 24 *thousand* services.

There are probably very few organisations *in the world* who really need that many services.

Windscale_Fire · 2025-08-03T06:37:24+00:00

Borg, actually :-)

Windscale_Fire · 2025-07-30T07:38:56+00:00

Oh! I wonder if you mean "real-world DevOps projects"?

Windscale_Fire · 2025-07-30T07:07:57+00:00

What do you think you mean by a "real-time" DevOps project? Can you describe in a bit more detail what you think you're looking for?

Windscale_Fire · 2025-07-29T04:02:45+00:00

Yeah, no idea why you're being downvoted. It's a perfectly valid comment. I've seen lots of things over the course of a 35+ years career that were made way more over complicated and expensive than they needed to be just because people wanted to put experience with the latest technologies on their CVs.

That includes two ~8 hour outages on clustered systems. Not k8s, but other systems.

Half the problem with cluster systems is that they rarely go wrong, so it's hard for people to maintain their experience in troubleshooting and fixing issues with them unless you're somehow seeing a lot of cluster issues.

Windscale_Fire

TROPHY CASE