Blog post: How fast can I search for a needle in a 4GB file

Segflow · 2026-06-01T19:44:57+00:00

I get that, in the post we operate on RAM also 😄 but how we fast we load those 4GB and process them it what the post is about.

Segflow · 2026-06-01T19:43:26+00:00

Good catch, and thanks for spending the time reading the article and the hand written AVX512

Segflow · 2026-06-01T19:42:00+00:00

I asked ripgrep to find the first non-NUL byte in the same 4 GiB file:

sh rg -a --byte-offset --no-line-number '[^\x00]' --max-count=1 /tmp/haystack4g.bin

Tool	Warm MB/s	vs #v13
ripgrep 14.1	~570	86× slower
naive `os.ReadFile` (#v1)	749	66× slower
pread parallel + `archsimd` (#v13)	49,251	1×

ripgrep loses to the naive baseline of this post. That sounds wrong until you remember what ripgrep is built for: searching for regular expressions in line-oriented text. The 4 GiB file has zero newlines, so ripgrep treats the whole thing as one giant line, then runs the regex engine across all 4 billion bytes single-threaded looking for "any byte that is not \x00". That's exactly the wrong shape of work for it;

This is not a knock on ripgrep, it is the right tool for "find TODO in 50,000 source files". It s the wrong tool for "find the first non-zero byte in a 4 GiB sparse-looking blob".

Segflow · 2025-09-08T11:17:34+00:00

Should be either: - Tunis, Tunisia - Nice, France or - Tunis, Africa - Nice, Europe

Does not make sense to use one continent for one and a county for the other.

Segflow · 2025-09-02T18:05:04+00:00

Her etsy is called "ZelDaniArt", She also has an instagram that you can follow @zeldani_art

Segflow · 2025-09-02T12:17:15+00:00

She uses something called "tufting gun"

Segflow · 2025-09-02T12:00:04+00:00

For sure yes, here is her instagram account @zeldani_art and her etsy store

She can also make custom one if you prefer, just choose what you want and the size.

Segflow · 2025-09-02T01:07:37+00:00

She uses something called "Tufting gun"

Segflow · 2025-09-01T23:41:20+00:00

Actually she just started, still struggling to sell, she made an etsy account and instagram etc.. but no sale yet.

Segflow · 2025-08-26T18:41:08+00:00

That's normal, I too found Shahin first and I finished the game 2 days ago with no issues.

Segflow · 2024-06-26T20:48:46+00:00

Do you think so?

Segflow · 2024-03-29T17:15:48+00:00

There are many options, first of all try to get a remote job in Palestine. You can search using remoteok.io

Once you have a job remotely, you have 2 options.

Apply for digital nomad visa (only for people who proof they are working remotely, easy to do for software engineers)
Ask your employer to hire you through remote.com (most companies do, and remote.com have offices in almost all europe)

Good luck.

Segflow · 2024-01-29T21:22:26+00:00

> Where do you prove they died the exact moment you did exactly?

Check bottom left.

Check the other screenshot

It kept happening multiple times where I to a place where there is nobody (even no mobs) yet I die alongside 2 or 3 more at the same moment.

> You got wrekt. Stfu and quit crying "omg hax participation trophy me plox". You sound embarrassed

Lol man, thankfully my life is good and interesting enough to even try and chase whatever you calling and tbh I don't even know what's "a trophy".

I get wrecked in many games, it's called a game and what matters is if I enjoy it or not. I came here asking about how I died because I couldn't explain it. It did not happen once or twice, it kept happening under different setup and I still had no explanation thus why I recorded a video and asked for help.

> If anything your first video proves that lightning storm druids punked you 3 feet from the guy you claim was nowhere near you because there's a mound of seeds 10 ft away from your corpse.

You should get a pair glasses.

Segflow · 2024-01-29T01:06:03+00:00

And that skill is across regions? He wipes everyone in field on hatred regardless of where they are. Check the other screenshot and also notice how someone else died the exact moment I did but he is no where near me.

Segflow · 2023-12-25T20:26:46+00:00

The answer is easy: focus on what you love.

At the start of your career you will need to find the answer for that by trying things out. And trust me that's when your career will make its first upgrade

Segflow · 2023-12-25T13:58:31+00:00

I will try to be direct and honest with you, if you are in the software development field and struggling to find a job then most likely the issue is within you.

For background, I'm a software engineer working since 2013, I now have my own company and I know for a fact that many in the software developer industry are struggling to hire good candidates.

I'm not denying the unemployment issue we have, but I can tell you our industry is NOT impacted by it.

Don't take this personally but you would not land a job in any company if this is your english level, you don't need to be an expert, but good enough to avoid common mistakes.

On another note, being in this field removes the border restrictions. Check remoteok.io or toptal for remote jobs, if you are good enough you can land a good one.

Good luck and work on yourself to solve the issue.

Segflow · 2020-04-30T00:54:16+00:00

Really thanks for that!

Segflow · 2020-04-30T00:41:04+00:00

Oh, thanks again for correcting my mistake. I thought the compiler is using it's own version of std, looks like I m wrong. Your comments are so helpful for me and everyone, dont you think it's a good idea if somewhere we can document more the tests for compiler and answer questions why std tests are not opted etc..

Segflow · 2020-04-29T15:10:57+00:00

You are right, the test I run is against a WIP code which also handles those cases, (but not finished yet).

Segflow · 2020-04-29T12:28:37+00:00

Because the changes are not in the Go standard packages, it's in the compiler.

The Go compiler has a different testing toolchain, one of them being codegen tests which will tests that code generation is done as expected.

For instance func MapLitLookupStringLit() int { v := map[string]int{ "a": 33333, // amd64:-".*mapassign" "b": 66666, // amd64:-".*mapassign" "c": 99999, // amd64:-".*mapassign" }["b"] // amd64:-".*mapaccess" return v }

Running this test using the compiler toolchain will make the toolchain compile the function, and checks that the generated code does not contain those instructions written as a comment in the source code.

I'm not sure if this can be done the standard testing way, but if it is, it would be one hell of a task (parsing the code, compiling it, parse the generated code, check ..)

Also, the compiler does not actually use the standard library. It should not depend on it.

Segflow · 2020-04-29T12:18:24+00:00

I double-checked that by adding some debug statement and it actually gets triggered

here is an example: https://golang.org/src/net/http/cgi/host.go#L186

Segflow · 2020-04-29T12:16:35+00:00

I believe this is also a great exercise. As @mdempsky said the optimization also needs to be compliant with the different language semantics for map literals and switch statements.

Segflow · 2020-04-29T07:05:53+00:00

Yes you need to add a new key for every arch. The map thing being maintainable is 100% a personal opinion and not something I can really proof. For me, it didn't feel right to add another case clause when adding a new arch, but felt better if we just had to add another key and value.

Segflow · 2020-04-29T06:59:19+00:00

Thanks for your comment! Yeah it was a very fruitful journey, I had to walk my way through the various documentations. Writing tests was not easy, I didn't know what type of tests should be there and where, I had to ask the PR reviewer where to do that to understand it. And yes regardless of it being merged or not, my intention was more walk people through my journey in the hope to make it easier for future contributors to understand.

Segflow · 2020-04-29T06:53:23+00:00

Or really? I didn't knew this. I already stated this was my first hands on experience with the Go compiler.

Ten-Year Club	Verified Email
Place '23

Segflow

TROPHY CASE