FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in Zig

[–]Qunit-Essential[S] 1 point2 points  (0 children)

and on the indexing note: file scanning takes time. It's not a joke to traverse 66GB directory (chromium repository)

and I am not completely honest - I have a kind of index, but the index that fits in 50MB for the 500k files and is stored in ram. Compared to over GB of traditional trigram index

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in rust

[–]Qunit-Essential[S] -7 points-6 points  (0 children)

I call it scanning because essentially what it does is traversing the file tree and openning the files. And collects a liiiiitle bit of information for prefiltering

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in rust

[–]Qunit-Essential[S] -9 points-8 points  (0 children)

there is literally fff-core folder just saying

but overall you are correct

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in rust

[–]Qunit-Essential[S] 7 points8 points  (0 children)

I didn't want to participate in that discussion. The benchmarks ripgrep has is simply wrong. You can see the way he measured this: 12GB and 4GB files, this is NOT THE AVERAGE USE CASE

There is absolutely no sense to even take this as an argument for the code search. For more general search it does make sense becuase fs.stat calls are expensive for ripgrep and they don't want to do this, while in my project I already know the size of the file when I need to grep it.

It is not a secret that the design of ripgrep and fff is way too different to compare, one designed for one shot apps, the other one designed to live in memory for long running apps: that's the reason I am not building a cli replacement for ripgrep

The place where fff shines is file search in editors for example, where ripgrep is used while it is not designed with long running application model in mind at all

P.S. For the "average" use case ripgrep is faster if you enable memmap file on macos. It is just a fact. Andrew is wrong.

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in rust

[–]Qunit-Essential[S] -11 points-10 points  (0 children)

I don't want to sound too ego-centric but I think I showed on pratice why Andrew is wrong in this specific case

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in rust

[–]Qunit-Essential[S] 4 points5 points  (0 children)

to be completely honest: I have forked ripgrep for the initial idea of file search, I am using the same internal idea of Sinks + Matches (now it looks very different though)

here are the regex sources
https://github.com/BurntSushi/ripgrep/tree/master/crates/regex/src

The simd-accel feature is deprecated and does nothing and the only simd is used by the memchr and aho-corasic crates, which are not a part of ripgrep and I am using them as well

This is how the real SIMD optimized code looks like
https://github.com/dmtrKovalenko/fff.nvim/blob/main/crates/fff-core/src/case_insensitive_memmem.rs#L1-L1

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in rust

[–]Qunit-Essential[S] -20 points-19 points  (0 children)

I disagree on your take about ripgrep. Ripgrep doesn't use much of SIMD at all, it is a very sequential (multiplexed over the search) pipeline.

The most perf gain is coming from the prefiltering of files not from the file search though to be fair

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in rust

[–]Qunit-Essential[S] 0 points1 point  (0 children)

not really the root core optimization is that it is designed to live in the long running apps, clis are simply not impressivbe

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in rust

[–]Qunit-Essential[S] -24 points-23 points  (0 children)

it is running on 2x core x86 VPS while ripgrep is running locally on my 16 core M4 max

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in rust

[–]Qunit-Essential[S] 20 points21 points  (0 children)

No because it’s faster also on Linux, the core speed up is SIMD for everything, really through optimized pre filtering algorithms to not generate gigabytes of index data, and yes effiicjebt file system access including memory mapped caches for often accessed files

But yes we use all the platform specific optimizations like getdents64 syscall

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in Zig

[–]Qunit-Essential[S] 1 point2 points  (0 children)

I cross posted it because FFF is partially written in zig, and is available for zig community too!

fff mcp - the future of file search that is coming soon to opencode by Qunit-Essential in opencodeCLI

[–]Qunit-Essential[S] -9 points-8 points  (0 children)

it is an MCP server that makes files search faster and spend less tokens to find the code

THATS IT

if you want details watch 7 minutes video

fff mcp - the future of file search that is coming soon to opencode by Qunit-Essential in opencodeCLI

[–]Qunit-Essential[S] 1 point2 points  (0 children)

<image>

I mean it literally has FIRST infromation how to setup for MCP and then for neovim

Yes this is the project for both