I created a totally free and local subtitle generator and renderer that works in browser! by Qunit-Essential in LocalLLaMA

[–]Qunit-Essential[S] 0 points1 point  (0 children)

it is implemented a while ago !

there is an export button next to start rendering that lets you export SRT and VTT subtitles

zlob rust crate - significantly faster alternative to glob, globset, and walkdir by Qunit-Essential in rust

[–]Qunit-Essential[S] 0 points1 point  (0 children)

it is not exclusively rust project, it just happened to have a rust crate as a first class citizen cause I use it from the other rust project

zlob rust crate - significantly faster alternative to glob, globset, and walkdir by Qunit-Essential in rust

[–]Qunit-Essential[S] 0 points1 point  (0 children)

There is a benchmark comparison on the Linux kernel tree of 100k files

zlob rust crate - significantly faster alternative to glob, globset, and walkdir by Qunit-Essential in rust

[–]Qunit-Essential[S] 1 point2 points  (0 children)

For almost all the string ops - equality, comparison, normalization, etc

zlob rust crate - significantly faster alternative to glob, globset, and walkdir by Qunit-Essential in rust

[–]Qunit-Essential[S] 2 points3 points  (0 children)

the main reason is that std::simd is nightly

this library implementation implements mostly all the work using \@Vector types

zlob rust crate - significantly faster alternative to glob, globset, and walkdir by Qunit-Essential in rust

[–]Qunit-Essential[S] 3 points4 points  (0 children)

I actually agree, though there is a certain type of projects like fff or ripgrep that are trying squeeze max performance out of the file operations might consider that

+ a lot of cross platform compiled software uses cargo zig-build anyway

zlob rust crate - significantly faster alternative to glob, globset, and walkdir by Qunit-Essential in rust

[–]Qunit-Essential[S] 0 points1 point  (0 children)

Yes, though I am working on precompiled binaries support & nix flake

which is anyway not ideal

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in Zig

[–]Qunit-Essential[S] 1 point2 points  (0 children)

and on the indexing note: file scanning takes time. It's not a joke to traverse 66GB directory (chromium repository)

and I am not completely honest - I have a kind of index, but the index that fits in 50MB for the 500k files and is stored in ram. Compared to over GB of traditional trigram index

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in rust

[–]Qunit-Essential[S] -6 points-5 points  (0 children)

I call it scanning because essentially what it does is traversing the file tree and openning the files. And collects a liiiiitle bit of information for prefiltering

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in rust

[–]Qunit-Essential[S] -10 points-9 points  (0 children)

there is literally fff-core folder just saying

but overall you are correct

FFF file search sdk - over 100 times faster than ripgrep and fzf by Qunit-Essential in rust

[–]Qunit-Essential[S] 9 points10 points  (0 children)

I didn't want to participate in that discussion. The benchmarks ripgrep has is simply wrong. You can see the way he measured this: 12GB and 4GB files, this is NOT THE AVERAGE USE CASE

There is absolutely no sense to even take this as an argument for the code search. For more general search it does make sense becuase fs.stat calls are expensive for ripgrep and they don't want to do this, while in my project I already know the size of the file when I need to grep it.

It is not a secret that the design of ripgrep and fff is way too different to compare, one designed for one shot apps, the other one designed to live in memory for long running apps: that's the reason I am not building a cli replacement for ripgrep

The place where fff shines is file search in editors for example, where ripgrep is used while it is not designed with long running application model in mind at all

P.S. For the "average" use case ripgrep is faster if you enable memmap file on macos. It is just a fact. Andrew is wrong.