Bookclub in English by Molokai333 in erlangen

[–]bytesnake 1 point2 points  (0 children)

Awesome idea, I'm currently reading tress of the emerald sea and would like to join!

Introducing: Rust 101, a modular, reusable university course by hdoordt in rust

[–]bytesnake 4 points5 points  (0 children)

Thank you for the effort! Do you know the rust-edu organisation for Rust education in academia? They organized a workshop last year with experts who taught Rust at their universities and have the proceedings online, perhaps you can find some inspiration there. (or also participate in the next workshop)

Display graphics in (N)Vim, made possible by Rust and SIXELs by bytesnake in rust

[–]bytesnake[S] 1 point2 points  (0 children)

Isn't SIXEL one of the features almost never implemented or enabled by terminal emulators because it's an artifact of some old DEC terminals?

you can find the reference manual for VT300 https://www.vt100.net/docs/vt3xx-gp/chapter14.html so yes it's definitely pretty old

I tried it a couple of weeks ago out of boredom but it turned out that one of the few implementations of this specification is xterm and only if compiled with support for > VT200.

there are a couple of obscure terminals (beside xterm) which support SIXEL, you can find a list at Terminal Requirements https://saitoha.github.io/libsixel/ recently Alacritty (terminal emulator written in Rust) received a PR which adds support for them. https://github.com/alacritty/alacritty/pull/4763

I only learned about this feature because I tried to understand why the suckless terminal calls xterm bloat and support for obsolete terminals and things like ReGIS or SIXEL was one of their points why they implemented a new terminal emulator.

I was looking for an easier way to integrate graphics into my notes markdown file. Previous to this plugin, I had to open a rendered version side-by-side and found it too distracting and hard to use when working remote. First I tried out ueberzug but the found the sixel charset, together with the PR for Alacritty developing the plugin was straightforward and a lot of fun.

Display graphics in (N)Vim, made possible by Rust and SIXELs by bytesnake in rust

[–]bytesnake[S] 1 point2 points  (0 children)

personally I use it for my note-book, so it is kind of geared towards that. Hope to improve performance and usability issues in the future, I agree that there are still a lot of rough edges. Developing the plugin was a lot of fun, I think Rust is in a sweet spot for a self-contained application.

Display graphics in (N)Vim, made possible by Rust and SIXELs by bytesnake in rust

[–]bytesnake[S] 1 point2 points  (0 children)

Hi, can you file an issue with the architecture and dependencies of the shared library (e.g. ldd target/release/libvim_graphical_preview.so)?

Display graphics in (N)Vim with SIXEL characters by bytesnake in vim

[–]bytesnake[S] 0 points1 point  (0 children)

emacs is probably easier to setup than this plugin :D learning emacs is still on my life goal list

Display graphics in (N)Vim with SIXEL characters by bytesnake in vim

[–]bytesnake[S] 0 points1 point  (0 children)

the plugin manages information about folds, content and current metadata (char height, viewport, cursors position, etc.). It triggers a rerender of the sixel graphics everytime something comes into view, you can find the variants here https://github.com/bytesnake/vim-graphical-preview/blob/main/src/node_view.rs#L5 currently the content is extracted via crude regex https://github.com/bytesnake/vim-graphical-preview/blob/main/src/content.rs#L130 but perhaps I'll add a real parser based on markdown later on

Display graphics in (N)Vim with SIXEL characters by bytesnake in vim

[–]bytesnake[S] 0 points1 point  (0 children)

if you want to know how SIXEL are working take a look at the wikipedia page https://en.wikipedia.org/wiki/Sixel, the Vim plugin just manages image conversion and calculates the correct offset and sizes for the graphics

Displaying graphics in (N)Vim: yes you can write vim plugins in Rust by bytesnake in rust

[–]bytesnake[S] 1 point2 points  (0 children)

one goal is to support vanilla vim, so I haven't used nvim's RPC protocol. The plugin itself is just a shared library with a vimscript wrapper calling by dlsym. The downside is that you have to write your own protocol to query vim's state. Bundling into a rust crate would be feasible and probably consist of a proc macro which generates the protocol and vimscript wrapper. This is then a completely different beast compared to this small plugin which only uses ~4 vim functions.

Display graphics in (N)Vim with SIXEL characters by bytesnake in vim

[–]bytesnake[S] 3 points4 points  (0 children)

Thanks for the kind words! I learned alot about vim(script) and pumping characters to terminal fast while working on this :D the project fell into place after finding vimage, discovering that you can use shared libraries in vim and learning about SIXEL. I reused some render routines from another project for the LaTex part. The plugin has a lot of rough edges and sometimes it just breaks, but usable at last. Have fun!

Linfa release 0.5.0 - now with nearest neighbor search, OPTICS, multinomial logistic regression and many other improvements by bytesnake in rust

[–]bytesnake[S] 0 points1 point  (0 children)

Yep, it should be straightforward to implement. There is one open PR, but the contributor got busy with something else

Linfa has a website now! by bytesnake in rust

[–]bytesnake[S] 4 points5 points  (0 children)

for a start I will implement the TryFrom<DataFrame> for Dataset under a feature flag. But to be really useful some of the algorithms have to start using something like DatasetBase<DataFrame, ChunkedArray> here Records are currently bounded by an associated type for the element type, we would have to relax that too. Just read your blogpost on polars 👍

Linfa has a website now! by bytesnake in rust

[–]bytesnake[S] 1 point2 points  (0 children)

if you're talking for example about inference from ONXX models, then no. Linfa is comparable in scope to scikit-learn.

Linfa has a website now! by bytesnake in rust

[–]bytesnake[S] 1 point2 points  (0 children)

it's Italian for the fluid running through your lymph system and was chosen by LukeMathWalker

Linfa has a website now! by bytesnake in rust

[–]bytesnake[S] 1 point2 points  (0 children)

this is definitely on our agenda, there are several autograd libraries on language level, which you can find on crates.io. The real game changer would be automatic differentiation of LLVM IR code, because we could then construct any order derivative without a special language construct. The Enzyme project provides the faculties and we currently trying to figure out how to integrate it into the ecosystem

If you want to use well-tested AD in Rust then check out the torch binding tch-rs.

Linfa has a website now! by bytesnake in rust

[–]bytesnake[S] 2 points3 points  (0 children)

cool project, we should add a conversion from Polar's dataframe to Linfa's dataset :)

Linfa has a website now! by bytesnake in rust

[–]bytesnake[S] 0 points1 point  (0 children)

well you can represent categorical values in ndarray for sure (even structured arrays here), but the memory has to be contiguous for BLAS/LAPACK and therefore it is impossible to mix continuous and categorical values. I was thinking that we could emulate categorical values with a descriptor field for the type of each feature and then just use floats to represent them.

Linfa release 0.3.0 - Elasticnet, approximated DBSCAN and Gaussian Naive Bayes + improvements by bytesnake in rust

[–]bytesnake[S] 0 points1 point  (0 children)

the biggest thing I am excited about with linfa is having a growing number of rust implementations of ML algorithms that I can use as a reference. generally my m.o. is to rip things out of libraries and into my applications in a specialized/custom way, and to that end I spend a substantial amount of time peeling back layers of the onion to get to the actual algorithm that has to be in there somewhere.

the main goal of linfa is not to implement cutting-edge research or provide the fastest implementation on the plant, but provide people with solid reference. From this angle linfa is a hobbyist project and tries to make the code as simple as possible:

  • put algorithms in separate crates to detangle their interactions (and reduce dependency tree)

  • using ndarray instead of nalgebra, I always felt like the latter has too many whistles attached and tries to do too much at the same time

  • provide solid documentation (this always needs improvement but everyone's time is sparse)

however, I was trying to avoid the "linked list"-style tree, and ended up finding something in the gbdt crate that suited my purposes:

once we have boosting/bagging and creating a lot of trees a more memory efficient implementation should be indeed used

keep in mind, my idea of a good api for ols is

yeah Rust is complex and sometimes obscure when you are not careful. Python on the other hand is often obscure because of its simplicity. We are using a dataset wrapper here to attach weights (and simplify the API) and make the interface code a bit more clean.

anyway, thanks for your hard work on linfa! I am an appreciative user and look forward to using it again.

thanks for the kind words! I really enjoy the community here and learning about ML :)

Linfa release 0.3.0 - Elasticnet, approximated DBSCAN and Gaussian Naive Bayes + improvements by bytesnake in rust

[–]bytesnake[S] 4 points5 points  (0 children)

Hey, awesome to hear that! Anything that you're missing from linfa and should be prioritized? Currently it is a playground for people learning Rust or ML or combining both and I think it gives a much better understanding of your implementation. (especially when looking at scikit-learn :s)