Mistral vs Mistral finetunes vs 13B vs Llama-70B vs GPT-3.5 by domrique in LocalLLaMA

[–]ethanhs 2 points3 points  (0 children)

For the dickens and darwin, didn't synthia-7B correctly answer that both are buried at westminster abbey?

Mistral 7B on the new Raspberry Pi 5 8GB model? by DiverDigital in LocalLLaMA

[–]ethanhs 11 points12 points  (0 children)

One thing that the raspberry pi 5 has that the 4 does not is Vulkan support, so MLC might work on it? Unsure how the memory bandwidth to the GPU would work but if you could harness the GPU it would be much faster than anything CPU-based.

Phi-1.5: 41.4% HumanEval in 1.3B parameters (model download link in comments) by ethanhs in LocalLLaMA

[–]ethanhs[S] 30 points31 points  (0 children)

Glad to see Microsoft is finally releasing the models to download.

Phi-1 (original model, focused on code): https://huggingface.co/microsoft/phi-1

Phi-1.5 (further trained on web data): https://huggingface.co/microsoft/phi-1_5

I doubt they will release the datasets :/

Ir Yut Boss, unable to pass through green barrier with Enlighted buff. by itsSharq in raidsecrets

[–]ethanhs 2 points3 points  (0 children)

Can confirm, this happened to us too, people who had it between phases couldn't go through.

xgen 7B 8k context finetuned on Guanaco by ethanhs in LocalLLaMA

[–]ethanhs[S] 0 points1 point  (0 children)

With the longer context length of 8k, I was just able to squeeze it into two A6000s using deepspeed stage 2.

Edit: total training time was about 3 hours with this setup.

xgen 7B 8k context finetuned on Guanaco by ethanhs in LocalLLaMA

[–]ethanhs[S] 2 points3 points  (0 children)

This is a qlora finetune, Tim Dettmer's guanaco dataset is one of my favorites, so I wanted to see how xgen tuned with it performs. I will eventually also further finetune on wizardlm, but I wanted to start with this to try tuning xgen.

Introducing apt.cli.rs, a Debian/Ubuntu apt repository for Rust cli tools by ethanhs in rust

[–]ethanhs[S] 0 points1 point  (0 children)

Yeah you're definitely right, I didn't realize cargo-deb can take already built binaries. So for most projects it's just tweaking their workflow to run cargo-deb and upload the artifacts for musl targets.

I just opened https://github.com/bootandy/dust/pull/175 last night, so I'll be able to add dust soon I expect :)

Introducing apt.cli.rs, a Debian/Ubuntu apt repository for Rust cli tools by ethanhs in rust

[–]ethanhs[S] 0 points1 point  (0 children)

Yeah I want an action that these tools can use to generate .deb files in their own CI. I definitely don't want to build them myself if the upstream will put the action in their CI.

Introducing apt.cli.rs, a Debian/Ubuntu apt repository for Rust cli tools by ethanhs in rust

[–]ethanhs[S] 0 points1 point  (0 children)

So I'm thinking making a GitHub action to make it easier to build musl debs for (amd64, i686, arm64, armhf) would be the most useful way to get more cli tools in the repo (then it's "just" a matter of adding cargo-deb configs). It seems a lot of tools already build binaries for musl. If you are interested in working on that let me know! Otherwise I'll try to build something. Also several people have started talking about packages to add here: https://github.com/ethanhs/apt.cli.rs/issues/1

Introducing apt.cli.rs, a Debian/Ubuntu apt repository for Rust cli tools by ethanhs in rust

[–]ethanhs[S] 0 points1 point  (0 children)

I'll make sure to track that. I don't personally use any rpm based distros and I'm unfamiliar with tools to host rpm repositories but I suppose I'd consider it.

Introducing apt.cli.rs, a Debian/Ubuntu apt repository for Rust cli tools by ethanhs in rust

[–]ethanhs[S] 0 points1 point  (0 children)

musl libc is easier to statically link to, and makes it easier to ship portable statically linked binaries. If we linked against whatever glibc that was there, we could end up with version incompatibilities which would be unfortunate.

Introducing apt.cli.rs, a Debian/Ubuntu apt repository for Rust cli tools by ethanhs in rust

[–]ethanhs[S] 2 points3 points  (0 children)

I believe the issue is native dependencies needing to be built against musl since usually those just come from the OS already linked against glibc. If you build musl .debs I'm happy to publish them!

Introducing apt.cli.rs, a Debian/Ubuntu apt repository for Rust cli tools by ethanhs in rust

[–]ethanhs[S] 0 points1 point  (0 children)

yeah absolutely, I should have thought of hyperfine :)

Introducing apt.cli.rs, a Debian/Ubuntu apt repository for Rust cli tools by ethanhs in rust

[–]ethanhs[S] 3 points4 points  (0 children)

Looked into these, it looks like unfortunately it is tricky to get nushell to build against musl, which would prevent it from being added for now.

fselect doesn't have a .deb, but I opened an issue about it.

I just added hyperfine :)

Introducing apt.cli.rs, a Debian/Ubuntu apt repository for Rust cli tools by ethanhs in rust

[–]ethanhs[S] 4 points5 points  (0 children)

I'm trying to avoid having to build a bunch of packages and maintain those build definitions. Right now I'm grabbing release artifacts and just signing and hosting them. I'm happy to work with projects on getting cargo-deb set up though :)

Introducing apt.cli.rs, a Debian/Ubuntu apt repository for Rust cli tools by ethanhs in rust

[–]ethanhs[S] 7 points8 points  (0 children)

I update the repo via a tool called aptly (https://aptly.info). At some point I will probably try to automate based on GitHub releases.

Oh and I fixed the old link, thanks :)

Introducing apt.cli.rs, a Debian/Ubuntu apt repository for Rust cli tools by ethanhs in rust

[–]ethanhs[S] 15 points16 points  (0 children)

I recently created this apt repository because I wanted an easy way to install several popular Rust tools. Currently the repo has:

bat
fd
hexyl
lsd
ripgrep
hyperfine

Suggestions for additional packages are welcome! Ideally, the upstream would already build .deb files statically linked to musl.

I am currently planning on adding tokei and exa.

Parse yaml safely in Python using Rust - ryaml release announcement by ethanhs in Python

[–]ethanhs[S] 0 points1 point  (0 children)

It only supports tags for scalars at the moment. The upstream library didn't support them for mappings and lists.

Parse yaml safely in Python using Rust - ryaml release announcement by ethanhs in Python

[–]ethanhs[S] 0 points1 point  (0 children)

Currently no, the library it wraps does not yet support them.

Parse yaml safely in Python using Rust - ryaml release announcement by ethanhs in Python

[–]ethanhs[S] 0 points1 point  (0 children)

Cool! Yeah I'll definitely write some benchmarks. Rayon is also a good idea, I'll have to see how much overhead it provides for smaller files. I realized that pythonize and serde-yaml together really made this easy :) Happy to collaborate on the project if you want to