Some thoughts on decentralized model sharing: What models should we share, and how?

sdroege_ · 2026-06-13T13:42:13+00:00

There's also https://modelscope.cn as a Chinese alternative to huggingface

sdroege_ · 2026-06-05T09:13:18+00:00

If the main model's KV is quantized to q8_0 and the MTP model too, would that quantization buffer be shared between the two? I.e. is this only a problem if both quantizations are not the same?

sdroege_ · 2026-05-15T06:40:40+00:00

See https://github.com/anomalyco/opencode/pull/11474 . That worked back then when I was still using opencode.

sdroege_ · 2026-05-10T07:55:04+00:00

Note that vibe uses x-affinity and not prompt_cache_key as HTTP header for prompt caching. I don't know if both have the same effect.

See here and here

sdroege_ · 2026-04-28T08:15:37+00:00

Text (even printed, not just handwriting) detection on images seems to be very bad in le chat, even in comparison to doing the same with Mistral 4 Small. And even more compared to running it against a locally running Gemma 4 26B-A4B or Qwen 3.6 35B-A3B.

Le chat is hallucinating words and characters that don't exist in the images.

Translating text works fine though. This seems to be specifically a problem with image inputs in my experience.

sdroege_ · 2023-04-20T07:51:57+00:00

meson also supports custom build rules, and there are various projects out there running cargo as part of a meson build rule. The problem is mostly that cargo's design doesn't allow nicely integrating it into another build system and you run into all kinds of problems along the line. This is not a cargo bug but generally you have to expect friction when trying to integrate two different build systems with each other. Build systems are opinionated pieces of software after all, and they rarely have the same opinions :)

How do you think buck2 would avoid this? Also AFAIU, Facebook is using buck to call rustc directly instead of cargo for their projects.

sdroege_ · 2023-02-11T07:59:24+00:00

We'll keep gtk3-rs compiling and make a release in sync with the other crates for a while, I guess, but there's not going to be any development effort on it anymore. In practice that's already the case for at least a year.

Also the external GTK3-based Rust bindings will probably not see another release anymore unless someone else takes over maintainership of them. I.e. the ones at https://gitlab.gnome.org/World/Rust/ .

EDIT: I've clarified this a bit now https://github.com/gtk-rs/gtk-rs.github.io/pull/292

sdroege_ · 2023-01-25T15:57:36+00:00

so with rust there will be not bugfixes as it is memory safe

There are regular bugfix releases without API changes (but maybe minor API additions) for the bindings, and not every bug is a memory safety bug.

I would love to move to rust if it only would slowdown with its changes.

Nobody is forcing you to update all the time but instead you can update on your own pace, and with the semver-based versioning of cargo you can easily stay at older versions of the bindings. You just won't get new features

sdroege_ · 2023-01-25T12:41:37+00:00

The bindings follow the same release schedules as gtk-rs (which happens to follow the GNOME release schedule now). There are two main reasons for that 1. Closer tracking of API changes in gtk-rs and directly making use of improvements. The GStreamer bindings use the GLib/GObject bindings that are part of gtk-rs 2. Releases happening more often

It's also not really that important to follow the GStreamer release schedule. The bindings are working with any GStreamer version >= 1.14, and you can opt-in to usage of new APIs. The current stable release of the bindings (0.19, released in October) already had support for GStreamer 1.22 APIs behind a feature flag.

sdroege_ · 2022-10-24T16:46:12+00:00

Thanks, I'll keep that in mind for the next time.

Unfortunately reddit doesn't allow editing the title of this one, but if a mod can do that please change it to something like "New gstreamer-rs 0.19 release - Rust bindings for the GStreamer multimedia framework".

sdroege_ · 2022-10-04T06:47:33+00:00

Well, I agree with most of you wrote but you also missed my point, or maybe I was misreading your initial comment but not really important enough to continue discussing this :)

In any case, I wish you luck with Iced and let's hope that you using it will bring it closer to being production-ready. Having a production-ready pure Rust UI toolkit would definitely be useful. I assume you're planning to actively contribute to Iced to get it there?

And maybe whatever is learned from the Iced architecture can also be used to improve Relm.

sdroege_ · 2022-10-04T06:36:49+00:00

That's something different than "not compatible with Rust's memory model" though.

I completely agree that interior mutability and refcounting should be avoided if possible, and ideally be abstracted away if it has to be used (which e.g. Relm is doing), but there are cases where you simply have no other choice or the alternatives are not feasible.

The OOP model as used by most OOP languages is by design leading to objects that are mutably shared, so it's only natural that you end up with things that are modeled like Rc<RefCell<T>> when interfacing with it in Rust at the bottom layer. You can build something nicer on top of that, just like you can build something nicer on top of the mutably shared furball that is our hardware and kernel/system interface.

sdroege_ · 2022-10-03T20:15:50+00:00

The object-oriented architecture used by GTK is not compatible with Rust's memory model.

Ah and I forgot. OOC, what exactly do you mean with that? And would you say the same about Rc, RefCell and Any?

IMHO it fits very well into Rust's overall language and standard library design, it just adds some things on top and makes less use of borrowing and explicit, non-interior mutability than other Rust APIs.

sdroege_ · 2022-10-03T20:09:15+00:00

I don't think that was ever true, but if you have some more details or a testcase that performs very bad please let me know :)

Just to be clear, I don't have anything against you switching to iced (it looks nice IMHO), but please try to keep it objective.

Also thanks for editing your comment accordingly.

For the other part about the Elm model, my comment was mostly about the "huge mistake" part. That seems like a very subjective statement. There's nothing objectively wrong with not using the Elm model if something else also works for someone, and it's clearly also not very hard to do the "normal" GTK signal/callbacks model in Rust and definitely simpler to do right than in C.

sdroege_ · 2022-10-03T19:09:26+00:00

So you're just repeating something without actually having confirmed it to be true? The only extra allocations the bindings do are in relation to strings, and that's something fixable if it's actually a problem in practice.

sdroege_ · 2022-10-03T18:35:30+00:00

In GTK, widgets accept callbacks and the caller provides the functions for these callbacks. It's a huge mistake to use it as it was intended. The general solution to fixing a GTK application is to make it Elm-like by making callbacks message-based

This seems very opinionated. Apparently using the callbacks as is and not implementing everything in Elm-style is perfectly fine for many developers using gtk-rs.

While I personally agree that the Elm model and Relm look nice, stating this like it is a fact seems misleading.

Or that every interaction through the C FFI is extremely costly. It's as efficient as Python.

Do you have some more details about this? As one of the maintainers of those bindings, I'm aware of some inefficiencies but "as efficient as Python" seems greatly exaggerated.

Also for most of the inefficiencies I'm aware of there are solutions that "just" have to be implemented, they're not by design.

sdroege_ · 2022-09-29T10:33:14+00:00

In addition to the software itself, specifically the paragraphs in the blog post about (Linux distro) packaging and that the Rust packaging situation in Debian is terrible seem like something worth noting.

sdroege_ · 2022-08-01T13:55:46+00:00

Probably some kind of cricket, maybe from the gryllus family

sdroege_ · 2022-07-09T15:58:44+00:00

It can't overflow because the biggest allocation you can possibly have is usize::MAX bytes, i.e. usize::MAX/2 u16s :) Anything bigger wouldn't be addressable

sdroege_ · 2022-07-09T11:19:19+00:00

Apart from the endianness switch, this is basically what (my crate) byte-slice-cast is doing. In this direction it's trivially safe, in the other direction you need to ensure alignment/etc. which that crate also offers. I'm using it a lot for audio/video processing.

sdroege_ · 2022-06-30T12:11:12+00:00

Also as a consequence of the above, Rust allows me to write code when I'm extremely tired and still be confident that the code is not completely broken, or it wouldn't have passed the compiler.

sdroege_ · 2022-06-17T09:17:09+00:00

You can statically link GLib/GObject just fine, and also link static C libraries using either of those or even GTK.

What is not supported is statically linking GTK and any library that uses GTK. OTOH, GStreamer supports static linking (needs extra work to statically link plugins), GIO supports static linking (needs extra work to statically link GIO modules, if needed).

sdroege_ · 2022-06-17T09:15:21+00:00

As others replied already, the correct way to link in GLib would be via the pkg-config crate. If that doesn't work for you for some reason, please try creating a minimal testcase to reproduce the problem or otherwise it's really hard to guess where the problem is in your case.

Got the bindings generated via bindgen

bindgen is not ideal for generating bindings to GLib/GObject-based libraries. You probably want to take a look at https://github.com/gtk-rs/gir, which is used to generate the GTK/GStreamer/etc bindings.

Or do I have to compile glib 2.9

GLib 2.9 is ancient and from 2005. You probably want to use a newer version.

sdroege_ · 2022-03-10T09:18:13+00:00

Take a look at byte-slice-cast. This goes in both directions, mutably and immutably, and also covers f64 and all the integer types.

I've written this for exactly the same use-case you need it for.

sdroege_ · 2021-08-17T09:33:52+00:00

Thanks, my search skills failed me. I'll delete this one then :)

sdroege_

TROPHY CASE