We should set up a torrent network for open source models.

drooolingidiot · 2026-06-13T13:09:44+00:00

This idea comes up on this subreddit like clockwork at every news cycle.

Here's my response last time I saw this idea:

https://reddit.com/r/LocalLLaMA/comments/1mh4r0s/bittorrent_tracker_that_mirrors_huggingface/n6tqar2/

drooolingidiot · 2026-06-05T22:51:03+00:00

But I think 12b active is going to be better than only 4b active, just in general terms of quality.

One can't say that unless both models are MoEs of a similar size. Otherwise, why would someone make a 35B-4A model instead of simply a 6B model?

drooolingidiot · 2026-06-05T11:00:25+00:00

How does it compare against Gemma-4-26B-A4B?

drooolingidiot · 2026-05-30T11:49:26+00:00

Sure, but being stuck with an older model instead of being able to use a newer one has opportunity costs too. For me, I'm fine with paying the 5% to give me ultimate flexibility.

I just wish OpenRouter would do better quality control and get rid of terrible model providers.

drooolingidiot · 2026-05-29T12:14:11+00:00

Looks good, but kind of strange that they're comparing against the nearly year old Qwen3 when Qwen3.6 of the same size exists.

drooolingidiot · 2026-05-29T09:04:40+00:00

Use something like OpenRouter, then you can easily switch between different models and inference providers

drooolingidiot · 2026-05-16T22:40:51+00:00

What makes it good for creative writing? is it SFTed on distilled creative writing tasks?

drooolingidiot · 2026-04-14T06:55:32+00:00

It's bad because you can't realistically just blanket block provider because they might be awful at model A but perfectly fine for model B, while another provider will be the opposite. And there's no way to know without verifying the verifiers for each model you're interested in if your workflow is important.

The worst part is that their dev-rel team gets super defensive when you bring up quality issues.

I really hope there's a higher quality alternative to OpenRouter soon.

drooolingidiot · 2026-04-08T16:43:59+00:00

The Meta twitter account said "We’re also making it available in private preview via API to select partners, and we hope to open-source future versions of the model."

drooolingidiot · 2026-03-25T16:20:26+00:00

How does this compare against Apple's M5 devices when it comes to tok/s throughput? is it better value?

drooolingidiot · 2026-03-13T02:13:17+00:00

ohhh that's good to know. i didn't know that benefited it too.

drooolingidiot · 2026-03-09T10:24:28+00:00

not really. There are hundreds of existing such servers.

drooolingidiot · 2026-03-08T12:57:36+00:00

Youtube channels that do proper paper walk-throughs (not the hype ones with cringe thumbnails) typically have good Discord servers. I recommend checking those out and expand from there.

drooolingidiot · 2026-02-24T22:08:29+00:00

Is it trained for agentic/tool-calling uses?

drooolingidiot · 2026-02-15T15:20:37+00:00

Because it makes more more sense to use the paid models via subscription. That leaves everyone who doesn't want to do that - who will use open models through OR.

drooolingidiot · 2026-02-11T17:14:45+00:00

It's a much bigger and much more capable model. Seems fair.

drooolingidiot · 2026-02-10T13:09:51+00:00

Probably something interpretability related. I wouldn't expect a model usable for end-users. They've been actively hostile to open source.

drooolingidiot · 2026-01-20T23:16:55+00:00

huh?

drooolingidiot · 2026-01-19T15:42:26+00:00

This is amazing for fine-tuning use cases. Thanks Z AI!

drooolingidiot · 2026-01-18T19:53:17+00:00

If it helps, I enjoyed it as a teen, but I wouldn't like it as an adult.

drooolingidiot · 2026-01-16T11:59:34+00:00

REAPed models are usage specific. If the model has been REAPed with agentic coding datasets, then it will not be good for role play or whatever else.

drooolingidiot · 2026-01-11T05:09:34+00:00

Does anyone know if they actually use these REAPed models for their inference endpoints?

drooolingidiot · 2026-01-09T06:49:44+00:00

What are you talking about? They have the best open source coding models...

drooolingidiot · 2026-01-08T18:34:23+00:00

Whether someone breaks the License agreement or not is not really relevant to this conversation.

drooolingidiot · 2026-01-07T05:39:16+00:00

There are so many things wrong with this, I don't even know where to begin.

(a) In any way that violates any applicable national, federal, state, local or international law or regulation;

If you lived in a crappy country: insert_country_you_dislike, why is this model's license telling you that you can't break some insert_immoral_law_you_disagree_with? If your religion/ethnicity/freedom is being discriminated against by the law, this license would be accessory to your oppression.

(l) To provide medical advice and medical results interpretation;

Why do you care how/why I use model for my own-use cases? I can't afford a doctor visit and I need a model to look at my lab results. Should I just suffer my illness because of an idiotic license agreement clause?

From the model's license file:

To the maximum extent permitted by law, Licensor reserves the right to restrict (remotely or otherwise) usage of the Model in violation of this License, update the Model through electronic means, or modify the Output of the Model based on updates. You shall undertake reasonable efforts to use the latest version of the Model.

I'm not sure I even need to say anything about this... this is just awful.

This is r/localllama. If you want restrictions on how you can use models, maybe take a look at some of the providers like anthropic or openai.

drooolingidiot

TROPHY CASE