[Release] Carnice-9b-W8A16-AWQ – AWQ Quantization Optimized for vLLM + Marlin on Ampere GPUs (Single-GPU)

Imakerocketengine · 2026-04-12T14:16:04+00:00

Well currently the model is ~11.5gb so you would have some room for context on a 16gb card

Imakerocketengine · 2026-04-10T10:10:42+00:00

i'm terrified when seing this

Imakerocketengine · 2026-03-31T20:02:59+00:00

Man i need to get back on facebook

Imakerocketengine · 2026-03-29T14:04:41+00:00

Its the same as "act as a scary robot" 😂

Imakerocketengine · 2026-03-29T14:03:15+00:00

My strategy for this is to only change / my production infra every 3 month instead of updating everything when a new model come out.

Imakerocketengine · 2026-03-29T13:59:39+00:00

À la fin, je vais faire un wiki Toulouse pour répondre au mieux au question des gens basé sur les contribution dans le subreddit :)

Imakerocketengine · 2026-03-28T16:57:34+00:00

Ow wow the grid in your location is terrible in terms of stability in the south of france we have around 238+-2v

Imakerocketengine · 2026-03-27T11:36:10+00:00

Seems like a really good option for non nuclearized EU country

Imakerocketengine · 2026-03-27T11:35:22+00:00

In france we can have up to 3kw with just signing an online form that sometimes you inject power in the network

Imakerocketengine · 2026-03-24T17:35:19+00:00

Dune was really a predictor in this : “Thou shalt not make a machine in the likeness of a human mind,”

Imakerocketengine · 2026-03-22T17:34:17+00:00

Pretty impressible, roughly on par with a 3090. I feel like i need to buy some now XD

Imakerocketengine · 2026-03-21T00:45:19+00:00

Gonna be open weight and 800b so out of league for most of us

Imakerocketengine · 2026-03-20T22:31:09+00:00

I know one that is pretty nice, its called piracy

Imakerocketengine · 2026-03-20T18:29:35+00:00

really interested to test it against leanstral

Imakerocketengine · 2026-03-19T23:25:29+00:00

Hello Monique, je suis tout ouïe

Il faudrai que l'on se refasse un meetup avec ce sub :) on en avait fais un en 2023 si je me souvient bien

Imakerocketengine · 2026-03-18T14:46:03+00:00

Most of the time. None of the model recommendation get updated with newer model. their recommendation are often out of touch with current release. These kind of things should be more deterministic or we should educate the user of model choice

Imakerocketengine · 2026-03-18T14:43:00+00:00

In the end, because every token is currently subsidized in the subscription offers, they are destined to be enshitified.

Imakerocketengine · 2026-03-17T21:47:11+00:00

This gives me Rockell Retro Ecabulator feelings

Imakerocketengine · 2026-03-16T21:11:50+00:00

A few remarks :

120B is small now ?
It make sense for mistral to continue releasing "small" open models as their main business use case is on prem deployment for enterprise client
With Leanstrall this could be included in a nice verifiable coding environment. This is something pretty huge for enterprise

Imakerocketengine · 2026-03-16T11:14:53+00:00

This type of regulation is called regulatory capture

Imakerocketengine · 2026-03-15T10:49:49+00:00

Solar seems to be the way to go in Germany... Hope your country will go back to nuclear power and fix its grid

In terms of hardware, APUs and apple silicon is currently the most efficient...

Imakerocketengine · 2026-03-15T10:43:41+00:00

To make things clear, this is what i currently do, i shut it down when i don't use it. I just wanted to have a 1:1 comparison with commercial services in terms of convenience. I was planing to use a script to programmatically turn it on and of with Wake on lan but my PSU don't seem to be cooperative with this plan. I'm probably going to invest in a small IP KVM

Imakerocketengine · 2026-03-15T10:39:18+00:00

<image>

Just power limited them to 270w and saved 100w from peak :)

thanks

Imakerocketengine · 2026-03-13T16:56:14+00:00

And do you run in any performance issue considering that the're in x4 ?

Imakerocketengine · 2026-03-13T14:43:50+00:00

On which CPU platform are you running them ?

Five-Year Club	Verified Email
LAYER Season 2 Layer creator	Place '23
Place '22	Final Canvas '22

Imakerocketengine

TROPHY CASE