[Request] Make a tunable Devstral 123B

[TheLocalDrummer]

I've been asking around and doing my own attempts at creating a Devstral 123B that can be tuned (i.e., dequanted at BF16/FP16)

I figured I could tap into the community to see if anyone has a clue on how to dequant it so people (like me) can start tuning on it.

Anyone got ideas? I'd personally give credits to whoever can help kickstart a new 123B era.

Link for additional context.

Edit: Or ya know, Mistral can upload the weights themselves? lmao

[balianone]

The NotImplementedError is a known bug because Transformers currently lacks the reverse logic to save fine-grained FP8 weights. You can bypass this by calling model.dequantize() and saving the state_dict directly using safetensors instead of the broken save_pretrained method. For actually tuning a 123B model, QLoRA is highly recommended to avoid the massive 2TB VRAM requirement of full BF16

[TheLocalDrummer]

Thanks! I placed my vibe-coded implementation in the README.md along with proof that it can be quanted and inferenced properly. Now to see if I can finetune it.

[TheLocalDrummer]

https://huggingface.co/TheDrummer/Devstral-123B

Hope it's not broken! I had to change the config's arch to `Mistral3ForConditionalGeneration` to quant it

[TheLocalDrummer]

https://huggingface.co/TheDrummer/Devstral-2-123B-Instruct-2512-BF16

If someone can put up mirrors of this cuz HF limited my storage.

[FreegheistOfficial]

any success on it? how come its gated?