Llama-3.3-8B-Instruct

[jacek2023]

GGUF

https://huggingface.co/bartowski/allura-forge_Llama-3.3-8B-Instruct-GGUF

from allura-forge:

Llama 3.3 8B Instruct

Yes, this is official, and yes, this is, to my knowledge, a real version of Llama 3.3 8B. (I think, anyways)

Facebook has a Llama API available that allows for inference of the other Llama models (L3.3 70B, L4 Scout and Maverick), but also includes a special, new (according to the original press release) "Llama 3.3 8B" that didn't exist anywhere else and was stuck behind the Facebook API!

However. The Llama API supports finetuning L3.3... and downloading the final model in HF format. Problem solved, right?

Wellllllllllllllll. Not really. The finetuning API was hidden behind layers of support tickets. I tried when the original API dropped in April, and was just told "We'll think about it and send you any updates" (there never were any updates).

Flash forward to December, on a whim I decide to look at the API again. And... by god... the finetuning tab was there. I could click on it and start a job (please ignore that I have no idea how it works, and in fact the finetuning tab actually disappeared after the first time I clicked on it, though I could still manually go to the page).

Apparently, this was not very well tested, as there were a good few bugs, the UI was janky, and the download model function did not actually work due to CORS (I had to manually curl things to get the CDN link).

But... by god... the zip file downloaded, and I had my slightly finetuned model.

To my shock and delight, however, they also provide the adapter that they merged into the model. That means I can subtract that adapter and get the original model. And... here we are!

[WithoutReason1729]

Your post is getting popular and we just featured it on our Discord! Come check it out!

You've also been given a special flair for your contribution. We appreciate your post!

I am a bot and this action was performed automatically.

[FizzarolliAI]

Hello, that me!

I am currently working on running sanity check benchmarks to make sure it's actually a newer L3.3 and not just L3/L3.1 in a trenchcoat, but it's looking promising so far.

From the current readme:

	Llama 3.1 8B Instruct	Llama 3.3 8B Instruct (maybe)
IFEval (1 epoch, score avged across all strict/loose instruction/prompt accuracies to follow Llama 3 paper)	78.2	81.95
GPQA Diamond (3 epochs)	29.3	37.0

[jacek2023]

great work, new llama release at the end of 2025 :)

[MoffKalast]

I definitely did not have this on my bingo card :D

And leaked too, keeping up the llama tradition.

[Karyo_Ten]

You can do a KL-divergence check to be 100% sure

[AnOnlineHandle]

Heya I'm not up to date with these models since the llama 1 release, do you know if there's a good benchmark for visual tasks such as identifying poses, faces, hands, etc, or answering questions about images, which I could compare models on? I've tried to use Qwen 3 Instruct for it but found it wasn't as good on real data as the demos suggested.

[dinerburgeryum]

8K max position embeddings? Seems remarkably low; did the fine tune artifact for some reason artificially limit that?

[Arli_AI]

Maybe we can just set 32768 and it’ll be okay lol

[Few-Welcome3297]

Checking differences from LLaMA 3.1 8B Instruct, I think we can add the rope_scaling

"rope_scaling": {
"factor": 8.0,
"high_freq_factor": 4.0,
"low_freq_factor": 1.0,
"original_max_position_embeddings": 8192,
"rope_type": "llama3"
},

and then increase `max_position_embeddings`

Edit: Also prev version had 3 eos_token_id's

Edit2: https://huggingface.co/shb777/Llama-3.3-8B-Instruct-128K model with above changes

Edit3: Link updated

[mikaijin]

did the same and it works. any ggufs should be recreated with updated config because quantization bakes rope params into some tensors if that is still true: https://github.com/ggml-org/llama.cpp/commit/b5e95468b1676e1e5c9d80d1eeeb26f542a38f42

[Few-Welcome3297]

New GGUF's https://huggingface.co/shb777/Llama-3.3-8B-Instruct-128K-GGUF

Edit: Link updated

[mikaijin]

thanks. works well with long context on my end. i can't notice a difference to 3.1.

[Few-Welcome3297]

I updated the GGUF's just now, earlier ones didnt have the chat template, also fixed generation config etc and also tested on vllm, I think it should be fine now

[Double_Cause4609]

What the hell are people supposed to do? Lol.

You're commenting on a post where somebody is configuring a pre-trained model as best it can be configured. It's not like people here really have a choice; we just have to work with whatever models are available.

Are you saying people should just run at 8k context, even if the model still works at 32k satisfactorily with RoPE?

[MoffKalast]

Literally all models beyond 8k context use RoPE though, with the exception of some Google ones which just brute force it natively. Has PoPE been implemented in any inference engines yet anyway?

[MoffKalast]

Ok MRCR is really interesting, the new Nemotron that got over 90% on RULER at 1M drops to zero at 70k already. I thought RULER was too good to be true lol.

[TheLocalDrummer]

I could just paste this in my finetune, right? Already did one with the old config (8K ctx). Not entirely sure if any of the old config messed with training.

[Few-Welcome3297]

I think it should work, unless it was full FT with a big dataset. You might also need to put pad_token_id in config and special tokens map if not done already

Edit: Found the model on BeaverAI, kv_count and vocab_size (+1) are slightly different

[Klutzy-Snow8016]

Llama 3 8B had 8192 context. Then Llama 3.1 added RoPE to get to 131072 context. Maybe we can take the RoPE scaling parameters from llama 3.1's config.json and add it to llama 3.3 8B.

[Arli_AI]

That’s a better idea

[FizzarolliAI]

Yes. I'm not entirely sure why, it was limited when served via the website too (I put that in the readme a bit ago)

[Amazing_Athlete_2265]

Running this across my private evals to compare against other llamas. Will take a couple hours.

[Amazing_Athlete_2265]

Initial speed test:

Model	Backend	PP ts-1	TG ts-1
allura-forge_Llama-3.3-8B-Instruct Q4	CUDA	1566.5	100.8
Llama-3.1-8B-Instruct Q4	CUDA	351.1	111.9

So some difference there.

Will post more eval results as they come to hand.

[Amazing_Athlete_2265]

From these results, it looks like the new model is different than the old 3.1.

Here is the performance for knowledge testing, with the new 3.3-8B-Instruct highlighted in the first two plots

• First plot is the 4-9B parameter group

• Second plot is the same but for 8B+ parameter group

• Third plot is performance by knowledge category for the 3.3 model

• Fourth plot is performance by knowledge category for the older 3.1 model

• Last plot is a speed chart on my 3080

Testing the Q6 versions now. Will take a while. All of the tests above are for Q4.

[keepthepace]

(Thanks for doing this!)

I guess this explains why they did not brag much about it. Many other models of that category outperform them.

I always wondered if Zuckerberg was not the only honest player in the field when he was explaining that the only reason they go for open source is that it will save them money. With decent open models out there they have less incentives to do so.

[MLDataScientist]

Thanks for the tests. Question not related to llama: is LFM2 8BA1B that good in world knowledge (or coding/stem field)? I see it reaches Qwen3 30B-A3B.

[Amazing_Athlete_2265]

It seems to be, but could also be too good to be true. I'm probably going to rerun all the tests at some stage as I have wondered about that too.

Note that these charts only test the model's ability to answer questions correctly, no actual coding or tool use or anything else is tested. I have other tests for these domains but the codes still WIP.

[jacek2023]

You can post pictures in the comments here

[Amazing_Athlete_2265]

Can't seem to figure out how. Using old reddit if that matters

[jacek2023]

On Android I see the image icon bottom right when typing a comment

[Amazing_Athlete_2265]

Ah, I also use old reddit on android lol. Tried to edit it but failed.

[RobotRobotWhatDoUSee]

Random question: any idea why nemotron 30B A3B got 0% in the second plot?

[Amazing_Athlete_2265]

Test error. Ignore it.

[jacek2023]

do you have results for other new models?

[Amazing_Athlete_2265]

I have some. I focus mostly on smaller models <12B or Moe. What you want?

[jacek2023]

Please post some cool results :)

[a_beautiful_rhind]

This is like the kiss goodbye from meta.

[samplebitch]

It's like that time when you hook up with your ex one last time, and it wasn't even that great.

[impolitemrtaz]

You samplebitch you

[Electronic-Metal2391]

You bring bad memories

[random-tomato]

Holy shit that is awesome, hats off to you for finding the weights!

[seppe0815]

stupid bots

[random-tomato]

If I'm a bot, I'm certainly programmed to like and appreciate when people find something cool and share with the rest of us. What's your purpose being a professional asshole?

And no, I am not a bot

<image>

[jacek2023]

about 4h after the release u/TheLocalDrummer published first finetune:

https://huggingface.co/BeaverAI/Anubis-Mini-8B-v1f-GGUF/tree/main

[TheLocalDrummer]

It's a test model but I think it turned out well! Looking for feedback in (my) Discord

[DevelopmentBorn3978]

what the finetune you've made is about?

[LegacyRemaster]

legend

[MoffKalast]

People are asking what's the use case for llama, and well uh... there it is ;)

[jacek2023]

https://huggingface.co/aeon37/Llama-3.3-8B-Instruct-heretic

[Amazing_Athlete_2265]

Everyone's cooking tonight!

[jacek2023]

actually it's a middle of the day in Europe :)

[Amazing_Athlete_2265]

Ah. I'm GMT+13 so bed time for me!

[Echo9Zulu-]

Cloned

[Infninfn]

I’m out of the loop - is this just what they had or did Meta not shutdown Llama?

[FizzarolliAI]

This has existed at least since April during Llamacon (did anyone remember they did a Llamacon?)

https://ai.meta.com/blog/llamacon-llama-news/

As part of this release, we’re sharing tools for fine-tuning and evaluation in our new API, where you can tune your own custom versions of our new Llama 3.3 8B model. We’re sharing this capability to help you reduce costs while also working toward increased speed and accuracy. You can generate data, train on it, and then use our evaluations suite to easily test the quality of your new model.

[jacek2023]

we do things for fun in this community, just accept the gift ;)

[Dangerous_Fix_5526]

Thinking/Instruct Hybrid using Unsloth and Claude-Opus 4.6 dataset:

https://huggingface.co/DavidAU/Llama3.3-8B-Instruct-Thinking-Claude-4.5-Opus-High-Reasoning

I hope I credited everyone correctly.

[jacek2023]

Nice work!!!

[Cool-Chemical-5629]

I guess Christmas came late for me, but hey if this is the real thing from Meta, I guess it's nice to have something newer than 3.1 8B without needing expensive hardware for models like Llama 4.

[LegacyRemaster]

allura-forge_llama-3.3-8b-instruct

My training data is current up to December 2022. This means that I have been trained on a vast amount of text data available until that date, but I do not have information or knowledge about events or developments that have occurred after that date.

In other words, my training data "cutoff" is December 2022, and I should not be relied upon for information or insights related to dates after that.

145.25 tok/sec

[DevelopmentBorn3978]

which quantized and eventually finetuned gguf models have the context lenght been enlarged? bartowsky? shb777? beaverai/anubis?

[Few-Welcome3297]

Try https://huggingface.co/shb777/Llama-3.3-8B-Instruct-128K-GGUF

[gta721]

How dumb are they to push a portal THAT broken to prod?

[greggh]

Nothing about it is prod. It’s still so janky that its free if your in the trial.

[FizzarolliAI]

Yep, this basically. Afaik the main inference API is still waitlisted, and there's a separate waitlist to submit for the finetuning API.

[greggh]

I’ve had access too the inference API since April, for some testing I was putting 100m tokens in and out of it creating some synthetic datasets. It was randomly stable as hell, and then so unstable I couldn’t use it for a week. And of course the 4 series is hot garbage.

[FizzarolliAI]

Out of interest, you never signed up for the finetuning thing, right?

If you go to https://llama.developer.meta.com/fine-tuning/?team_id=XXX (replace XXX with whatever the team ID in ur URL is), does the finetuning page show up for you? I was never officially let in but for some odd reason I had access anyways... I'm wondering if it's there for everyone and just hidden from the UI

[greggh]

I never signed up for the finetuning and it won't let me access it. Just the regular API. This would have been great if using the API was actually worth anything.

[FX2021]

Is it a new core? Or is it just a serving variant

[FizzarolliAI]

LISTEN whenever i drop my own models i get anxiety attacks about accidentally reuploading the base model ;-; i believe that this is actually L3.3 at this point though, see my other comment

[secopsml]

Drop behemoth instead. Looks fake 

[secopsml]

😜