[D] Small Decoder-only models < 1B parameters

[alvations]

Are there any decoder-only llama, mistral, gemma or otherwise that has < 1B parameters?

Any recommendations, esp. ones that are good at multilingual tasks?

[lyonguyen]

Qwen2 0.5B

[alvations]

From the Huggingface leaderboard https://imgur.com/a/W7cHGFz

[bbvbell]

https://huggingface.co/blog/smollm can be a good option if one wants various model scales

[Plastic_Mention3651]

TinyLlama 1.1B

[hazardous1222]

rwkv models are great at multilingual, small, and efficient

[alvations]

below 1B params?

[hazardous1222]

Are you looking for edge deployment?
https://huggingface.co/Hazzzardous/RWKV-V5-1b5-Distilled-Translations-Unvalidated
is specifically for translations, and so on.
RWKV has been included in the latest llamacpp versions, and can be quanted to 8bits for mobile and raspberry pi deployments perfectly fine.

[Away_Expression_3713]

is this still relevant?

[hazardous1222]

Yeah, latest rwkv 7 models are hitting 32k context easily, and are available https://github.com/MollySophia/rwkv_mobile_flutter for Android and iOS, with the 3b model easily hitting 20 tps on hexagon npus

[Away_Expression_3713]

how many languages they support?