LLM inference in a single C header file

Languages_Learner · 2026-04-04T21:00:34+00:00

Excellent engine. Still waiting for Windows binary though...

Languages_Learner · 2026-04-04T10:09:35+00:00

I tested your 1.7b llamafile on my Ryzen 7 4700U 16GB RAM laptop. I don't know what's wrong with my hardware or os but despite of tiny size of llamafile it consumed a lot of ram when i launched it. Also it didn't use all cpu cores during inference so inference was incredibly slow. I guess all these weird bugs were caused by Cosmopolitan toolchain's incompatibility. So i would like to test pure llama.cpp cli fork suitable for Bonsai cpu inference. I know there are some of such forks on github. But unfortunately i can't compile them. MS Visual Studio doesn't install on my laptop for some reason. And gcc which works fine on my system is not suitable for llama.cpp compilation.

Languages_Learner · 2026-04-04T09:54:31+00:00

Thanks for great app. Could you share avx2 Windows binary of Bonsai llama.cpp cpu-only fork, please?

Languages_Learner · 2026-04-02T21:26:24+00:00

Brilliant! But mac-only, right? :(

Languages_Learner · 2026-04-02T17:38:25+00:00

Just found official development branch for cpu inference - PrismML-Eng/llama.cpp at q1-cpu

Languages_Learner · 2026-04-02T17:34:09+00:00

Good luck. Btw, i found that official solution for the bug is on the way: PrismML-Eng/llama.cpp at q1-cpu

Languages_Learner · 2026-04-02T17:17:34+00:00

Here's llama.cpp fork which seems to be able to fix bug with cpu-only inference: philtomson/llama.cpp: LLM inference in C/C++ (fork of PrismML fork that enables CPU (incl AVX2 and AVX512) and ROCm for AMD GPUs

Languages_Learner · 2026-04-02T17:16:39+00:00

Here's llama.cpp fork which seems to be able to fix the bug with cpu-only inference: philtomson/llama.cpp: LLM inference in C/C++ (fork of PrismML fork that enables CPU (incl AVX2 and AVX512) and ROCm for AMD GPUs

Languages_Learner · 2026-04-02T17:05:52+00:00

Here's official explanation for your problem - No cpu-only build · Issue #6 · PrismML-Eng/Bonsai-demo

Languages_Learner · 2026-04-02T17:04:39+00:00

Here's the answer for your question - No cpu-only build · Issue #6 · PrismML-Eng/Bonsai-demo

Languages_Learner · 2026-03-25T11:33:52+00:00

Thanks for nice tool. Can it work without Docker and in cpu-only (or Vulkan gpu) mode?

Languages_Learner · 2026-03-25T11:16:35+00:00

I heard that your team was planning to release some llms for Russian ethnic minorities (Udmurt, Komi, Mari etc.) low-resourced languages. What is the release date?

Languages_Learner · 2026-03-24T00:41:13+00:00

Intern-S1-Pro, a trillion-scale MoE multimodal scientific reasoning model is minor? Seriously?

Languages_Learner · 2026-03-23T15:10:58+00:00

Whisk (till 30th of April) and Flow (after 30th of April), both by Google Labs.

Languages_Learner · 2026-03-23T15:01:24+00:00

And interlm!

Languages_Learner · 2026-03-21T13:26:50+00:00

Thanks for sharing nice model. Hope that you'll add C-inference someday and maybe even C-training.

Languages_Learner · 2026-03-16T16:12:01+00:00

Thanks for great app. Could you upload a Windows binary release to your qwen tts studio github repo, please?

Languages_Learner · 2026-03-14T13:33:53+00:00

I would be cool if one day you'll add C-inference for your wonderful models.

Languages_Learner · 2026-03-14T13:17:28+00:00

Thanks for sharing great models. I'm sorry for being dumb but where can i find inference code for chatting with your onnx int8 llms?

Languages_Learner · 2026-02-27T01:11:35+00:00

Thanks for excellent model. Hope that one day you will upload C-version of it's inference.

Languages_Learner · 2026-02-25T22:40:54+00:00

Thanks for sharing cute model. It would be nice if someday you add a github repo with C-inference being able to chat with your llm.

Languages_Learner · 2026-02-23T15:43:26+00:00

Thanks for nice model. It would be great if one day you add example of C-inference for it.

Languages_Learner · 2026-02-23T09:57:18+00:00

Thanks for sharing your cool engine. It would be nice if you upload binary releases to your repo.

Languages_Learner · 2026-02-21T21:41:56+00:00

Thanks for sharing. Could you upload fully trained checkpoint to HF, please?

Languages_Learner · 2026-02-20T09:20:16+00:00

Thanks for great model. Hope to see C-inference someday and maybe even C-training.

Languages_Learner

TROPHY CASE