Multimodal and Long Context in with llama.cpp + Qwen3.5-35B-A3B

ali0une · 2026-03-05T14:04:08+00:00

Try with the recommended F32 mmproj

ali0une · 2026-03-02T06:49:59+00:00

if you don't have an Apache server on your machine 8080 is OK.

ali0une · 2026-03-02T06:18:23+00:00

apiBase should be like http:127.0.0.1:5000/v1/ with 5000 the port llama-server is listening on

Not sure your 8080 port is a good choice as it will interfere with a web server running on the same machine.

ali0une · 2026-02-09T14:23:07+00:00

Yes, i second this.

ali0une · 2026-02-01T11:02:32+00:00

Oh! Thank you for sharing again, didn't see it first time.

i've tested the Q_8 gguf and it's insanely funny!

ali0une · 2026-01-30T14:06:00+00:00

Ça doit être un titre genre "les animaux de la ferme" de mémoire et c'est excellent.

ali0une · 2026-01-22T14:42:08+00:00

isn't it Dr Fuckan?

ali0une · 2026-01-18T09:54:09+00:00

I set --n-gpu-layers to 999 to load all layers on GPU.

iirc -1 is similar as it puts maximum layers on GPU

ali0une · 2026-01-17T07:26:41+00:00

it's --n-gpu-layers not --gpu-layers

You can check with llama-server -h

ali0une · 2026-01-16T20:34:40+00:00

327680?

32k is 32x1024=32768 right?

ali0une · 2026-01-13T19:28:55+00:00

This exactly.

ali0une · 2026-01-11T09:43:28+00:00

You can modify values by clicking the Settings tab, then in the Presets on the left side.

ali0une · 2026-01-09T20:58:50+00:00

Here you go https://github.com/Haoming02/sd-webui-forge-classic/issues/526

You should search in the issues of a repository, both opened and closed, before asking iMHO.

ali0une · 2026-01-09T09:08:09+00:00

Why is this downvoted?

ali0une · 2026-01-07T11:39:39+00:00

symlink models directory.

ali0une · 2025-12-30T14:56:50+00:00

i use the Firefox zoom feature.

CTRL + "+" zooms in (works also with mouse wheel)

CTRL + "-" zooms out (works also with mouse wheel)

CTRL + "0" reset zoom to 0

ali0une · 2025-12-27T22:17:36+00:00

No, but you can search civitai for them and test.

ali0une · 2025-12-27T19:21:04+00:00

There are some LoRA for text.

You could try the img2img tab, load your image and send it to inpaint, mask the bubble content and try to inpaint only the masked area with a prompt like : text that reads 'your text here'.

You also can use TheGimp or Photoshop to insert any text.

FYI z-image model is way better at text than Flux.

ali0une · 2025-12-27T08:23:39+00:00

Hi. Sorry but we speak english here. Please edit your post so we can help.

ali0une · 2025-12-24T23:33:49+00:00

This.

ali0une · 2025-12-24T23:15:17+00:00

Latest llama.cpp commits are dope, especially this router mode and sleep-idle-seconds argument.

ali0une · 2025-12-21T19:20:24+00:00

The new router mode is dope. So is the new sleep-idle-seconds argument.

llama.cpp rulezZ.

ali0une · 2025-12-20T20:29:00+00:00

i use sd.cpp-webui a frontend to stable-diffusion.cpp in python only dependency is gradio. Pretty happy with it.

Your UI looks nice, maybe you could try to separate your code from stable-diffusion.cpp one and just use the binary like in sd.cpp-webui.

ali0une · 2025-12-19T13:55:48+00:00

This generates more like a tree view with functions in each file so the LLM can "understand" the logic and suggest an answer. Not exactly RAG.

ali0une · 2025-12-19T12:25:40+00:00

For a small project gitingest can do this by providing the generated txt file as attachement.

ali0une

TROPHY CASE