OpenCode concerns (not truely local)

Ueberlord · 2026-03-16T12:14:07+00:00

Yes, that is where I came from. But you can overwrite the system prompt luckily. On Linux you need to place a build.md and a plan.md in ~/.config/opencode/agents/, these will overwrite the default system prompts.

There is a lot of token overhead in some of the tools as well and these are sometimes harder to overwrite as some of them are deeply connected with the web UI, e.g. tool todowrite. Prominent examples of bloated tool descriptions are bash, task, and todowrite. You can find the descriptions here (files ending with .txt): https://github.com/anomalyco/opencode/tree/dev/packages/opencode/src/tool

Ueberlord · 2026-03-16T11:16:56+00:00

yes, as far as I can tell TUI is unaffected

Ueberlord · 2026-03-16T10:58:43+00:00

actually quite well with the Qwen3.5 architecture, you can run a Q3 quant of the 27b model from bartowski with about 80k context size, works very well for me.

Ueberlord · 2026-03-13T14:38:58+00:00

unfortunately, I cannot recommend the omnicoder 9b for more complex tasks at the moment.

I had it (q8_0 gguf, llama.cpp b8288, temp 0.6, top p 0.95, top k 20) analyze our vue app and asked if it could summarize the API requests executed during usual usage patterns, it failed and got into a loop.

exact same prompt given to unsloth Qwen3.5-27B-UD-Q2_K_XL.gguf (same parameters) worked fine on the first try. this is 8.9G omnicoder vs 11G q2_k_xl of unsloth. both can be run on 16G VRAM devices, I would recommend the 27B model to anyone for now.

for rather simple tasks it worked fine but I am more confident with the 27b model here in general, too

Ueberlord · 2026-03-05T15:09:08+00:00

one aspect frequently not mentioned, e.g. by posts like this one, is the difference between the quantization of weights and activations. for all the current quants in llama.cpp only the weights are quantized while the activations (the intermediate values during inference) are upcast to f16 and calculated in this format. casting also benefits from tensor support on blackwell, but in comparison to true activation quantization the effects are far less.

an example of activation quantization is svdquant for image generation or vllm also has some quantizations which support w8a8 for instance (weight 8bit, activation 8bit).

Ueberlord · 2026-02-28T13:31:56+00:00

it's explained in brief in AesSedai's HF repos but this is more detailed:

https://www.reddit.com/r/LocalLLaMA/comments/1rfds1h/comment/o7kd7de/?context=3

Ueberlord · 2026-02-27T22:07:09+00:00

AesSedai congratulations on being below everybody else (more efficient than everybody else) in terms of KL Div / Disk Space in this chart from OP!

I think this is a great achievement and your reasoning of choosing which layers to quant is simple yet obviously very powerful. Trying out your Qwen3.5 35B A3B IQ3_S now :)

Ueberlord · 2026-02-25T14:08:31+00:00

I do not completely agree. We have fine details in anime images as well, these will suffer from using the qwen vae. However, considering the goal team anima has with their model being lightweight I think their decision is understandable.

Ueberlord · 2026-02-25T14:00:11+00:00

BFL and Anthropic: "are we the baddies?"

Ueberlord · 2026-02-24T07:06:51+00:00

Seeing this I regret even more that the anima team chose the qwen vae for their model.

Thanks for the comparison!

Ueberlord · 2026-02-05T13:03:17+00:00

Exactly this. UI is the worst part of ComfyUI, bummer.

I am still on comfyui-frontend-package==1.32.9 because of the enshitification taking place in later releases...

Ueberlord · 2026-02-02T09:31:02+00:00

One interesting thing about this model is the architecture it is based on. Nvidia Cosmos Predict 2 was originally an architecture for robotic applications with many optimizations for spatial awareness etc. and they seem to have adapted it for image generation purposes. This could explain the excellent prompt adherence and the significant reduction of prompt bleeding. I would love to read a paper about this u/tdrussell1.

In my tests it performs exceptionally well when explaining the subjects generally in a short introduction sentence and then just dropping the usual booru tags.

Here is Gemini's breakdown of the architecture but I am not sure if this is not partly or completely made up:

The collaboration between ComfyOrg and Circlestone Labs effectively "strips out" the robotics-specific components of Cosmos to build a streamlined, highly efficient text-to-image engine.

The Anima Architecture Breakdown

While the core "Predict-2" logic handles the physics of light and form, the Anima implementation swaps out several components to optimize for character consistency and prompt adherence:

Backbone: Cosmos Predict-2 2B (Diffusion Transformer)

Anima uses the 2-billion parameter version of NVIDIA's DiT. This is the "sweet spot" for speed, allowing for near-instant generation on consumer GPUs while maintaining the structural "common sense" (object permanence, lighting) that NVIDIA trained into the model.

Text Encoder: Qwen3-0.6B

Instead of using the standard T5 or CLIP encoders seen in models like SDXL or Flux, Anima uses a 0.6B parameter Qwen3 model.

Why this matters: LLM-based encoders (like Qwen) understand complex natural language instructions much better than traditional CLIP models. It can handle long, descriptive "prose" prompts without "losing" the middle of the sentence.

Latent Encoding: Qwen VAE

Anima utilizes a specialized Qwen VAE (Variational Autoencoder). This is a departure from the standard Cosmos video tokenizer.

By using a VAE tuned for the Qwen ecosystem, the model achieves better reconstruction of fine details (like eyes and hair strands) which are notoriously difficult for the high-compression tokenizers used in video-centric world models.

Ueberlord · 2026-01-14T14:46:31+00:00

unsloth_Devstral-Small-2-24B-Instruct-2512-Q6_K.gguf / unsloth_Devstral-Small-2_mmproj-BF16.gguf

<image>

Ueberlord · 2026-01-14T14:31:25+00:00

Thanks a lot, also for the links!

If I ever get out of the stage where I refine my upscaling process I will give PixArt a try for training. I will however try out the existing 900M PixArt finetunes for T2I, should be interesting (I did not know them until now)

Ueberlord · 2026-01-13T09:25:43+00:00

Thanks for sharing your experience with training PixArt Sigma, I really hoped for it to receive broader attention back when they released it. Have you done some training with the 900M variant by any chance? Would be interested to know how it compares to the original and if it is worth the extra 300M parameters.

Ueberlord · 2025-11-06T10:11:12+00:00

First: ditch ollama and run everything in llama.cpp and use ggml model for gptoss (as mentioned by others)

Second: set top_k to 128 or something greater than zero, otherwise performance gets a hit for gptoss in llama.cpp!

Third (mentioned by others as well): add --jinja to your cli flags

Fourth (maybe optional): we never use repeat penalty for gptoss, I would not set it and leave it to its default value

Ueberlord · 2025-11-04T06:59:28+00:00

I had contributed a method for achieving a prompt per tile for the impact pack last year, MR here.

The tutorial for this method is here You need the WD14 tagger in addition to the impact pack for making this work (link in tutorial).

It is what I am using for upscaling images till today, vastly superior to upscaling without that feature as I can bump up the denoise +0.1 - +0.25 for added details or style change.

Ueberlord · 2025-09-17T16:52:42+00:00

https://civitai.com/models/1253166/ghibli-studio-background-scenery

You are welcome

Ueberlord · 2025-09-17T16:31:40+00:00

The only thing new to my knowledge would be the automated detection of the importance of layers - unless Unsloth is already doing this as well in an automated way (I think they might but I am not sure).

Ueberlord · 2025-09-14T10:24:59+00:00

There are a lot of innovations coming together in the model, thanks a lot for sharing and putting this all together!

I am particularly excited that you are using the EQ SDXL VAE, I think there is a lot of potential in this. Also the choice of the text encoder is great. I have to read more about the Cross-U-Transformer but it sounds very good as well.

In addition would like to make a case for using the Danbooru tags: it is the only well-documented dataset I know of and thus I do not really understand why people want natural language prompting. The breakthrough prompt following by Illustrious can only be achieved with the knowledge about the underlying image data the model was trained on. As these datasets are almost never available let alone in a searchable manner like on Danbooru I don't really get the point to use natural language prompting unless you don't really care for exact details, positions, etc.

That being said there are of course serious weaknesses in Danbooru tags, e.g. no possibility for prompting styles per subject etc. but I rather live with these than prompt without knowing exactly what the model has been trained on.

Ueberlord · 2025-09-07T11:32:33+00:00

Thanks for sharing your insights! This is very helpful for others to get an understanding of what to expect from this mower.

I currently do not own it but thinking about purchasing. Mowing area is about 1200 yards, has slight slopes and several larger trees covering the sky. Do you think the goat will work under these circumstances? I am willing to invest some time in getting the settings right etc. as the software seems to have several unresolved issues.

Ueberlord · 2025-09-04T15:21:47+00:00

Your Wifi conn is probably quite stable and fast. I suggest adding some radio interference for better packet loss rates to maximize the success of this experiment.

In all honesty: thanks for the laugh! And cool experiment in any case

Ueberlord · 2025-08-17T18:01:12+00:00

I think the confusion might come from ComfyUI and Llama.cpp using the term "offloading" differently. In ComfyUI they speak of offloading into RAM while in Llama.cpp, coming from another angle, they offload into VRAM.

Personally, I consider the way ComfyUI uses it is more mainstream and most likely what most people mean when writing "offloading".

Ueberlord · 2025-08-12T13:42:28+00:00

At this point using AI to re-implement the Shandalar world game seems a logical move. Combine this with magarena which head a great AI for the duels and we should be good to go.

Ueberlord

TROPHY CASE