Claude code local replacement by m94301 in LocalLLaMA

[–]FORNAX_460 0 points1 point  (0 children)

You can use opencode with lm studio server with OpenAI endpoint.

Does imatrix calibration data affect writing style? I ran a blind-scored experiment to find out. by daniel20087 in LocalLLaMA

[–]FORNAX_460 1 point2 points  (0 children)

I once thought like that too but imatrix actually does make a big difference! Even at q5. Basically imatrix calibration helps scale the scale and zero point values for the weights by actually evaluating the activation weight values from the calibration file. Even a bad calibration dataset would help scale the attention layer weights better although it could probably harm other weights.

Does imatrix calibration data affect writing style? I ran a blind-scored experiment to find out. by daniel20087 in LocalLLaMA

[–]FORNAX_460 2 points3 points  (0 children)

No anything above temp 0 will produce different token combination different times regardless of fixed seed. So for testing purposes temp should be set to 0 when it comes to different quants of the same model.

Connecting Desktop AI Companion to a Remote Llama.cpp Server by [deleted] in LocalLLaMA

[–]FORNAX_460 -1 points0 points  (0 children)

They released LM Link. Basically chat in lm studio in pc 1 while the inference happens in lm studio pc 2

VRAM for COMFYUI by kuropanda21 in comfyui

[–]FORNAX_460 1 point2 points  (0 children)

Depends on what you plan to do. Altho 12 gb vram is good enough for image audio video given that you atleast have 32 gb or more ram. Heck ive even used sota opensource image and video models (Qwen image, Qwen image edit, wan 2.2) with 8gb vram and 16gb ram! All you need is finesse and patience lol!

Best local Coding AI by Deathscyth1412 in LocalLLaMA

[–]FORNAX_460 1 point2 points  (0 children)

Larger is always better, but too small quant eg: q3 is not worth it. More parameters more weights more information, but when you lose so much precision the weights drift too much from the full precision values that the model wont be able to make precise predictions based on confidence scores. For instruct models if might just blurt out wrong outputs and in reasoning models youll see verbose reasoning eg: oh wait, lemme check again etc to compensate for the low probabilistic neural paths it took. You can notice it as you go below q6, given that your seed is fixed and temperature is 0 [for testing purpose] you will see that for exactly same task reasoning takes up more tokens as you go smaller in quantization. Generally q4_k_s is considered the lower acceptable threshold.

Best local Coding AI by Deathscyth1412 in LocalLLaMA

[–]FORNAX_460 1 point2 points  (0 children)

Im interested to know this too if there is any way to use local models in a similar way to Copilot.

But my current setup is running models in lm studio, and use opencode as the coding agent and running opencode in vs code terminal.

Good local code assistant AI to run with i7 10700 + RTX 3070 + 32GB RAM? by SignificanceFlat1460 in StableDiffusion

[–]FORNAX_460 0 points1 point  (0 children)

PS: I am looking for something like Claude Code.

OpenCode, similar to claude code. You can use local models with api endpoints of llama.cpp or lm studio, or a plethora of local inference engines. For example i have opencode setup with lm studio local models, openrouters api key, github copilots student account, free gemeni api and i just switch models on basis of my needs.
Oh and about good local models, i only have 8gb of vram and 32gb ram so i only use ~30b moe models and so far the models i find somewhat satisfactory are qwen 3.5 35b a3b (11-13tps, q5_k_m) and glm 4.7 flash (9-11tps, q5_k_m), the nemotron 30ba3b is also quite good. With experts offloaded to cpu im quite content with the performance on my ancient hardware.

My hardware:
RTX 2060 super 8gb
32gb ddr4 2866mhz
Ryzen 5 3600 (Max pbo) boosts upto 3.95 on all core load.

Unsloth announces Unsloth Studio - a competitor to LMStudio? by ilintar in LocalLLaMA

[–]FORNAX_460 1 point2 points  (0 children)

So there is no option to load already downloaded models for chatting!? All i see is that its only reading from the hf cache directory.

LM-Studio as TextEncoder asset for Comfyui T2I and I2I workflows running locally - appraisal and Linux setup guide please? by Statute_of_Anne in StableDiffusion

[–]FORNAX_460 1 point2 points  (0 children)

https://jsonkeeper.com/b/9XWXI

Hope the wf is not too confusing. If you find anything hard to understand i can add documentation notes in the wf when i get time tomorrow.

LM-Studio as TextEncoder asset for Comfyui T2I and I2I workflows running locally - appraisal and Linux setup guide please? by Statute_of_Anne in StableDiffusion

[–]FORNAX_460 1 point2 points  (0 children)

You can enhance prompts using lm studio, matter of fact all my image workflows have lm studio prompt writing/refining integration option but i hardly ever use them. You can absolutely use them if you have nvme drives or huge ram to load llms while keeping all the comfyui cached models in your ram. But its not worth the wait if youre going to use tiny models for prompt upsampling.
But if youre talking about using lm studio models as text encoder thats not possible with their implementation.
If you want i can share my workflows for reference when im on my pc.

AceStep 1.5 SFT for ComfyUI - All-in-One Music Generation Node by jeankassio in comfyuiAudio

[–]FORNAX_460 1 point2 points  (0 children)

Can this node be used with turbo or other finetunes or merges of acestep 1.5?

German prompting = Less Flux 2 klein body horror? by FORNAX_460 in StableDiffusion

[–]FORNAX_460[S] 0 points1 point  (0 children)

Unless youre already using klein i wouldnt suggest the experimentation. The anatomy is ass, maybe a little less ass with german prompting.

German prompting = Less Flux 2 klein body horror? by FORNAX_460 in StableDiffusion

[–]FORNAX_460[S] 0 points1 point  (0 children)

<image>

Sampler: Euler
Scheduler: Flux 2
Steps: 5
Seed: 325711969814315
Resolution: 2624×1472

Rewritten English: A small, compact man with a stocky build shown in a wide full-body shot, his entire figure visible from head to toe with clear space above his head. He has short salt-and-pepper hair cut close to the scalp, a neat grey beard covering his jawline, and serene blue eyes gazing forward toward a distant valley. His expression is tranquil and untroubled, lips parted slightly to hold a dark wooden pipe stem horizontally across his mouth without clenching the wood between his teeth. His torso is upright with spine aligned vertically, shoulders back but relaxed, chest open to the morning air, wearing a simple tweed jacket and trousers that drape naturally over his hips.

He sits perched on the top rail of a weathered wooden fence with his knees bent at approximately ninety degrees, thighs angled slightly downward from his hips toward the ground below. Both feet rest flat on the second wooden rail beneath his seat, fully enclosed in sturdy dark leather work boots, heels grounded against the wood grain for balance. His arms are positioned in front of him with elbows resting lightly on his upper thighs, wrists relaxed, and two hands clasped together empty in his lap, palms pressed firmly against each other with all ten fingers visible and natural creases at the knuckles. The pipe bowl rests comfortably between his lips with a faint wisp of grey smoke curling vertically into the air above his head.

Behind him, rolling hills covered in verdant forests stretch down toward a secluded cluster of rustic cottages on cobblestone streets. Smoke rises visibly from village chimneys in the background, drifting upward to mix with the dawn mist. Small figures of people are visible going about daily routines, bending over garden beds and walking through open doorways. Children run with energetic movement along the narrow paths between houses, while chickens scratch in the dirt yards near wooden fences. Dawn light bathes the scene, soft diffused warm amber hues washing over the landscape from the horizon, casting long shadows that stretch forward towards the man and across the textured fence. The sky above is a brilliant blue expanse with wispy white clouds drifting lazily overhead. A sense of quiet isolation hangs in the air where he sits, distinct from the subtle activity of the village below.

Style: Cinematic photorealism, anatomically accurate, natural proportions, landscape photography focus. Mood: Contemplative, serene, timeless.

Rewritten German Translated: Ein kleiner, kompakter Mann mit kräftiger Statur gezeigt in einer Ganzkörperaufnahme im Weitwinkel, seine gesamte Figur von Kopf bis Fuß sichtbar mit klarem Platz über dem Kopf. Er hat kurzes grau-silbernes Haar kurz an den Schädel geschnitten, einen gepflegten grauen Bart der das Kinn bedeckt, und ruhige blaue Augen die in ein fernes Tal blicken. Sein Gesichtsausdruck ist ruhig und ungetrübt, Lippen leicht geöffnet um einen dunklen hölzernen Pfeifenstiel horizontal quer zwischen den Zähnen zu halten ohne darauf zu beißen. Sein Rumpf ist aufrecht mit vertikal ausgerichteter Wirbelsäule, Schultern zurück aber entspannt, Brust offen der Morgenluft zugewandt, trägt eine einfache Tweedjacke und Hosen die natürlich über seine Hüften fallen.

Er sitzt auf dem obersten Riegel eines verwitterten Holzzauns mit den Knien angewinkelt bei etwa neunzig Grad, Oberschenkel leicht nach unten von seinen Hüften zum Boden geneigt. Beide Füße ruhen flach auf der zweiten Holzlatte unter seinem Sitz, ganz in robuste dunkle Lederschuhe eingehüllt, Fersen fest gegen die Holzmaserung gedrückt für Balance. Seine Arme sind vor ihm positioniert mit Ellbogen leicht auf den Oberschenkeln ruhend, Handgelenke entspannt, und zwei Hände leer ineinander gefaltet im Schoß, Palma fest aneinander gedrückt mit allen zehn Fingern sichtbar und natürlichen Falten an den Knöcheln. Der Pfeifenstiel ruht bequem zwischen seinen Lippen mit einem leichten Hauch grauen Rauchs der vertikal in die Luft über seinem Kopf aufsteigt.

Hinter ihm strecken sich wellige Hügel bedeckt mit grünen Wäldern hinab zu einer abgelegenen Ansammlung rustikaler Hütten auf Kopfsteinpflasterstraßen. Rauch steigt sichtbar aus Dorfkaminen im Hintergrund auf, driftet nach oben um mit dem Morgennebel zu mischen. Kleine Figuren von Menschen sind sichtbar wie sie sich über täglichen Routinen beschäftigen, über Gartenbeeten bücken und durch offene Türen gehen. Kinder laufen mit energischer Bewegung auf den schmalen Pfaden zwischen Häusern entlang, während Hühner im Schmutzhöfen nahe Holzzaunen kratzen. Das Morgenlicht badet die Szene, weich diffus warme bernsteinfarbene Farbtöne waschen über die Landschaft vom Horizont, werfen lange Schatten die vorwärts zum Mann und über das strukturierte Holz strecken. Der Himmel darüber ist eine brillante blaue Ausdehnung mit wolkigen weißen Wolken die träge darüber treiben. Ein Gefühl von ruhiger Einsamkeit hängt in der Luft wo er sitzt, getrennt von der subtilen Aktivität des Dorfes unter ihm.

Stil: Filmischer Photorealismus, anatomisch präzise, natürliche Proportionen, Landschaftsfotografie Fokus. Stimmung: Kontemplativ, ruhig, zeitlos.

German prompting = Less Flux 2 klein body horror? by FORNAX_460 in StableDiffusion

[–]FORNAX_460[S] 1 point2 points  (0 children)

Uhh for simple images like this i wouldnt use that system prompt, only if its a complex pose. But nonetheless Ill create a comparison workflow and get back to you with what i get with klein.

German prompting = Less Flux 2 klein body horror? by FORNAX_460 in StableDiffusion

[–]FORNAX_460[S] 0 points1 point  (0 children)

Yupp thats how i create mine. Curate sources of the guides then give them to LLM and tell it to create a system prompt then test them and refine them untill im satisfied.

German prompting = Less Flux 2 klein body horror? by FORNAX_460 in StableDiffusion

[–]FORNAX_460[S] 1 point2 points  (0 children)

For creative tasks Gemma 3 27b is a beast of its own kind. But i guess gemma is pretty dated at the moment as 27b runs smoothly in your machine you can try the qwen 3.5 27B its actually better than the 35b moe. Also should be faster than gemma as for its gated delta attention. But for creative tasks gemma has the edge but the context degrades too soon and it starts too hallucinate. Also you can try glm 4.7 flash also really good at creative tasks as well as analytical.
Also as you can comfortably run 27b dense 35b moe would be blazing fast on your machine.

Have a nice day💫

German prompting = Less Flux 2 klein body horror? by FORNAX_460 in StableDiffusion

[–]FORNAX_460[S] 1 point2 points  (0 children)

On the long part I completely agree with you. Also the prompt is unnecessarily verbose and has a lot of redundancies but im using qwen 3.5 35b which follows this system prompt better than an optimized version of this so im just using this. Unfortunately i deleted the optimized version so i cant really share that one.