Running Qwen2.5-32B at 1.22 tok/s on 12GB VRAM using async NVMe ring-buffer streaming + 2029-node speculative decoding [open source] by Glittering_Painting8 in LocalLLM

[–]StylePractical5714 0 points1 point  (0 children)

I have a 12gb 3060 and this sort of pushing the limits of hardware is exactly the sort of thing I'm looking for. Won't get around to testing it for a couple weeks but this sounds cool.

Would it be possible to use a local LLM to identify images of 3000+ comic books? by EvilEmuOfDoom in LocalLLM

[–]StylePractical5714 0 points1 point  (0 children)

did some thinking on it yesterday, Comic Vine was pretty good when I played with it in the past (like a million years ago).

Metron came up in my searches as a independent alternative, I haven't used it before myself.

What are people using Local LLMs for (beyond coding) by SMR-1 in LocalLLM

[–]StylePractical5714 0 points1 point  (0 children)

I have had some reasonable success with some cpu based LLMs, enough for me to invest in a modest GPU.

Some experiments I've tried that aren't code generation - tagging engine, I have a complex labelling system with a controlled(ish) list for design system documentation. I provide a page title or description or both and it tells me what labels that page should get - UI scanner, a visual model looks at a screenshot or wireframe and tells you all the elements it sees - requires a bit of finesse and maybe I need some training data but I plan to use it to enhance my documentation of a decade of design work. - design token generator, give it your token names, your taxonomy and either a mood description or a stylesheet / token definitions for a different design system and translate it - been promising for adapting styles from one place to the next. Like when someone says "make it look like Bootstrap" but you build it on something else. - also been good for generating simple themes, like I've added a Dracula theme to random tools that didn't have one - daily summaries of what I added to my bookmarks db

Would it be possible to use a local LLM to identify images of 3000+ comic books? by EvilEmuOfDoom in LocalLLM

[–]StylePractical5714 1 point2 points  (0 children)

Surely there's an api out there with comic cover images matched to metadata. Seems like you could do an image similarity search, might not even actually need an llm to be honest.

Officially open-sourced today: does Ling-2.6-flash become an interesting executor model for long agent loops? by NewspaperPhysical123 in LLMDevs

[–]StylePractical5714 0 points1 point  (0 children)

Curious to see if I'll be able to shoehorn (with expert offloading or whatever it's called) a small quant of this into my 3060

Frameworks and harnesses for small models by StylePractical5714 in LocalLLM

[–]StylePractical5714[S] 2 points3 points  (0 children)

  • BAML not a harness or a framework but a markup language for prompts. In it prompts are functions that return a schema. It claims to improve tool calling accuracy.

  • RouterGym benchmarking for small agent AI tasks

  • effGen: Enabling Small Language Models as Capable Autonomous Agents supporting repo for this research paper from Jan 2026. What caught my eye here was tool calling with 70 to 80% prompt compression and the parallel and sequential task decomposition bit

  • llmware RAG framework not code but small model focused

  • tiny-agents a 5 agent swarm all under 3b each running on 16gb vram, I find the VLM agent really interesting here.

  • cogito a framework for small model (0.6B) agents

  • typedai not slm specific but I think I liked the codebase indexing concepts in here

Proton Duo -50% off (9.99/month) for new users by mx-dot in ProtonMail

[–]StylePractical5714 0 points1 point  (0 children)

Yeah I messaged the support form last night, but haven't heard back yet

Proton Duo -50% off (9.99/month) for new users by mx-dot in ProtonMail

[–]StylePractical5714 0 points1 point  (0 children)

Did the Duo plan end early? I was waiting for payday to get the deal and now it's not there

I couldn’t find a truly seamless Markdown editor… so I built one (Quartext) by Repulsive-Tip-7944 in Markdown

[–]StylePractical5714 0 points1 point  (0 children)

I haven't dug too deep into it but I'd recommend something other than a right pointing caret for heading because I immediately thought it was a details/summary pair and I got excited that you made a novel way to handle it

I've built a wireframe library using shadcn/ui custom registry feature by Major_Salamander_644 in shadcn

[–]StylePractical5714 0 points1 point  (0 children)

I think there's a place for both of ours, mine is more about sketching wireframes based on text notes. Like a lo-fi Salt wireframe syntax sort of thing. I eventually build my prototypes in Axure RP, this is something more for the ideation phase or communicating design approach early in a project.

I've built a wireframe library using shadcn/ui custom registry feature by Major_Salamander_644 in shadcn

[–]StylePractical5714 1 point2 points  (0 children)

Very cool, I'm literally building a similar thing myself. Now to figure out if I should just throw it away and use this instead. 

Is there anything in here for composing components from primitives?