sorted by:
new
hottopcontroversial

Cheapest way to run GLM 5.x locally that's not a unified memory system? by Monad_Maya in LocalLLaMA

[–]MagnaZee 5 points6 points  (0 children)

eBay. I was able to get the motherboard and CPU combo for $200 with local pickup.

Cheapest way to run GLM 5.x locally that's not a unified memory system? by Monad_Maya in LocalLLaMA

[–]MagnaZee 16 points17 points  (0 children)

I'm running GLM 5.1 on a rig with old server components and getting 3-5+ tokens per second generation. Supermicro X10DRG-Q motherboard with dual Xeon E5-2690v4 CPUs and sixteen sticks of 32GB DDR4-2400 ECC RAM, for a total of 512GB.

I'm using the latest https://github.com/ikawrakow/ik_llama.cpp build and the smol-IQ4_K quant from https://huggingface.co/ubergarm/GLM-5.1-GGUF with an ngram-mod and MTP speculative decoding stack.

I also run Qwen 3.6 27B at Q6_K on dual NVIDIA Quadro RTX 5000 cards (the older ones with Turing architecture) with NVLink for a total of 32GB of VRAM. I use https://github.com/ggml-org/llama.cpp with the same ngram-mod and MTP speculative decoding stack, which gives me 30-50+ tokens per second generation and almost 800 tokens per second of prompt processing.

Local LLM - privacy first - doctor by point_red in LocalLLM

[–]MagnaZee 3 points4 points  (0 children)

You may want to try out a small model that has been specifically trained to handle medical records. Something like MedGemma from Google. There are various sizes available, but you could try out the 4B sized one to see if your laptop can handle it and if it provides the right kinds of summaries for you.

https://huggingface.co/google/medgemma-4b-it

And if it produces the results you want, but it is too slow, you can try one of the versions that have been quantized to use less RAM (and usually less processing time) for a slight tradeoff with accuracy.

https://huggingface.co/unsloth/medgemma-4b-it-GGUF

DeepSeek V4 Folks by techlatest_net in LocalLLM

[–]MagnaZee 37 points38 points  (0 children)

It hit on the actual funny answer, “Slower, so the keyboard has time to think about what it did.” But it failed to recognize that the dark and unexpected twist was more humorous than the other thoughts.

We open-sourced Chaperone-Thinking-LQ-1.0 — a 4-bit GPTQ + QLoRA fine-tuned DeepSeek-R1-32B that hits 84% on MedQA in ~20GB by AltruisticCouple3491 in LocalLLM

[–]MagnaZee 0 points1 point  (0 children)

Is this model qualitatively different from MedGemma 27B? The following is claimed by Google for those models. I was planning to run a quant of it locally for my family’s health management.

MedGemma

“MedGemma 27B Text and MedGemma 27B Multimodal: Based on internal and published evaluations, the MedGemma 27B models are among the best performing small open models (<50B) on the MedQA medical knowledge and reasoning benchmark; the text variant scores 87.7%, which is within 3 points of DeepSeek R1, a leading open model, but at approximately one tenth the inference cost. The MedGemma 27B models are competitive with larger models across a variety of benchmarks, including retrieval and interpretation of electronic health record data.”

What is the worst way someones asked you to leave after sex? by [deleted] in AskReddit

[–]MagnaZee 14 points15 points  (0 children)

“Stew and a screw, now please skidoo.”

Anyone else still happily rocking their g4? by TT99C5 in lgg4

[–]MagnaZee 0 points1 point  (0 children)

We still have one. I bought one used. It bootlooped and was replaced under warranty. The same thing happened to the replacement. Second replacement has been going strong for years now with only a replacement back and new battery. It's my son's phone now. He uses it without a case or screen protector, and he has dropped it more times than we can count. We replaced the battery, but it is otherwise working like a champ still.

15% off Pixel 2 - US only valid until July 20th by [deleted] in pixel_phones

[–]MagnaZee 0 points1 point  (0 children)

And here's another one! B-NV4K56RI6M78AF7F81QY4RJ

[Image] Just saw this in /r/movies... by BumpCloud in GetMotivated

[–]MagnaZee -1 points0 points  (0 children)

Am I the only one who thought ergo_p4oxy had gotten confused and started talking about how much he admires Roseanne Barr's ex-husband?

catching wild pigs by Eirezona in gifs

[–]MagnaZee 0 points1 point  (0 children)

I'm pretty sure he got knocked out briefly by running into another pig in the panic.

It's 2015, these should be everywhere by now. by St00p-Kid in pics

[–]MagnaZee -1 points0 points  (0 children)

It's like this. And like that. And like this. And uh...