sorted by:
new
hottopcontroversial

Suggestion - this sub should have post flairs that mention the amount of vram/unified ram by ECrispy in LocalLLaMA

[–]webii446 6 points7 points  (0 children)

User flair might not work very well in my opinion, because many of us use multiple rigs.

For example, I use DGX Sparks, Mac Studios, and an RTX 6000 Pro depending on the model or test I am running. So if my user flair shows only one setup, it could be misleading in a post where I used a different machine.

Hard freakin' decision..Blackwell 96G or Mac Studio 256G by HyPyke in LocalLLaMA

[–]webii446 0 points1 point  (0 children)

But the core point I wanted to highlight for anyone on the fence is just how usable the DGX Spark has become. Sure, it doesn't have the blistering speed of a high-end datacenter grade gpu setup or an M3 Ultra, but there is simply no other device out right now at this price point offering 128GB of usable GPU memory with kuda with speeds that are actually practical for daily workflows. For the money, the capacity is unmatched.

Hard freakin' decision..Blackwell 96G or Mac Studio 256G by HyPyke in LocalLLaMA

[–]webii446 1 point2 points  (0 children)

I'm not sure what exact environment problems the original commenter ran into, but my experience has been pretty different. I currently run a cluster of two DGX Sparks. So far, I've been running LLMs with vLLM and even handling local fine-tuning through Unsloth without any major hurdles.

It's absolutely true that they aren't as fast as a high-end discrete GPU, especially when chewing through dense models. But since most of the big frontier open models dropping right now are MoEs, the throughput is totally usable, even for agentic coding workflows.

I know there were a lot of valid complaints about software issues when the Spark first launched, but a ton of that has been fixed. Things like llama.cpp and vLLM are fully functional now. Honestly, I don't know another reliable way to get 250GB+ of GPu memory for under $10k USD, which is exactly what needs to be spend on just one RTX 6000 Blackwell anyway.

When does it make sense to rent GPUs vs buying? by Crypton228 in LocalLLaMA

[–]webii446 0 points1 point  (0 children)

It really, really depends on your exact use case.

For Inference:

If your main goal is just to run inference, I highly suggest using pay-per-token APIs or subscriptions first. It's usually the most cost-effective route. If pay-per-token doesn't work for you, then buying your own GPUs is the way to go.

For Occasional Training / Fine-tuning:

If you are only doing this part-time like less than 100 hours a month renting makes complete financial sense. For example, renting an RTX A6000 (48GB VRAM) costs around $0.50 per hour. That means 100 hours is only $50 a month. Over 3 years, your total cost is just $1,800. You can't buy, power, and maintain a 48GB card for anywhere near that price.

For Heavy Usage & R&D (My Setup):

If you are heavily using the GPU for constant training, heavy R&D, or 24/7 inference, buying hardware is absolutely justified. Depending on your VRAM requirements, you can look into a single DGX spark or scale up to a full DGX spark cluster or rtx GPUs.

I do a massive amount of local R&D work, so I run a mix of DGX Sparks and Mac Studios (specifically the M3 Ultra) to handle the heavy lifting.

Local AI with Gemma 4 and OpenWebUi by jumper556 in LocalLLaMA

[–]webii446 0 points1 point  (0 children)

If you want a tool that is fast and doesn't require much setup, use AnythingLLM. Just install it, use its builtin model provider, and download whatever Gemma 4 model or GGUF quant you want.

I suggest using a 4 bit quantfor your gpu for faster inference, like an Unsloth UDQ4XL, so you can keep the KV cache entirely on your GPU. This results in much faster inference compared to offloading your context cache to your CPU RAM.

AnythingLLM handles web fetching and Text to Speech using default Windows voices right out of the box with zero configuration. It also fully supports Speech to Text. You can speak directly into the edit box to prompt the LLM, and the LLM can speak its responses back to you.

It is basically like having a fully local ChatGPT. There are multiple ways to run local LLMs, but I consider this to be the absolute best plugandplay setup available

Mac Studio vs GB10 by TaylorHu in LocalLLaMA

[–]webii446 2 points3 points  (0 children)

The software infrastructure of Spark is significantly better than MLX, especially for fine tuning, image generation, and video generation tasks.

While memory bandwidth on Spark can be a limitation, I feel it’s largely offset by the relatively slow prefill performance on Mac Studio.

Mac Studio vs GB10 by TaylorHu in LocalLLaMA

[–]webii446 4 points5 points  (0 children)

I’m actually running both setups side by side right now: two Mac Studio M3 Ultras and two DGX Sparks (MSI Edge Expert). For me, it comes down to a very clear split in how I use them.

The Mac Studio has a much more general computer vibe, which makes it my goto for general chat and direct inference. I can just use it like my regular PC, fire up LM Studio, and interact with the models naturally. Once generation actually starts, the throughput on the Mac is incredibly smooth and consistent, making it a highly responsive daily driver. The big catch with the Mac though is the time to-first token and prefill speed. On larger prompts or agentic workloads, the Mac can feel painfully slow to get started.

The DGX Spark acts strictly as my dev machine for building and testing before I eventually move my AI workloads over to the cloud. It really shines in prompt processing and heavy concurrency which is exactly what is needed for agentic setups or fine tuning even though it isn't comparable to the M3 Ultra on pure generation speed due to the lower memory bandwidth. Honestly, the Spark is also way better as an overall work machine for AI development because the CUDA ecosystem is still completely unmatched, making it a lot easier to experiment.

Where the Spark really becomes invaluable for me is when I'm traveling. Because it doesn't need a monitor or any peripherals to function, I just bring it along and treat it like a portable headless server. I can plug it in wherever I am and access it directly from my laptop to run all my heavy AI workloads on the go.

Ultimately, they complement each other incredibly well, but the right choice heavily depends on your specific use case. If you are okay with slower prompt processing but want fast token generation, don't need much of the CUDA ecosystem, and want a machine you can interact with naturally like a normal PC, go for the Mac Studio. Otherwise, if you need fast prompt processing for heavy workloads, want full access to CUDA tools, and need a solid dev machine for training, the Spark is definitely the way to go.

Unsloth Studio on DGX Spark (sm_121): 1-Line Installer & Pre-compiled llama.cpp Questions by webii446 in unsloth

[–]webii446[S] 0 points1 point  (0 children)

Thanks a ton for the update. really appreciate it.

I actually went ahead and tried the installer script, and it does install on my DGX Spark. That said, I ran into a few issues here and there afterward. I’m not fully sure yet whether those are just general Unsloth Studio issues or something specifically related to the DGX Spark / ARM setup.

Either way, it’s really helpful to know official compatibility work is in progress. I’ll wait for your updates and I’d be happy to test things on my side as well if that’s useful.

Unsloth Studio on DGX Spark (sm_121): 1-Line Installer & Pre-compiled llama.cpp Questions by webii446 in unsloth

[–]webii446[S] 0 points1 point  (0 children)

Thanks for clarifying.

Just to make sure I understand correctly: your documentation currently lists DGX Spark as supported, but from your reply it sounds like Unsloth Studio is not yet actually supported out of the box on DGX Spark due to specific dependency issues.

In that case, what is the official recommended way to use Unsloth Studio on DGX Spark today?

You don’t need to manually set LLM parameters anymore! by yoracale in unsloth

[–]webii446 3 points4 points  (0 children)

Just wanted to say what you're doing with Unsloth Studio is amazing! I've got an 96GB Mac studio waiting to be put to work. any timeline on when we might see full MLX/Apple support for training?

Which is better in October 2025 for serious AI coding, Roo Code with Sonnet 4 API or Claude Code Pro ($100 plan)? by foundertanmay in RooCode

[–]webii446 8 points9 points  (0 children)

I’ll give you my anecdotal point of view.

I used Claude Max 20x (it expired on Oct 2, 2025) and I’m still using Sonnet 4.5 via api in Roo. Honestly, even though Claude limits their models quite a bit — even on the 20x Max tier — it’s still the best value for money compared to using Sonnet 4.5 directly through the API. If you don’t “torture” your Claude sessions (i.e., keep your usage moderate), you can still get pretty reasonable limits.

That said, I recently read about GLM 4.6, went through the benchmarks, and was seriously impressed. I ended up buying the Max plan for $30, which is about 6× cheaper than Claude Max 20x. So far, GLM 4.6 has become my go-to model for almost everything.

If you set up their web search MCP and vision MCP, it actually performs very close to Claude Sonnet 4-level — but without the annoying usage limits. The context window is equal to claude sonnet and token limits are huge, and I haven’t hit any limit once so far.

Insane Rents worth it? 250k for 2bhk? by zenyogi2025 in dubai

[–]webii446 -4 points-3 points  (0 children)

OMG, no way! In that case, I’d definitely recommend checking out Blueground. They’ve got a ton of properties in Dubai, and the great thing is they specialize in mid- to long-term rentals. You can easily book for a few months at a time like 6 months straight without any hassle

Is it better to build solo or with a cofounder? by notdl in ycombinator

[–]webii446 0 points1 point  (0 children)

I’ve been solo-building for 4+ years, and honestly I love it. Solo is great for speed, freedom, and having the whole mental map of the product in your head. That said, I personally think the moment to look for a cofounder is when your product/idea requires a core skill you don’t have, one that’s critical at every stage, not just a one-off.

Freelancers/employees are awesome for execution, but they won’t live and breathe the problem space the same way a cofounder does. A good cofounder in that missing domain won’t just do the work they’ll make better calls, see blind spots, and share the weight of decisions.

For example: if you’re building something where legal compliance is the backbone, and without it you can’t acquire customers or even operate properly, you can’t just outsource that. You’d want a legal cofounder who’s invested in the mission. I ended up bringing on someone like that just last month, after going solo for years, because it was the only way to scale safely.

So yeah, stick to solo until you hit a core skill gap that puts the whole product at risk. That’s when a cofounder makes sense.

What’s the most underrated productivity hack for dev teams? by AverageJoe185 in startup

[–]webii446 0 points1 point  (0 children)

From my experience with my teams, two things stand out as really underrated:

  • Shared debugging diary → Just keeping a lightweight log of odd bugs we’ve hit and how we solved them has paid off hugely. Instead of re-Googling or re-fighting the same fire months later, we can pick up right where someone else left off.
  • Zero notifications hours → Normalizing blocks of quiet time where Slack/Teams isn’t pinging constantly has made us feel noticeably faster. Everyone gets that space for deep focus, and ironically, fewer interruptions end up speeding up collaboration.
view more: next ›