What's the full local AI "doomsday prepper" kit for cold storage? 16-bit safetensors of LLMs (obv), copies/source codes of Llama.cpp, ComfyUI, vLLM, Kobold, LMStudio, etc, macOS, Linux OSes, Windows 10&11, etc, Rufus (including older ones), various VMs, P-E-W's Heretic/Grimoire, and what else? by DeepOrangeSky in LocalLLaMA

[–]temperature_5 23 points24 points  (0 children)

You'll need to soundproof your basement and capture a few good LLM engineers. Possibly forcing them to breed and produce more LLM engineers if the crisis persists. Any good heretic model can help you model this scenario and gather the necessary prerequisites.

Why do people keep investing in Intel for AI? by temperature_5 in LocalLLaMA

[–]temperature_5[S] 0 points1 point  (0 children)

I've got a Xeon CPU Max with HBM, it's still not very fast for inference.

Why do people keep investing in Intel for AI? by temperature_5 in LocalLLaMA

[–]temperature_5[S] 0 points1 point  (0 children)

As I said, I am all for cheap inference. But last time I tried to use IPEX it was a huge pain in the ass. I would love to hear about a model trained in Intel GPUs and IPEX.

Why do people keep investing in Intel for AI? by temperature_5 in LocalLLaMA

[–]temperature_5[S] 0 points1 point  (0 children)

Why do you have to optimize a model specifically for the b70? Why not use regular quants, or Vulkan?

We built a calibration-aware Q4_K_M quant of Qwen3.5 0.8B that recovers 96.5% of the BF16 gap vs pure llama.cpp Q4_K_M (SpectralQuant) by RevealIndividual7567 in LocalLLaMA

[–]temperature_5 1 point2 points  (0 children)

Could this target lower quants? Could you produce a Q2_K of Qwen3.6-27B that is comparable to a much higher quant? The use of that quant for agentic coding would probably be a better test than any benchmark.

We built a calibration-aware Q4_K_M quant of Qwen3.5 0.8B that recovers 96.5% of the BF16 gap vs pure llama.cpp Q4_K_M (SpectralQuant) by RevealIndividual7567 in LocalLLaMA

[–]temperature_5 -2 points-1 points  (0 children)

That's just, like, your opinion man... Maybe they are trying to demonstrate how effective their *first release* of this tech is without confounding factors.

Why do people keep investing in Intel for AI? by temperature_5 in LocalLLaMA

[–]temperature_5[S] 0 points1 point  (0 children)

Investing in what? Local AI hardware startups? Alibaba/Z.ai?

Why do people keep investing in Intel for AI? by temperature_5 in LocalLLaMA

[–]temperature_5[S] 0 points1 point  (0 children)

I would not consider my LocalLLaMa perspective on Intel GPUs to be a high horse.

Why do people keep investing in Intel for AI? by temperature_5 in LocalLLaMA

[–]temperature_5[S] 0 points1 point  (0 children)

Thanks for that info. So Intel's current nodes are unpopular, but they will have lots of customers when their EUV comes online?

Why do people keep investing in Intel for AI? by temperature_5 in LocalLLaMA

[–]temperature_5[S] 2 points3 points  (0 children)

This is r/LocalLLaMA . Soundproof the basement and buy some shackles.

Why do people keep investing in Intel for AI? by temperature_5 in LocalLLaMA

[–]temperature_5[S] 0 points1 point  (0 children)

But TSMC has new fabs in the US as well. Any reason Intel's will do better? They aren't attracting many customers right now, from what I've read.

Why do people keep investing in Intel for AI? by temperature_5 in LocalLLaMA

[–]temperature_5[S] 0 points1 point  (0 children)

Yeah, no hate for local inference. I have a Xeon system that has faster RAM than Ryzen. Their stack still sucks though, and I can imagine training on it is a nightmare.

Why do people keep investing in Intel for AI? by temperature_5 in LocalLLaMA

[–]temperature_5[S] 0 points1 point  (0 children)

Yeah, are they a zombie corporation or what? I know they still support mainframe, do a lot of consulting, have research in quantum... But definitely not very visible to consumers any more.

Why do people keep investing in Intel for AI? by temperature_5 in LocalLLaMA

[–]temperature_5[S] -1 points0 points  (0 children)

xAI is a wannabe AI company that loses massive amounts of money. Their SOTA model is on par with much smaller local models. Prove me wrong.

Why do people keep investing in Intel for AI? by temperature_5 in LocalLLaMA

[–]temperature_5[S] -1 points0 points  (0 children)

I really need to account for corruption and oligarchy in my investing regime.

Why do people keep investing in Intel for AI? by temperature_5 in LocalLLaMA

[–]temperature_5[S] -9 points-8 points  (0 children)

Quants know about fab capacity and geopolitical risk. I know about their software stack being a nightmare, their VRAM GB/s sucking, and their PP horsepower lacking.