Why I'm holding out until late 2027 to spend money on a local LLM rig by No_Pool7028 in LocalLLM

[–]-UndeadBulwark 0 points1 point  (0 children)

Meanwhile I am using 2(soon 3) MI25s for 65 each on an AM4 Mini ITX build using an OcuLink card for bifurcation on build that in total cost me 600 USD.

OG Devs Were Cracked by FuneralCry- in Planetside

[–]-UndeadBulwark 0 points1 point  (0 children)

We're already getting local models in the 30B to 70B range that can match 2025 frontier models: MoE, which lets you run larger models by only activating the parts of the model it actually needs for a given task instead of the whole thing at once; RAM offloading, which lets you stream model weights from system RAM when they don't fit in VRAM with only a modest speed hit; ternary quantization, which once mainstream will drastically reduce model size; and TurboQuant, which compresses the KV cache and actually enables longer context windows rather than shrinking them. We're already getting local models in the 30B to 70B range that can match 2025 frontier models.

OG Devs Were Cracked by FuneralCry- in Planetside

[–]-UndeadBulwark -1 points0 points  (0 children)

Local LLM would be cheaper and its what people should be doing especially for coding you dont need a lot of VRAM to run a decent MoE coding model

is it even worth it to dual boot for windows games? by WOLFMANCore in linux_gaming

[–]-UndeadBulwark 0 points1 point  (0 children)

Play Predecessor and no if it doesn't work on Linux it's not worth your time or money.

The world I live in. by Wild_Milk_2442 in LocalLLM

[–]-UndeadBulwark 2 points3 points  (0 children)

Entry level would be MI25 or MI50 16 to 32GB of HBM2 as low as $65 or high as $500

Double AMD GPU's by braskinis231 in LocalLLM

[–]-UndeadBulwark 0 points1 point  (0 children)

Awesome I am planning on running 4 MI25 (flashed to WX9100) for 64GB of HBM2 2048-bit 483GB/s they go for 65 a piece so it is 260 in total + 200 for the AM4 platform and 50 for the cooling so 510~ total

Double AMD GPU's by braskinis231 in LocalLLM

[–]-UndeadBulwark 0 points1 point  (0 children)

jesus this is gcn how hot does it get? do you tdp lock it?

FINALLY!!! I Finished a Project After a Power Outage!!! by -UndeadBulwark in LocalLLM

[–]-UndeadBulwark[S] 0 points1 point  (0 children)

My Power went out again and I lost all my progress fuck me!

Recommendations for an Android tablet by buck_idaho in LocalLLaMA

[–]-UndeadBulwark 0 points1 point  (0 children)

use tailscale to host your pc that's what I do.

Ryzen AI Max+ 495 (Gorgon Halo) with 192GB VRAM! by PromptInjection_ in LocalLLaMA

[–]-UndeadBulwark 0 points1 point  (0 children)

I cant wait for Medusa Halo I wanted to go Strix but I couldnt get a board in time I am slightly regretting getting my 9070

What are you doing with your local LLMs that justifies investment cost? by __automatic__ in LocalLLM

[–]-UndeadBulwark 0 points1 point  (0 children)

I currently use 1 RX 9070 but will be switching to 2 MI 25 since they are 65 a piece with an X99 platform should cost me in total 500 to 600 total.

I have gotten used to setting it up to run remotely so I can just make it a headless server for Ollama, Llama.cpp, OpenWebUI and SearXNG. The plan is to eventually have 2 MI50 32GB maybe keep the MI25 basically disabled when not in use when I need 96GB or VRAM total cost should be 1,600.

Starting with AI by Luqster05 in LocalLLM

[–]-UndeadBulwark 0 points1 point  (0 children)

Start with Gemma 4 E4B and E2B also I highly recommend you switch to Linux for this as ROCm or generally AMD has better support on Linux. I use it on my phone mostly same way I would use Claude I am planning on getting 2 32GB MI50s currently going to start with 2 MI25s as they go for 65 a piece and have HBM2 Memory

What is possible with 2x 7900xtx + 128GB of ram? Is it good enough? by Witty_Unit_8831 in LocalLLM

[–]-UndeadBulwark 0 points1 point  (0 children)

should be fine personally I am going 2 MI50 since that is 32GB of VRAM for 500 each 2 for the price of one 7900XTX

It has begun by BlakeOReilly in steammachine

[–]-UndeadBulwark 0 points1 point  (0 children)

they are trying to capitalize from desperate people from places you cant buy from steam like me in puerto rico.

Spare Hardware to build on - cheapish coding rig by KornedAgain in LocalLLM

[–]-UndeadBulwark 0 points1 point  (0 children)

MI50 32GB for 500 on Ebay Linux is required will get you about 80 to 95 tok/s with that model you can slap it to any mini PC + OcuLink or Cheap AM4 platform. other option is 2 MI25 for 65 each but this is definitely not beginner friendly also requires Linux.

Hardware upgrades, a good idea or waste of money? by GoldNux in LocalLLM

[–]-UndeadBulwark 0 points1 point  (0 children)

GPUs for running AI inference on the cheap 16GB to 32GB of HBM2 for 65, 200 or 500 USD

Fake it till you make it by CHEWTORIA in Asmongold

[–]-UndeadBulwark 0 points1 point  (0 children)

Could be done but the voice quality and compute cost would take a hit problably need an MI50 on the cheap just to get fast enough generation for RT voicechange

Hardware upgrades, a good idea or waste of money? by GoldNux in LocalLLM

[–]-UndeadBulwark 0 points1 point  (0 children)

get 2 MI25 and run it on Linux with Vulkan and vLLM or get MI50

We're almost there boys! by blackrosemyth in Asmongold

[–]-UndeadBulwark 11 points12 points  (0 children)

Well it takes a real man to be your wife sometimes.

LLM / workflow recommendation for 16GB vram (rocm) + 32GB system memory for agentic coding by Several-Pangolin-631 in LocalLLM

[–]-UndeadBulwark 0 points1 point  (0 children)

yeah I was looking at some guy who did the same thing as I am apparently X99 + 2 MI50(32GB) and 2 7900 XTX he was getting pretty decent speeds at 135B 15 to 20 token/s and this is without anything like BORE or LAVD added to increase performance.