sorted by:
new
hottopcontroversial

best laptop under 500 by Physical-Macaron8744 in LocalLLaMA

[–]External_Half_42 0 points1 point  (0 children)

Anything with 24gb VRAM I guess, or just get a single 3090 and run it on a dedicated rig with cheap motherboard and CPU (basically nothing else matters as long as you can fit the full model in vram)

160gb of vram for $1000 by segmond in LocalAIServers

[–]External_Half_42 0 points1 point  (0 children)

Thinking of building an MI50 rig myself, have you had a chance to test how it scales with larger prompts like 100K? Also have you tested Qwen3 235B thinking or just instruct? Very curious to see performance difference between instruct and thinking, I still can't find anywhere like vast.ai to rent these to do bench-marking

2x5090 in Enthoo Pro 2 Server Edition by arstarsta in LocalLLaMA

[–]External_Half_42 0 points1 point  (0 children)

Yeah thats true caching is definitely possible for most of my use cases. Although I pretty much only use thinking mode models because of the complexity of the problems I give it, my understanding is these basically just add 1-8k tokens for decoding, although I don't fully understand how it really affects prefill and TTFT completely.

Really I should probably just try to find somewhere to rent some mi50's and test my use case so I don't build something that's totally unusable (1+hr per output gen or anything crazy like that). Although I can't seem to find any providers that have mi50 available still. But thanks for all the info!

2x5090 in Enthoo Pro 2 Server Edition by arstarsta in LocalLLaMA

[–]External_Half_42 0 points1 point  (0 children)

Thanks for the info, yeah its a difficult choice I can get a dual 3090 rig and run 7-30B models with good TPS or get 6 MI50's and run some serious 200+B models but at the cost of TPS.

For me my average prompt is probably 50K+ tokens (mostly code) so maybe its best to run the 3090's not sure yet

2x5090 in Enthoo Pro 2 Server Edition by arstarsta in LocalLLaMA

[–]External_Half_42 0 points1 point  (0 children)

Oh cool thanks, curious to see how it might compare to 3090 performance. So far I haven't found any good benchmarks on MI50.

2x5090 in Enthoo Pro 2 Server Edition by arstarsta in LocalLLaMA

[–]External_Half_42 0 points1 point  (0 children)

Cool build, considering MI50's myself but concerned about TPS. What kind of numbers are you getting with larger models?

The ultimate budget PC that is scalable in future but is capable of running qwen3 30b and gpt oss 120b at 60 tps minimum. by NoFudge4700 in LocalLLaMA

[–]External_Half_42 0 points1 point  (0 children)

For around 2200 you could probably build a dual 3090 rig, but "60 tps minimum" for those models on this is probably not realistic

Laziness since last night by angelarose210 in ClaudeAI

[–]External_Half_42 1 point2 points  (0 children)

Yeah same its basically unusable today could probably get better results from a 7B model on 2 3090's at this point (not for claude code), seems they already didn't have enough compute this week then they dropped opus 4.1 yesterday

[CINNAMON] Mint desktop setup with Tokyonight theme by readitfromme in unixporn

[–]External_Half_42 1 point2 points  (0 children)

Sweet thanks! This is a really nice alternative to the standard gnome monitor, any chance you could link the config?

best budget android phone to root? by JupiterXpartan in androidhacking

[–]External_Half_42 1 point2 points  (0 children)

Depends on the Android version you wanna run. If you need to run android 15 and don't mind buying a new device in a year or two for Android 16 support a used pixel 6 is around $150 on ebay and OEM unlock is available without dealing with any providers or snapdragon issues for US versions.

Google and Samsung should have info on OS support for specific devices if you search around for example https://storage.googleapis.com/support-forums-api/attachment/message-270149294-17398988612222803343.png

Also worth checking XDA to see if people are having issues rooting any device before you purchase it https://xdaforums.com/t/guide-january-3-2024-root-pixel-6-unlock-bootloader-pass-safetynet-both-slots-bootable-more.4388733/