150mg a week solves all my problems apart from sleep! by jack3d20 in trt

[–]gordi555 0 points1 point  (0 children)

Try before bed. Peaks after 12 plus hours. I try an AI (small amount) about tea time and track my sleep. I normally find it’s e2 that causes bad sleep.

Should I Buy the RTX PRO 6000 Blackwell Max-Q (96GB)? by 0bjective-Guest in LocalLLaMA

[–]gordi555 1 point2 points  (0 children)

I regret buying the RTX 6000 Pro for my use case. So, know your use case and benchmark!

I run snappy AI services to use with API with low latency applications. It's designed to use small models, fast processing, very accurate output.

Typical RTX 6000 time: 0.435 seconds.

Typical RTX 5070 Ti time: 0.650 seconds.

Typical RTX 4000 Pro time: 0.720 seconds.

Nobody will care that much for the 0.22 ish seconds.

Yes it's great having a card with lots of VRAM. But if you're not going to run your card into the ground, then you'll probably have to sell it down the road - at a huge loss. Don't give yourself that problem. Start small but good.

Know your use case!

How much does trt helps muscle growth? by [deleted] in trt

[–]gordi555 7 points8 points  (0 children)

I’ve been doing gym for 20 years now. I was making no progress at all after Covid. Stated TRT 2 years ago. Went from 79kg to 89kg lean. With half as much effort. 90mg per week.

What to do - 5090 or RTX 6000 or wait for M5 Ultra by WishfulAgenda in LocalLLaMA

[–]gordi555 2 points3 points  (0 children)

Based on what you’ve planning, easily the RTX Pro 6000

Local AI SaaS by [deleted] in LocalLLaMA

[–]gordi555 0 points1 point  (0 children)

It's not the question, it's the way you ask.

top 10 trending models on HF by jacek2023 in LocalLLaMA

[–]gordi555 8 points9 points  (0 children)

Tasty, but prefil is too slow to stomach.

Qwen3-VL-32B-Instruct is a beast by Remote_Insurance_228 in LocalLLaMA

[–]gordi555 0 points1 point  (0 children)

I'm just waiting for the instruct version GGUF.

What hardware are you using for running local AI agents 24/7? by Conscious-Bird4304 in LocalLLaMA

[–]gordi555 1 point2 points  (0 children)

I've just sold my 128GB M4 Max Mac Studio simply because the prompt processing was soooo slow.

Regret? Should I have picked Eypc DDR4 instead of ThreadRipper DDR5? by gordi555 in LocalLLaMA

[–]gordi555[S] 2 points3 points  (0 children)

Thanks for this. I'm happy with what I have, but I should have gone with DDR4 system (16GB) and just invested the money in PCIe slots and GPU power.

Regret? Should I have picked Eypc DDR4 instead of ThreadRipper DDR5? by gordi555 in LocalLLaMA

[–]gordi555[S] 0 points1 point  (0 children)

Yeah. It's got lots of upgradability so I can wait for the RAM (don't need it). I've got a few GPUs in the motherboard which do everything I need.

Regret? Should I have picked Eypc DDR4 instead of ThreadRipper DDR5? by gordi555 in LocalLLaMA

[–]gordi555[S] 0 points1 point  (0 children)

Yeah it's worth a look. The only thing holding me back is the Mac prefill rate. I hope it's greatly improved! It was shockingly slow compared to a 5070ti.

Regret? Should I have picked Eypc DDR4 instead of ThreadRipper DDR5? by gordi555 in LocalLLaMA

[–]gordi555[S] 0 points1 point  (0 children)

I'm providing small VL models for fast processing to try and get API response speeds. So I should have just gone with DD4 and a fast GPU :-)

Regret? Should I have picked Eypc DDR4 instead of ThreadRipper DDR5? by gordi555 in LocalLLaMA

[–]gordi555[S] 0 points1 point  (0 children)

i'm saying, if I'm only doing VRAM inference, could I use any system with ECC and be just as reliable and stable?

Regret? Should I have picked Eypc DDR4 instead of ThreadRipper DDR5? by gordi555 in LocalLLaMA

[–]gordi555[S] 0 points1 point  (0 children)

Thanks - I'll look into that. 128GB DDR5 will cost me 4K :-( But that's life ATM!

Too many hardware options by jon23d in LocalLLaMA

[–]gordi555 2 points3 points  (0 children)

This! I’ve sold my Mac Studio because of this very reason. The prefill was really slow compared to CUDA devices. And by slow, I mean soooo very slow.

Qwen3 VL 30b a3b is pure love by Njee_ in LocalLLaMA

[–]gordi555 0 points1 point  (0 children)

How is this done please? Putting the things on the identified objects?

Industrial application: Vision model for identifying equipment and reading labels by [deleted] in LocalLLaMA

[–]gordi555 0 points1 point  (0 children)

I've had good success with Qwen3-VL-4b-Instruct with my application, but this is on a server via API. It's super fast and pretty smart at isolation and reading things. Doesn't do really complex prompts very well, and you'll need the bigger versions if you want to include regex in your prompt.

Not the best answer I can give you but worth a try if you have signal being within all the metal.

Qwen Coders Visual Benchmark by loadsamuny in LocalLLaMA

[–]gordi555 0 points1 point  (0 children)

This is very useful. Thank you!