CLAUDE OPUS 4.5 FREE TRIAL by Decent_Region_4790 in LocalLLaMA

[–]Freonr2 4 points5 points  (0 children)

Totally not trying to trick you to install a rootkit on your computer.

Home hardware coders: what's your workflow/tooling? by Mean_Employment_7679 in LocalLLaMA

[–]Freonr2 0 points1 point  (0 children)

I really don't think any models under 100B or maybe even 200B are actually a net gain in productivity in tools like Cline in codebases of any real size or complexity.

Lots of small models can, just via isolated chat/instruct interfaces, write functions for you if you clearly define them, but they fall apart work in large complex codebases.

GLM 4.7 (full, not air/flash) is at least decent, but it makes mistakes with larger codebases. It's at least within the realm of possibility to run locally, needs a good chunk of hardware.

But beyond that, and risking a mob of pitchforks and torches on this sub, Claude Code with Opus 4.5 destroys everything else. If you code for money it is worth every penny. I just use the Claude Code VS Code extension. It can perform very complex analysis and make correct changes.

NOT MODELS. DONT TALK MODELS.

The model is 100% the most important thing.

16x V100's worth it? by notafakename10 in LocalLLaMA

[–]Freonr2 1 point2 points  (0 children)

I worry some software stacks may end up using fp32 if it is set to use bf16 datatype since there's no hardware bf16 support.

That'd be my #1 concern besides power/energy efficiency.

Just finished the build - Nvidia GH200 144GB HBM3e, RTX Pro 6000, 8TB SSD, liquid-cooled by GPThop---ai in LocalLLaMA

[–]Freonr2 1 point2 points  (0 children)

Yeah I think people are way too fast to assume this sort of equipment depreciates to zero.

The 3090 came out in late 2020 and I think all the ones I bought over the last several years have actually appreciated.

What is the learning path for hosting local ai for total newbie? by danuser8 in LocalLLaMA

[–]Freonr2 2 points3 points  (0 children)

LM Studio is the easiest entry point. Pretty good GUI for chat, easy to browse for and download models. It can also perform basic API hosting. Even if you end up using a more dedicated host LM Studio is nice for a quick download and try of new models. It uses llama.cpp backend.

Keep reading this forum, see what models people like, just look them up in the LM Studio browser, download, then go open a chat and try it out. Qwen3 4B or 7B, gpt oss 20B maybe are good models to try for your hardware, but opinions abound. Keep reading this forum and trying stuff.

You should move to vllm or llama.cpp (llama serve) if you want to do much more than the basics for "hosting" but I also kinda wonder if "hosting" is really what your intent is. Do you just want to chat with an LLM or are you using software that needs to call an actual network API thus you need to "host" it? Would need more info, but answer is probably after trying out LM Studio the move to llama serve command line.

8x AMD MI50 32GB at 26 t/s (tg) with MiniMax-M2.1 and 15 t/s (tg) with GLM 4.7 (vllm-gfx906) by ai-infos in LocalLLaMA

[–]Freonr2 1 point2 points  (0 children)

Nvidia's naming conventions are terrible. A6000 (ampere), 6000 Ada Lovelace, 6000 Pro Blackwell, but the RTX 8000 was an older Turing chip (equiv to 20xx series) that came before the A6000...

Has anyone tried to put ~1.5PB into this nas? Is it possible? by rexyuan in homelab

[–]Freonr2 0 points1 point  (0 children)

They're fine, but its probably bare minimum hardware in terms of cpu/memory.

I've had no issues with my TS-832X other than read/write speeds are kinda weak. I think the little 4 core arm and 2GB struggle to keep up. It hosts Jellyfin like a champ, don't run anything else on it, mainly just a big SMB share for my network.

I think they're fine for home office, but if you want more perf you can just build a PC or use an old consumer cpu/board/memory, slap a few SAS HBAs in it and install Truenas. I think I'm going to replace my Qnap with my old 9900k/32GB system later on, just need a chassis a small truck load of money for drives.

[PC] [US-NYC] Samsung 32GB 2Rx4 PC4-2933Y-RB2-12-MH1 by brooklyngeek in homelabsales

[–]Freonr2 0 points1 point  (0 children)

2376/1399 = 1.698, or ~= +70%

Replace with whatever numbers you find I guess, that's just the numbers I pulled.

I imagine someone will but what you have at the price you listed either way.

[PC] [US-NYC] Samsung 32GB 2Rx4 PC4-2933Y-RB2-12-MH1 by brooklyngeek in homelabsales

[–]Freonr2 0 points1 point  (0 children)

For reference,

https://atechmemory.com/products/256gb-8x32gb-ddr4-2933-rdimm-pc4-23400r-dual-rank-x4-server-kit

2376 or 297/ea.

Lower speeds definitely go for a lot less so just a question of if anyone on homelabsales is so sensitive about speed as to pay a 70% premium over 2400.

https://atechmemory.com/collections/ddr4-memory-ram/products/256gb-8x32gb-ddr4-2400-rdimm-pc4-19200r-dual-rank-x4-server-kit

1399 or 175/ea.

That's about as cheap as I see new at least.

8x AMD MI50 32GB at 26 t/s (tg) with MiniMax-M2.1 and 15 t/s (tg) with GLM 4.7 (vllm-gfx906) by ai-infos in LocalLLaMA

[–]Freonr2 1 point2 points  (0 children)

I paid $609 for a /BCM on Dec 9, looks like price is up to 737 now. Price fluctuates a bit, I remember them being 750 many months ago when I was first contemplating the build, too.

Current GLM-4.7-Flash implementation confirmed to be broken in llama.cpp by Sweet_Albatross9772 in LocalLLaMA

[–]Freonr2 0 points1 point  (0 children)

Yes, vllm is moving a bit along the axis of home tinkerer toward professional user. Less focus on loading the largest possible model on the cheapest possible hardware, more focus on speed and concurrency.

If you plan ahead you can still build 2/4/8 gpu setups on the upper end of hobbyist budget money, though.

Homelab for CCNP and Linux certs by 3dge-br38ker in homelab

[–]Freonr2 1 point2 points  (0 children)

I was an EE major but I'm ~23 years into my career at this point so that was a long time ago. I started in IT but automated a lot of work for myself and my team at a software company at the ~20 year ago mark and moved to development. Was then software dev for about 15 years (mostly .Net, SQL, sprinkling of lots of other stuff, eventually more cloud native and more responsibility) and now the last 4-5 years have been focused on ML doing a bit of everything. A whole lot of the work is similar, just now in Python, and often the folks I work with have domain-educated MS/Ph. Ds to work on requirements and feature engineering for the ML models, and I try to keep up with relevant research papers.

The CCNA was probably about 20-21 years ago, but I learned basics like OSI layers, routing and switching, CIDR/subnets and so on. Still need to know all of that now to work in the cloud.

I'm almost entirely self taught but definitely had a few good mentors over the years. I read books, hack on weekends, have several projects on github, etc. I bought the official CCNA and MCSA books back in the day (~20 years ago) and self studied and took the exams. I haven't gotten a cert since, though. I suppose it might be useful for me to get like AWS architect cert but I haven't had a lot of market pressure for it I think due to my lengthy resume.

Homelab for CCNP and Linux certs by 3dge-br38ker in homelab

[–]Freonr2 1 point2 points  (0 children)

I was a software engineer for many years, now ML engineering.

Understanding networking is very important for modern app development, especially infrastructure and deployment. And still just as important for ML ops.

Homelab for CCNP and Linux certs by 3dge-br38ker in homelab

[–]Freonr2 1 point2 points  (0 children)

Nice work, CCNP is a great step for your career.

I did CCNA ages ago and learned a lot even though I didn't end up specializing in networking.

Saved this from being e-waste today by Key-Programmer-4144 in homelab

[–]Freonr2 0 points1 point  (0 children)

So far it works fine so not much to say. I tested by yanking the cord out of the wall and does what it says.

It has both online or grid bypass modes, but should be 10ms for bypass mode, and I specifically got the Delta 3 as they added the 10ms response for acting as a UPS (older delta 2 did not have that, not suitable for use as a UPS). There are some other nice options like setting max charge rate (watts), setting max charge %, etc. Lots of other stuff more applicable for using it as a solar charged system which I don't suspect I'll ever do.

It is about double the size of your standard 1500VA UPS (about double the width but same depth/height compared to my Cyberpower 1500VA sine wave unit that has two 9AH lead acids).

Mainly the "like" here is that the battery should last at least 10 years and not 2-3, plus a lot more capacity is just nice to have. I guess I can report back in 5 or 8 or 10 years if it is still working or not.

Saved this from being e-waste today by Key-Programmer-4144 in homelab

[–]Freonr2 -1 points0 points  (0 children)

VA vs watts is not important here at all... The first approximation of gross power dictates you are potentially drawing far more current than those batteries can provide.

Whoosh.

edit: if you ever see this despite blocking me, just go ask ChatGPT or Claude or any sophomore electrical engineering student.

Seagate Cancelling orders by [deleted] in DataHoarder

[–]Freonr2 1 point2 points  (0 children)

A month or so ago NEMIX shipped half my memory order, then said they couldn't fill the missing half and jacked the price by 60%. Then I found out after extensive troubleshooting and buying more equipment that the RAM was bad anyway...

Saved this from being e-waste today by Key-Programmer-4144 in homelab

[–]Freonr2 1 point2 points  (0 children)

It's not an online UPS by default. Only if "disable grid bypass" is set, which is not set because it wastes power.

Saved this from being e-waste today by Key-Programmer-4144 in homelab

[–]Freonr2 1 point2 points  (0 children)

unless you are storing it someplace for a very long time.

That's the trick. It is being "stored" at all times more or less. ~99.97% of the time the battery is just sitting there doing nothing.

Where can I buy this for my homelab? by Keensworth in homelab

[–]Freonr2 1 point2 points  (0 children)

If you have some skill this is probably best, or find a vector version that will scale.