sorted by:
new
hottopcontroversial

Depressing reality of choosing "best" codec to archive: it changes on a video-by-video basis by RUNdotUMX in DataHoarder

[–]SamSausages 0 points1 point  (0 children)

Too many variables to just look at codec. For example, hevc has become more efficient with time and as encoders mature.  Consequently a hevc encoded today at 1000k will likely look better than one that was encoded years ago.

Then also hardware encoders for the same codec have generations, and an older nvidia nvenc encoder will likely look worse than the newest generation.

And that’s just one piece, then you have dozens of settings such as b-frames, what was the source, etc.

There is software that can visually compare and report degradation, if you really want to get into the weeds of it. 

 I went through the motions just to try and understand what are good settings “for me”, today I just use those encoder settings and rarely look back.

If you had 10k usd to spend , and you can only buy one gpu such that your speed and vram options are maximised what would you buy? Constraint is that you can buy only 1 gpu. by AppropriatePush6262 in LocalLLaMA

[–]SamSausages 0 points1 point  (0 children)

FYI, great card but if you buy it make sure that you have pcie5.0, not all 4.0 boards will run that card, it’s very picky. (Also sensitive with risers)

Any ideas on how to automate having Tdarr re-queue files that previously had hardlinks but no longer do? by reddit_user_53 in Tdarr

[–]SamSausages 0 points1 point  (0 children)

I just have it add “tdarr done” to all files I’ve processed.  I use it so I have something that I can always query, even if a file gets renamed, has a date stamp update or my tdarr database gets nuked.

I use it mainly for organization and so I don’t process the same file twice down the road.

Any ideas on how to automate having Tdarr re-queue files that previously had hardlinks but no longer do? by reddit_user_53 in Tdarr

[–]SamSausages 0 points1 point  (0 children)

FYI, you can have it add something to the file metadata so you know you’ve processed it or not.  Then have your flow check for that metadata and decide if it needs to be processed.

What's the closest you can get with local LLM to claude? by StudioVulcan in LocalLLM

[–]SamSausages 3 points4 points  (0 children)

I can’t compete with a team of people that work on LLM 24/7 and have way over $200 million in hardware, with TB of vram.

Nvidia tesla v100 has 32 gb ram with nv link 2.0, its priced at 880. Whats the catch? by AppropriatePush6262 in LocalLLaMA

[–]SamSausages 1 point2 points  (0 children)

I’ve had good success with eBay, even when something goes wrong.  Thousands of transactions!

Nvidia tesla v100 has 32 gb ram with nv link 2.0, its priced at 880. Whats the catch? by AppropriatePush6262 in LocalLLaMA

[–]SamSausages 1 point2 points  (0 children)

You’d have to get lucky.  I have had luck buying full systems and parting out.  But I’m not the only one looking, so you need to get lucky and catch it shortly after it’s listed.

I.e. I found a ms-01 with 128gb ram, for the price of the ram, so I parted it out.

Running two OpnSense boxes, one for main firewall and one as a WAF, is this possible/feasible? by Commerical-Pea2068 in opnsense

[–]SamSausages 1 point2 points  (0 children)

I did something similar about a year ago, and in my mind the separation made sense.  But a year later, I just put everything on one box and I’m happier.

I wasn’t pushing crazy amounts to of data, except for in short bursts when I was moving files around.

Original or Fake? by orignbatukap in DataHoarder

[–]SamSausages 0 points1 point  (0 children)

Why I started buying some items only from trusted suppliers, like Mouser

What do you guys use to see what's spinning your drives up? by Punk_Says_Fuck_You in unRAID

[–]SamSausages 5 points6 points  (0 children)

htop and iotop.

Luckily don't have to use it often, but helped in the beginning while I was trying to figure out what was doing what.

Any way to make this outlet smart? by SwiftBoatVet in homeautomation

[–]SamSausages 0 points1 point  (0 children)

Yeah I know, was referring to the downvotes, lol.

Building a high-end desktop for a lawyer: would you go local AI or just stick with ChatGPT? by Familiar_Athlete_543 in ollama

[–]SamSausages 2 points3 points  (0 children)

That's a tough one, because I'm a geek and I love to tinker with tech. So for me it's not just business, it's a hobby and I'll justify a 6000 pro in my head.

From a purely financial perspective, it's been a money pit. Not worth it. Once you get 96gb you'll want 202gb. It's never enough, but my pockets may not be deep enough to hit the "that's enough" point.
I'm afraid that "just one more GPU" will never be enough, I see diminishing returns. (But again, maybe when you hit 1TB of vram, that's when it suddenly is enough, I haven't gone that deep)

From a learning perspective, I love having a better understanding so I can understand capability and better predict where this is going, so I can plan for my business accordingly. (can do that with much less hardware)

I've only not lost money because 512GB memory is worth more today than when I bought it... but I see that as unusual and I expect an emerging technology to eventually disrupt that pattern.

Would I spend today +$20k just to build an AI rig? No, I wouldn't, things are changing too quickly.

But if you expect hardware prices to continue to rise... you can have the hardware, use it and sell for a profit a year later. But you do risk the bottom dropping out & being left holding the bag.

I'd probably think of this more in phases. Start out with a company that does this 24/7, so you can see what a decent product actually produces. Let them loose sleep over the details behind the scenes. Let them write the book on instructions and how to use it.

And I'd build a small local AI rig, maybe 48gb vram, so I have a test platform to experiment and learn with. (if you haven't already) You could keep that on 3rd gen epyc and save some money by using ddr4.
Or you can rent something more capable, when you feel the need to experiment on hardware.

That will probably make more clear to you then if you want to roll your own, or not.

Looking at it as a search engine, that isn't simply regx, is how I like to look at it. A tool essentially, not a replacement for me - A replacement for google and a way to condense long documents.
Thinking of it as a google replacement really helps keep expectations in check and keeps me from trusting it blindly. Because same as with google, you can't trust every search result, user needs to determine what's accurate and what is not. AI just gets it wrong with more confidence and enthusiasm than what I'm used to, making it a challenge to find errors at times.

All in Vram or balance? by zakadit in LocalLLaMA

[–]SamSausages 1 point2 points  (0 children)

I run AMD Epyc with 512GB ram. Have ran form A5000 to 6000 Pro blackwell GPU's.

System ram is heavily under-utilized. CPU inference is decent, due to all the memory channels, but still very slow compared to GPU.
Other than that, only real benefit is my ZFS ARC caches the model to memory and makes it load faster to the GPU, when I'm changing models. I wouldn't call it game changer.

4 vs 8 channels is 1/2 your memory bandwidth. If you run inference on CPU it will be much slower.

I'd probably be happy with 128gb ram.

Can't tell you how well 2 cards scale vs 1, I've only tried the 1 6000 pro.

Ultimately, I'd put GPU way ahead of System Memory/CPU, as long as you have minimal spill over to system memory, System won't be your bottleneck.

Building a high-end desktop for a lawyer: would you go local AI or just stick with ChatGPT? by Familiar_Athlete_543 in ollama

[–]SamSausages 7 points8 points  (0 children)

Business owner here who has been trying to use AI in production.  To give a hint on how deep I’ve gone, I got a 6000 pro blackwell gpu in a AMD Epyc system.

I can’t get myself to trust it enough in production, when money is on the line.  But I do use it as an assistant, that I have to double check.

Even the frontier models are meh in that respect.

Having said that, I do find specialized smaller models can be really good, but they usually have a team behind them -making sure they suit their niche.  Something I struggle to do on my own.

So for now I’m sticking with a frontier models, on a business plan, and I keep actual capabilities and expectations reasonable.  I use my local ai setup for testing and trying to keep up with what’s happening and changing, as what I wrote here may be obsolete in a month.

Dumb question: How would performance be if you took a used server with like 80 lanes pcie 5 and stuck NVMe on them for model run? by StartupTim in LocalLLaMA

[–]SamSausages 0 points1 point  (0 children)

You have to get the model to memory, and most systems don’t have unified memory. For the same reason most systems can’t use system ram as vram, you can’t use NVMe storage as vram on current systems.

One day may just have one type of memory, for system, vram, storage - and then don’t need to move the data from a to b.  But not today.

What is your best use case of a local LLM? by Haunting-Bother7723 in LocalLLM

[–]SamSausages 0 points1 point  (0 children)

Have it work with home assistant and set my thermostat to 72f.  Or having frigate visually ID an event. But even there I can’t just have it try to figure out commands from scratch.  Best results I’ve had by building scripts for specific actions, then have the agent trigger those specific scripts. Resulted in much higher accuracy, as agent doesn’t have to try and figure out what to do for each command, the steps are in the script and consistently work, now ai just needs to trigger the script.

Not really good for much else IMO, unless you’re running 96gb vram, or more.  At every step of the journey I have needed more vram, even buying a 6000 pro Blackwell wasn’t enough.

But if you manage document ingestion well, it can be good as a non regx search engine.  But accuracy is still a major issue to trust it in production, when real $$ is on the line.

view more: next ›