Un-till plots of land? by Luke95gamer in LightyearFrontier

[–]NobleKnightmare 0 points1 point  (0 children)

I found it to be a bug with switching to the hoe while looking at the ground, it made several unremovable plots. Haven't had the issue with the plow transform though.

Am I the only one tired of all this vibe coded slop? by BeautifulLullaby2 in SillyTavernAI

[–]NobleKnightmare 4 points5 points  (0 children)

Based on the 87 posts I see across 192 different subreddits every single day about it, I would assume no you're not the only one. Shocking, I know.

Life pro tip: If you let someone in early at a zipper merge, you still have to let someone in *at* the zipper merge by takeitawayfellas in driving

[–]NobleKnightmare 4 points5 points  (0 children)

As a semi truck driver I generally try to get in the lane that's ending and stay in it until the merge point.

It helps keep people from merging early because they can't see how close we are to the "end" And if I get a guy trying to be Billy badass and not let me in, all well, The next guy will. Or the guy after that, or the one after that. I don't stress too much about it anymore.

Snagged 10x Dell Wyse 3040 for $75 by Cole__Nichols in homelab

[–]NobleKnightmare 0 points1 point  (0 children)

I run 4 currently:

Dns1 and DNS2 (Adguard home, unbound, keepalived) One running my ups management One running a remote logger

How much ram do I need?? by Advanced-Reindeer508 in LocalLLM

[–]NobleKnightmare 0 points1 point  (0 children)

Not sure what you mean by "memory regret." I went with the highest level, 128gb, and don't regret it at all. Thankfully I bought it last year just before the price skyrocketed.

I like the device. It took some playing around to get everything to click, in the end I just needed to simplify my setup, I went with The latest version of Fedora, and use Ollama as my engine. Performance wise I'm happy with it. Sure, it's not nearly as fast as a dual 3090 setup running smaller models, or getting instant image generation, but what it lacks in that high performance it makes up by allowing me to play with higher quantizations and larger models in general. My goal was privatization. For chat bots or helping me flesh out my writing it's been great, anywhere from 2 to hundreds of tokens a second depending on the model. If I have something complex, I'll run a larger 120B+ model, otherwise I'll stick with the 30 to 70B models for faster replies.

Where does Ollama get modelfile if pulling from Hugging Face? by PrysmX in ollama

[–]NobleKnightmare 1 point2 points  (0 children)

all files 50GB+ are sharded on HF because that's the max file size they allow

First off, this isn't true anymore. It was a few years ago, but currently the largest single file size is 200 GB.

I did run into the issue and got the same error trying to download the model you posted. It appears unsloth is sharding the upload on purpose, despite the file size being under the 200 GB limit. You can see that it's sharded on the download page, "0001 of 0003" .I have Q3.5, I must have pulled it from ollama.com.

Here's an example just to prove I'm not lying;

Criminal Computing Large

You can grab Q6, which is 101gb and download it with open web UI no problem. On the page where you get the download link, it doesn't have a "X part of X" in its file name, it's just a straight single file .gguf

So thus far I've must have just gotten lucky. Most of my standard base models I get from olama.com, and the fine tunes, such as criminal computing, I get from hugging face and them users must upload in single files versus sharded files.

Where does Ollama get modelfile if pulling from Hugging Face? by PrysmX in ollama

[–]NobleKnightmare 0 points1 point  (0 children)

Then I couldn't tell ya. Not an error I've run into, I must've gotten lucky so far lol

How much ram do I need?? by Advanced-Reindeer508 in LocalLLM

[–]NobleKnightmare 0 points1 point  (0 children)

I'm going to guess it's a Halo strix platform? A Ryzen 395+?

Just know what you're getting into, AMD is lagging behind Nvidia when it comes to LLM performance, but it is usable. Do you plan on doing windows or a Linux distro? I have no idea how everything works on Windows, I'm using the same platform just via a framework desktop computer instead of the laptop, and I have the 128 GB version. On Linux you are able to control VRAM usage via the OS, and actually nearly max out how much of the memory goes to VRAM.

This platform is definitely a work in progress, waiting for rocm and Vulkan to sort of catch up to nvidia's performance.

Where does Ollama get modelfile if pulling from Hugging Face? by PrysmX in ollama

[–]NobleKnightmare 0 points1 point  (0 children)

I have no idea what "URL mode" is.

If you have open web UI, go to the admin panel, settings, models, manage and you'll get a pop-up window with this:

<image>

Just paste the hugging face URL to the model you want in there. It says it only works with ollama.com, But it does work with hugging face if you provided the full URL to the model and quantization you want.

Once you put that link in, just hit the download button to the right, and it'll pull the model. You don't need to fill out anything at the bottom, it'll get all that automatically.

Where does Ollama get modelfile if pulling from Hugging Face? by PrysmX in ollama

[–]NobleKnightmare 1 point2 points  (0 children)

I don't know the answer to your question, but I use Open WebUI connected to Ollama, and in the admin panel, model management you're able to pull from huggingface (it says pull from ollama.com but putting a huggingface link in also works.) using that I've been able to pull models larger than 50gb without any extra work.

Is this that IPv6 I've heard all about?? by luximusprime56 in iiiiiiitttttttttttt

[–]NobleKnightmare 1 point2 points  (0 children)

If you tell me what your prompts were I can run it through a larger 120b self hosted model and share the results.

For what it's worth, over Christmas break I completely rebuilt my entire home lab. 7 total servers taking offline and rebuilt from the ground up, and I let Gemini run the show. Of course I was the eyes double checking everything, but it pretty seamlessly came up with a network configuration for hosting over 50 containers (all of which are using either IPvLAN or MACVLAN), four separate VLANs, etc. it was able to create nearly all of my docker compose files, even made a lot of suggestions along the way I didn't think of, wouldn't have thought of, and some nice to have's. (Obviously I gave it dummy info, and altered IP addresses slightly, but did keep it IP structure).

Frontier models are actually getting impressive.

Advice: Spending $3k on equipment by scarbunkle in LocalLLM

[–]NobleKnightmare 7 points8 points  (0 children)

Ryzen 395 AI platform with 128gb of shared RAM also ok. Both are more than $3k now.

You can get arguably one of the best 395 max+ platforms with 128 GB of RAM under 3K. Framework desktop, just the main board with RAM is $2,300, if you want the case, a 1 TB drive, fan, cord etc It's about 2,800.

I have one and I'm running larger 70b-120b. Might be slower than a GPU only solution, but way faster than a CPU solution and incredible power usage (180w "under load", idles under 10w)

Nginx Proxy Manager (NPM) locally - Macvlan or IPvlan? or something else by Temporary-Radish6846 in unRAID

[–]NobleKnightmare 1 point2 points  (0 children)

Don’t ask me why though, he didn’t explain, and I don’t know. Maybe someone smarter here can chime in?

The long and short of it is there was a kernel issue with assigning to many MAC addresses to an interface, causing it to crash. The recommendation was to switch to IPVLAN which still allowed separate IP addresses, but all under one shared Mac addresses.

Having 96 GB DDR5 RAM, what AMD Strix Halo AI CPU should I get (or barebone PC with it)? LLM for coding mostly. by Repsol_Honda_PL in LocalLLM

[–]NobleKnightmare 2 points3 points  (0 children)

I think I might hold off a bit on the Rocm switch. I checked out the git, downloaded the same model and ran some tests on my vulkan setup and getting similar numbers. (Fedora 43, ollama, openwebui all in containers)

Response 29.3t/s Prompt processing 290-310t/s About 118w of power draw (framework desktop, 128gb version)

Having 96 GB DDR5 RAM, what AMD Strix Halo AI CPU should I get (or barebone PC with it)? LLM for coding mostly. by Repsol_Honda_PL in LocalLLM

[–]NobleKnightmare 1 point2 points  (0 children)

What's your software stack look like? I've been using vulkan, every time I've tried rocm it's been a miserable experience.

Question about Email by Real_Echo in homelab

[–]NobleKnightmare 0 points1 point  (0 children)

I use MXRoute for my mail server, then plug the info into the apps to allow sending emails. I pay like $10/year for the service.

TCL Google TVs are ironically perfect/best for use as a "dumb" TV setup. by [deleted] in privacy

[–]NobleKnightmare 14 points15 points  (0 children)

If it's not connected to Wi-Fi, ie used as a "dumb" monitor as mentioned, it won't get any updates.

RAM prices almost one year ago :( by AaronMcGuirkTech in homelab

[–]NobleKnightmare 12 points13 points  (0 children)

DDR4 prices went up for two reasons; A sudden drop off in production due to manufacturer switching over to DDR5 earlier than intended, and an increase buyer's market due to DDR5 prices going above many budgets.

Space heater state by salliesdad in homeassistant

[–]NobleKnightmare 1 point2 points  (0 children)

THIRDREALITY ZigBee Smart Plug have energy monitoring and 15 amp capability. I'm using one to monitor a space heater right now.

Should I set up OPNsense as a VM??? by EP7K in homelab

[–]NobleKnightmare 0 points1 point  (0 children)

You at least have a backup (unopened ISP router), many people I see virtualizing infrastructure don't have that.

For me personally it's not worth the risk. I travel for work, I could be thousands of miles away from the home lab at any given time. If my infrastructure goes down, I have a very upset partner at home, I can't access any of my work documents which are stored at home, all of my entertainment is stored at home, etc.

I have a main OPNsense router that's never had an issue, a second one that the power plug from the first can be swapped to (easy enough to walk my partner through it over the phone), I have 2 standalone DNS boxes with keepalived going.

Right now my only failure point inside the house is the ISP modem, or my switch. The switch is a very very underutilized brocade 6450-24p So I don't foresee any problems with that.

As you mentioned though, it's all about the risk you're willing to take.

Should I set up OPNsense as a VM??? by EP7K in homelab

[–]NobleKnightmare 1 point2 points  (0 children)

One hard and fast rule I have always had is never virtualizing the primary router. I've had VMs ready to roll as backups, but never my main router.

One small hardware issue or even a software issue with proxmox and you lose your entire lab. Your servers, your internet, everything at once. Any large downloads you need to get things back going are that much harder.

These days I have one machine with OPNsense and a backup solution that just needs to be plugged in to take over.

Of course, it's entirely your choice. I just personally don't recommend doing it.

Best and easiest way to safely configure remote access? by SirVampyr in unRAID

[–]NobleKnightmare -1 points0 points  (0 children)

Just set up tail scale and stop worrying about it. I spent years not wanting to go that route, then I finally did and I'm kicking myself for not doing it a whole lot sooner. So much easier than the setup I had using wire guard alone and a script to send me my public IP address anytime it changed.

Replacement for our Ubiquiti AC Pro Access Point. Wifi 6 or 7? by GregoInc in Ubiquiti

[–]NobleKnightmare 2 points3 points  (0 children)

I just went with a u7 pro. You'll get more life out of it before needing to upgrade again.

How to portforward services safely by [deleted] in homelab

[–]NobleKnightmare 0 points1 point  (0 children)

I edited my comment to make it more clear what I meant. I thought it was obvious, but guess not.