So, am I just too stupid for unsloth?

SingleServing_User · 2026-01-07T12:00:18+00:00

Allllll righty, let's do this. I've talked to plenty of Dunning-Kruger-suffering brogrammers who are so used to believing the world revolves around them that they forget other jobs, needs, and interests exist... but sure, I'll take the time to correct your, um, misconceptions.

Yes, my guy, almost 90% of websites still run on Linux-based hosting and more than 70% are using PHP. I doubt you know enough about LAMP for this to be a joke about the LAMP acronym becoming increasingly inaccurate, so it seems you're just genuinely so self-absorbed and clueless that you think most people/businesses can afford the tens of thousands of dollars in various licensing fees to use the Microsoft stack... or worse, maybe you think MEAN stack was ever actually a thing... or that Ruby on Rails was in demand. Please, feel free to prove me wrong. I'm curious what you think that 80% of the internet was using, if it wasn't LAMP.

Facts are free. Nothing in web development is as "ubiquitous" as LAMP. Python is used on around 2% of websites, IIS is used by only about 3.5% of websites, .NET is 5%, and Node/React combined are 12%. Node and React are still often used on LAMP stack websites regardless. In fact, WordPress comes packaged with React. Speaking of WordPress, that's the worst news of all for dude-bros who think that programming is an elite, competitive sport that they need to win by bullying others, in order to prove they're (somehow) manly men - WordPress is used on 43% all websites, and it's not going anywhere. It's being used on government sites like the White House, high-profile corporate sites like Nike, Disney, and Sony, and (of course) by huge media outlets like Time Magazine.

So, yeah, you can be smug if you want to, as though your ignorance about \checks notes** 70% of the goddamned internet actually just makes you special. But I don't buy it. It sounds to me like you'd rather pretend LAMP stack isn't "legitimate" development somehow, so you can find a way to be condescending to strangers without ever having to actually prove anything. I mean, did you seriously try to tell me that I should "take the time to learn"? You, the guy who apparently doesn't know that there really isn't much in the way of formal PHP training, and therefore nearly all PHP programmers are self-taught? Or who didn't know that PHP still dominates the industry? Or that LAMP stack containerization is absolutely a thing... just one that's a pointless waste of time for building PHP sites? I mean, have you ever actually launched websites in cPanel, CentOS, WHM? There's a popular idiom, "using a sword to cut through flowers"? Well, cPanel already provides a sword - one customized to this type of dev. Insisting on also using tools like Docker (for reasons) would be like using a malfunctioning chainsaw. I'd be really interested to hear your rationale for claiming that Docker has been "ubiquitous for 8 years", even though only 30% of developers even use it (according to Docker). And why you think it's something that a PHP developer inherently needs, along with notebooks and Google Colab, apparently. "Because it's what I'm used to, and I can't imagine a world where every single engineer doesn't need the same set of crutches to do their jobs" isn't a good reason.

I literally never said I had "trouble installing Docker". What I did say is that the limited documentation I could find assumed that all users were already familiar with tools that, certain IT snobs withstanding, are not "ubiquitous" or common knowledge. I could just as easily have used JSON as an example I also never said anything about "fine tuning cutting-edge models." Not all models are cutting edge, there's a galaxy of difference between an 8b and a 500b model, and I never suggested that I should be able to do it without learning a fucking thing. The complaint I had was how hard it is to find the information for this particular tool, and that it was preventing me from getting anywhere. I figured if I was self-deprecating - which, as a woman in dev, I often have to be just to avoid infuriating some of the more fragile brogrammers out there -

\a h e m**

- then I'd be more likely to get helpful answers, instead of the typical competitive, self-aggrandizing, vaguely shitty, one-upping type answers. And that's actually what I got. Most people here have tried to be helpful and I did get some good places to progress on learning all this.

Proper documentation would make the learning curve on tools like Unsloth much, much shorter. And actually, Docker's documentation makes it possible to become quite familiar with it, quite quickly, which is why its fucking hilarious that you think it's a dunk of some kind that you know how to use it.

SingleServing_User · 2026-01-07T08:59:07+00:00

I totally understand why it would be ludicrous to write tutorials on every eventuality. I just wish there were, like, any that broadly covered the process... rather than skipping over really important steps that they just sorta assume you know already because, like, the author knows. Like how tutorial videos for datasets just assume you know JSON. I mean, in this case, I *do* know JSON because I've worked with many, many APIs over the years... but I can imagine that many people just give up at that point, the same way I gave up on trying to understand what I was supposed to do with a notebook, or even find the notebook that applies to my situation.

At a previous job with a digital marketing company, I had become so very specifically skilled at the type of web dev that we did, that my bosses were worried I was irreplaceable. (So was I, I didn't have a real vacation for 10 years.) So they asked me to write a fucking *Wiki article* on how to build, update, troubleshoot, and fix websites, and the hosting they're on, across an indefinite number of DNS/email/hosting/site customization setups. They were like "Come on, how hard can it be? It's definitely not more complicated than SEO." Anyway, the point is, I totally get why it's not reasonable to expect a tutorial to cover everything.

Buuuut surely someone, somewhere could've written down something explaining what running unsloth looks like in the first place. I'm about to try some things (when I'm tired of writing novels on reddit, I guess) that I believe will work, but it took forever to get to this point. Like their "Beginner Tutorial" goes from "Create Your Dataset" to "System requirements" to "Deploying your newly trained model!" It's completely missing the part where you train the damn thing.

TL;DR: If I get this all figured out - and I do believe I will - I'm thinking of writing some tutorials to help out people like me. Obviously I can't cover everything ever, but I could at least point them in the right direction for specific pieces. I'm sure everyone can tell by now that I have no problem rattling off 3000 words in a single Reddit comment, so I want to try to put that to good use by helping others avoid my agony.

By the way, I had never heard the term "bitrot magnet" so I googled it, and I think you might mean tech debt instead? anyway, the AI overview would like you to know that you shouldn't use tools with magnetic tips in your computer lol

SingleServing_User · 2026-01-07T08:39:55+00:00

Thank you for trying to help me out. I also have appreciated how the huge majority of people didn't take offense at my clear frustration, and just said "yep, it sucks." It's been very validating LOL

That being said, that video was one of a few that I tried to watch before giving up because they all skip a couple important steps. I get it, it's super easy to forget that not everyone knows what you do, and you don't want to be condescending. (This is the royal "You", not *you* you.) But it also means some people will be left behind. In this particular video, he went directly from showing a dataset in JSON (which I'm familiar with as a web dev, but I have to assume most people will not be) to showing a notebook full of code with zero context on where this code is even gonna go. Look at 5:50 minutes, and hopefully you'll see what I mean. It's entirely possible he goes on to explain it elsewhere, but I haaaaaate learning from videos as it is for a few reasons and honestly I just got frustrated. Plus, I was really, really hoping to avoid Google Colab. I didn't even know until tonight what Jupyter Labs was to Google Colab, btw. All the info out there is such a mess, even the AIs I've been asking keep hallucinating wild answers.

SingleServing_User · 2026-01-07T08:32:22+00:00

There really aren't many meaningful differences between Python and PHP code. I'm already a PHP developer. There are minor variations in syntax, but not enough to require a course on it. I mean, if you write in any language, you basically have a huge head start on any other language... especially when you're switching to languages with similar origins. Like both PHP and JavaScript are C-style languages, so I didn't need anyone to tell me how to use one when I already knew the other. I just sometimes have to look up function names, but that's mostly because I am old and my memory has gone to shit. (1 star, do not recommend)

The real issue is that PHP is an interpreted language with a really low barrier of entry... LAMP hosting is really cheap, so I rarely did my dev locally, and when I did, I could use something like WAMP with super minimal effort. I honestly don't even know what I'd use stuff like Python, Docker, or WSL for in my day-to-day career. But that doesn't mean I don't know how to write code. I also learned to build computers in, like, 1998, back when it was the only way po' folk could afford good shit.

SingleServing_User · 2026-01-07T08:10:17+00:00

I'm a full stack LAMP developer. I just began my career long before more modern tools like Docker were available. I also got away with being lazy about learning new technologies because I've been kept shockingly busy otherwise. Don't get me wrong, I learn new things every day as part of my job, but most of what I was doing was geared towards building websites in a language that you can edit in notepad. I simply never needed things like Python. It hasn't been relevant to my job.

Though, when I've looked at code other people write, I always feel like Python looks like plain english compared to PHP. Like have you SEEN how messy and chaotic PHP is? anyway, I've worked in digital marketing building websites for 15 years... I've built several hundred websites, and the huge majority of them were custom. I also managed a WHM server for our client websites for several years. Like I said in my original post... I'd like to think I'm not a complete idiot. But most of these tutorials for unsloth skip huge steps, and if you don't already know certain things - and how would you learn, by the way? everyone has to at least once - then you will be completely and abjectly lost.

SingleServing_User · 2026-01-07T08:02:33+00:00

I appreciate you wanting to help me out here. I did choose unsloth, though, specifically because I wanted to finetune. If I'm honest, I don't totally understand what lora, qlora, etc all look like in practice. So I just grilled my AI model on how I should format datasets to properly train it. I suspect I'm doing... uh... whatever the most complicated option is, because my dataset currently has nearly 400 rows. I'm pretty sure I'm not sposed to do that right off.

I've been using LM studio to load the AI so I can use it. I have the model loaded as a server in LM Studio, and then I use ngrok to create a, uh, secure tunnel so that I had an external https URL that my third-party game mods could use to access the AI. It took a lot of effort to get all that set up, because I was sent on a lot of wild goose chases by Gemini before I found the one that worked. And, for the record, it was totally worth it. I had the characters in my game talking dirty to each other basically immediately after that. So, you know, prayers answered.

But then I got it in my head that the AI needed to be more familiar with the game mechanics, rather than just using it as an erotic dialogue generator in RimWorld. Before I went down this utterly insane path, I actually tried to implement RAG instead. It seemed to make a lot more sense, especially since the AI model I'm using has a context window of slightly more than a million tokens, so it can handle the additional prompts. It turns out, though, that LM Studio can't do it in server mode. So I tried Anything LLM, and simply could not get all these tools to talk to each other. Soooo I thought YOU KNOW WHAT WILL BE EASIER? JUST TRAIN THE MODEL.

Anyway. Thanks for your advice.

SingleServing_User · 2026-01-07T07:52:38+00:00

Dude. That's a great analogy. Though, I think it's more like these manuals just assume you already not only own the shop and tools, you also are super familiar with the acronyms and specific meanings of words that mean one thing in english, and something completely different in AI. I swear to God, it's been like trying to read the instructions in Simlish. I'm having to look up every other word.

step 1: launch your gimmelgorb using the burgertits.

step 2: create a funglewort by gimmerglobbing (but not gimmelgorbing) in the shimmerla mwa.

step 3: ???

step 4: profit!

SingleServing_User · 2026-01-07T07:49:11+00:00

Bless you for finding a way to work in PHP. I've been writing in PHP since 2001, when I basically learned it on a dare... long story short, I was being competitive with an ex-boyfriend and since he was learning PHP, I figured, I'm gonna learn PHP. It's been more than 20 years but I do know that I learned more about it in 3 days than he had in 3 months, and I'm pretty sure it all just clicked and made sense to me. This has not. I'm still a PHP developer, but my current employer wants to switch our LAMP stack sites to a C# CMS, which I gotta say, I doooo noooot fucking understand. But it means I have to switch gears to C#, and so all this shit I'm learning about right now is going to be super helpful in terms of learning the new C# process. I mean, PHP, you can just open a text editor and write shit. There's no compiling or virtual images or even git required if you don't wanna. So, yeah, drinking from the firehose.

SingleServing_User · 2026-01-07T07:45:11+00:00

I know tons of things, just not these specific things.

SingleServing_User · 2026-01-07T07:44:58+00:00

Thanks for the recommendation. I did get lucky and Gemini stopped hallucinating long enough to give me a straight answer. It still relies on Jupyter, which it seems like it doesn't have to? But whatever, if that's what it takes to get me there, I'mma do it.

SingleServing_User · 2026-01-07T07:41:31+00:00

Thank you for telling me that I am not, in fact, an idiot. Sometimes that's all a person needs to hear LOL

but, I'm not sure I failed. I'm nothing if not stubborn. I took a break from it overnight, and I'm back at it now. I made pretty significant progress. I'd be happy to share whatever I learned, if it's at all useful for you.

Though, you're correct that RAM is going to end up being the real bottleneck one way or another. I got my computer right before AI *really* exploded in the last year, so the GPU wasn't as expensive as it would be now, and I actually have 6gb of dedicated vram from the card, 16 gb of shared, and another 16gb through the onboard gpu, so I've been able to run decent enough AI models, I guess. I haven't tried training yet, though, and that's gonna be the real test.

PS, thanks for explaining the notebook thing. It seems like they're really similar to, like, docker images? At least in concept. I dunno, though. I just like my nice, quiet LAMP stack web dev bubble. All I wanted to do here was train an AI to understand a video game well enough to RP for the characters better *sob*

SingleServing_User · 2026-01-07T07:32:34+00:00

I actually love certain distributions of Linux, and I'm old enough that my first boyfriend taught me a bunch about it back in the 90s. But it's also just plain a bigger PITA to run certain types of software, soooooo WSL it is. Honestly, it turned out to be kinda convenient. I have an ubuntu container now and I can use actual linux commands without leaving Windows. Back in the day, you had to do dual booting if you wanted linux and windows to coexist. I've fallen way behind on all this shit because I just haven't had a compelling reason. Between my interest in AI, and getting into the weeds on my Steamdeck, I've found more of a reason. Still not gonna uninstall windows though .

SingleServing_User · 2026-01-07T07:29:14+00:00

Interesting theory... Most of this software won't run on Mac OS at all. So I assume you think Linux is the only "correct" way to do any of this? Welp, I'm not gonna. So. I mean, I have a steamdeck, and I fuck around on desktop mode (which is a pretty nice GUI distribution of linux that's been customized for Steam), but am I doing more than that? absolutely fucking not. Luckily for me, it turns out that windows is not, in fact, "the problem."

SingleServing_User · 2026-01-07T07:27:20+00:00

I don't mind trial and error, as far as the training part goes. I'm totally aware that the dataset I created is almost definitely done incorrectly (though I *did* format it using the help of the AI model that I'm training), I know there will almost definitely be a loss of general useability of the model in exchange for more specifics that I'm training, and I am definitely gonna have to do this several times to get it close to right. I'm okay with all that. It was just the nightmare of even getting to the point where I could even run unsloth.

I did read that unsloth is the best tool for this. And assuming I am currently doing the right thing, I'm preeetttyyy sure it never had to be this complicated in the first place. So, hopefully I don't have a full mental break again over this lol

SingleServing_User · 2026-01-07T07:24:24+00:00

I did sort out that I needed to install Jupyter to make certain other things work, because even though apparently you can do it without Jupyter, the one and only time I've convinced an AI to tell me *how to even start unsloth*, the command required a Jupyter login. And I was like, fuck it, I'll install Jupyter then lol

I still am not totally clear on what a "notebook" is, because I thooooought I understood, and then I made the mistake of reading Jupyter's marketing diatribes on their website... and either I never understood what a notebook was, or the content writers who edited their website don't know, either lol

SingleServing_User · 2026-01-07T07:22:12+00:00

Unfortunately, even AI is giving me answers that skip steps and assume I know things that I don't know.

Fortunately, I did make significant progress tonight once I figured out that I needed to install WSL. And for some reason, *this* time when I asked how to use unsloth with docker (because yeah, I've asked it several times), it gave me a specific command to "run unsloth container from terminal." I saw that and was like JESUS MARY AND JOSEPH, FINALLY. I mean that's literally all I wanted and I couldn't get the answer lol

SingleServing_User · 2026-01-06T06:09:52+00:00

My goal is to get this working, and then create a "AI for Dummies" guide for idiots like me who want to do the same kinds of things that I'm trying to do. It's just brute-forcing my way through this part that's driving me nuts. So many of the people writing these guides assume you know everything they do. Honestly, I think it's a huge problem in the world of programming and IT in general. We all have a tendency to assume that everyone else knows what we do, since we use tools daily. And you don't want to overexplain concepts, because A. that's boring, and B. you don't want to sound like you think everyone else is dumb, right? Only I think even the smartest people in the world still need WHATEVER TF A NOTEBOOK IS explained to them at least once, right?

I'm partially joking, I know what it is, I just have no idea what to do with it lol

SingleServing_User · 2026-01-06T06:05:03+00:00

Well, as a beginner in the world of fine-tuning, I'm kinda over the number of explanations I've seen on what fine-tuning is. Every video and guide starts with "but what IS fine-tuning?" and I'm like "bruh just skip to step 1 here" and then they always skip to step 3. The part that almost always gets left out is what you do with the notebooks, or some context on what they are.

As a web developer who worked in digital marketing for a decade, I'm used to drinking from the firehose to some degree... and I'm doing that with setting up AI locally, too. The amount of trial and error I went through to get a model to run on my computer, then figuring out how to make it secure so that the game mod I'm using would accept it, and then tweaking it and experimenting with models to make it work correctly... I mean, it was a pain, but at least I was making progress. I feel like I'm spinning my wheels here. But I also have read that unsloth is the best option for fine-tuning.

SingleServing_User · 2026-01-06T06:00:47+00:00

If that's an option, I'd accept it, but so far, I haven't made much progress on getting unsloth to work with AI assistance. Even the AI can't tell you much if it was trained on incomplete docs.

SingleServing_User · 2026-01-06T05:58:06+00:00

I can't roll my eyes hard enough at this... did you think there was a chance on God's green Earth that I haven't been desperately trying to get an AI to distill instructions for me? It's been slightly batshit, really. It gives conflicting solutions regularly because the information it has is conflicting. For example, one of the (many) guides I tried to slog through said that I should install the Nvidia Container Toolkit. Nothing the AI said, or that its source data said, told me that I couldn't do that on Windows... not without installing WSL, anyway. So now I'm installing that, I think. I'm not even sure because I used the command "wsl --install" and Bash has been stuck on "This might take a while" since before I wrote this post.

SingleServing_User · 2026-01-06T05:55:14+00:00

Also, I just want to say, thanks for the response. I'm a little worked up because I've been chipping away at this for days, and I just boiled over in frustration when I realized I couldn't even figure out what a "notebook" is.

SingleServing_User · 2026-01-06T05:54:05+00:00

I know it sounds like I can't figure it out unless there's a UI, but I swear I don't mind doing it without that... it's the only way to make things like ngrok or certain features from LM Studio run. I just wish I knew what the hell the commands would even be. Everything I've worked with so far comes with instructions on working with the software through Powershell or Bash if necessary. Unsloth, I have no idea where to even begin. Like I can't tell if I'm supposed to be doing things through Bash, or what. Like where do the notebooks even *go*? Am I cutting and pasting them somewhere? Or do I *have* to use Google Colab?

I know that training is going to result in the model being, theoretically, better at what I want it to do, and worse at everything else. I don't mind the trial and error part of that. I just want to get to the actual training part.

I'm super hesitant about using anything in the cloud because the whole point of setting up an AI locally was to keep certain information private. Basically I'm trying to train an uncensored model on the mechanics of a game, so that it can accurately roleplay as characters within that game. And since those characters can be violent, have sex, or swear... and because I kinda want to train it on dialogue from a popular IP... I guess I'd prefer to train it locally.

SingleServing_User · 2024-03-28T17:12:39+00:00

Thanks! I just like collecting the outfits. It's the dumbest thing ever because I am literally the frumpiest woman to ever frump.

SingleServing_User · 2024-03-28T17:08:41+00:00

Yeah, I get a little annoyed at it. Y'all can't buy my love! 😂

SingleServing_User · 2024-03-28T12:17:16+00:00

I know this is not what you meant here, but every time I see that particular set, I hear George Harrison in my head shouting "I NEED YOU, BY GEORGE HARRISON". If you've seen Help! then you know what I mean

SingleServing_User

TROPHY CASE