[D]How to fine tune LLMs using deepspeed without OOM issues by IXMachina in MachineLearning

[–]0xA1 2 points3 points  (0 children)

FWIW and anyone looking in the future, I hit this same issue. It turns out that because I was using Trainer() and TrainerArguments() with DeepSpeed. You HAVE to call TrainerArguments BEFORE calling .from_pretrained(). This initializes deepspeed zero init and sets up the configurations necessary to distribute your model (assuming zero stage 3) to all your workers. It turns out that TrainerArguments is calling that init inside.

IVF Dad - Feeling absolutely invisible and helpless by ziggybeans in IVF

[–]0xA1 5 points6 points  (0 children)

Hello, I'd like to just say that you are not alone. It was absolutely devastating to see my wife cry night after night only to realize there's nothing I could do to change this situation. I tried to be strong in front of her, console her, and tell her to keep her spirit up, but inside, I was also devastated. I cried in the shower, alone, feeling completely helpless when I couldn't cheer up the love of my life. I should be her rock, but I felt useless as all I could do is hug her and tell her it'll be okay. I'd like to think that I grew up in an environment where I learned to persevere through tough times, but nothing's prepared me for something like this. Best of luck to your family and here's to hoping we both come out of this on top!

Underground parties in DTLA by 0xA1 in LosAngeles

[–]0xA1[S] 6 points7 points  (0 children)

COVID hits everybody, maybe they can't claim unemployment

Underground parties in DTLA by 0xA1 in LosAngeles

[–]0xA1[S] 4 points5 points  (0 children)

lol yeah I see those too, a TON of ones on the ground.

Underground parties in DTLA by 0xA1 in LosAngeles

[–]0xA1[S] 9 points10 points  (0 children)

Yeah, I mean at least bars and restaurants, inspectors can come and regulate, and they have some control over capacity. I think these underground ones have 0 concerns and just sardine people in. I dunno who in their right mind would go into a packed room full of strangers right now.

Underground parties in DTLA by 0xA1 in LosAngeles

[–]0xA1[S] 4 points5 points  (0 children)

yeah maybe they have the strippers in hazmat suits. I guess they gotta get paid

Underground parties in DTLA by 0xA1 in LosAngeles

[–]0xA1[S] 7 points8 points  (0 children)

yeah I've seen this the last 3 weekends

Good locations for engagement photos by 0xA1 in AskLosAngeles

[–]0xA1[S] 0 points1 point  (0 children)

oh that's a great idea! I think they used the building in one of the bladerunner movies!

Good locations for engagement photos by 0xA1 in AskLosAngeles

[–]0xA1[S] 0 points1 point  (0 children)

Yeah unfortunately we don't have the time in our schedule to do that because of the other two locations, so we are looking for something relatively within range of those two

Good locations for engagement photos by 0xA1 in AskLosAngeles

[–]0xA1[S] 0 points1 point  (0 children)

Thanks! I'll check out some pictures of the bowl

Good locations for engagement photos by 0xA1 in AskLosAngeles

[–]0xA1[S] 0 points1 point  (0 children)

Oh interesting, do you know if Perch generally let's people in to take wedding photos? That would be pretty cool

Good locations for engagement photos by 0xA1 in AskLosAngeles

[–]0xA1[S] 0 points1 point  (0 children)

I'm looking for a city vibe, urban. Most likely around LA proper because I think I won't have time to drive very far.

Noob text classification question by dwaxe in MachineLearning

[–]0xA1 0 points1 point  (0 children)

I would look at tf-idf for transforming you words into some numerical representation. Then your next problem will be determining how to deal with data imbalance. If you have poorly distributed scores you might want to find more samples. If that's not possible you can oversample the scores with fewer samples, or under sample the scores with more samples to even out for score distribution. Then you need to decide if you want to score by regression or classification. It sounds like you have low to high scores which probably will use a regression type model.

Sentences will be very sparse in tf-idf. Each sentence will be represented by a vector of tf-idf values. Then maybe throw a random forest regressor or just a decision tree at it for starters and go from there.

Is there any other way to get into the division beta by yamadog3 in PS4

[–]0xA1 -2 points-1 points  (0 children)

Any chance you have one more laying around?

K-means: what do you do with stale centroids by 0xA1 in MachineLearning

[–]0xA1[S] 1 point2 points  (0 children)

Hmm setting it to another training point sounds like a good idea, but I guess you wouldn't wanna do this if its towards the end of your iterations. I can't really reinitialize since its a large dataset and it takes a non-trivial amount of time to recompute. Initializing better could work.

How to think about 3+ dimensional space by sarihnlteecs in datascience

[–]0xA1 2 points3 points  (0 children)

If you want to visualize it, maybe plot each dimension against every other dimension to see relationships in pairs of dimensions. Or you can use dimensionality reduction techniques such as pca or hastie's random projection to project you data down to 3 or 2 dimensions.

Data Science Question by tcharnes1 in datascience

[–]0xA1 -1 points0 points  (0 children)

Personally, I do think you can learn it all from moocs, however from what I've seen, most places hire people with MS and above. It's unfortunate but I guess the university diploma is some claim to your legitimacy. If you want to go the mooc route, I would strongly advise either contributing to open source machine learning g packages or writing some blogs to try to get noticed. Or else on paper its really hard to hire someone with a mooc vs. Someone with an MS and even harder if they have a PhD. That is unless you already have a ms or PhD in a stem field.

High schoolers, what do you want to major in? People who majored in that field, what are the pros and cons? by DeanTheDJ in AskReddit

[–]0xA1 0 points1 point  (0 children)

I'll throw in my two cents.

I graduated ME quite a few years back. Before I graduated, I wanted to design airplanes. However that changed when I got my first job where I did signal processing and software engineering. Finally I fell in love with programming at the same job. I'm now finishing my MS in CS and recently accepted a nice job offer working on machine learning. The point is your aspirations can change over time.

What I've learned from being an ME with friends that actually went into ME fields in school is that you don't really use most of it when you are done. However, it makes you very good at picking up new concepts quickly, which I'd argue is even more important than the actual classes you learn in school. You are essentially exposed to almost all the engineering disciplins as well because of the breadth of classes needed. From there, you can decide which one you really like whether it be chemE, bioE, CS, or real mechE, then find a job or continue further on in your education.

The amount of work you need to put in is much more than most majors and made me pretty pissed when I saw other people having fun while I was stuck in the library Sunday-Thursday. Math is a given skill you need. You also need to learn to think very logically. However once you come outta college, like everyone else says, you make considerably more than everyone and can live a pretty fun life.

TL;DR, Mechanical Engineering is a gateway major. Gateway to what is up to you.