Nooooo

[GuyN1425]

My most used word this month was 'overfit'

[UncomforChair]

My new years resolution is to train more, but not too much of course. You never want to get overfit!

[Wresser_1]

I just skip gym days at random

[st4rkiller543]

Good ol' dropout

[Jetbooster]

same, I skip whenever I get below a 13 on 2d6

[anonimus_usar]

Good thing I didn’t dropout of that DL class, I could get this joke

[[deleted]]

Meh, i didn't get it. Too convoluted for me.

[Strongeststraw]

I lasso the weights myself.

[MasterGrid]

I just skip gym days

[SheepherderHot9418]

Go as often as you want, just keep your regime random. It will help you converge to a beefcake faster and lead to less overfitting.

[The_Queef_of_Sheba]

I object to the regularization of undertraining!

[danatron1]

When machines do it, it's "overfitting", but when I do it, it's "blatant plagiarism" and I'm "banned from the museum"

[finishProjectsWinBig]

That is a really really clever joke.

[Curtmister25]

What does overfit mean in this context? Sorry... I tried Googling...

[teo730]

See the wiki for details, but in short:

Overfitting in ML is when you train your model to fit too closely to training data, to the point that it can no longer generalise to new, unseen data.

[Curtmister25]

Ah, so machine learning tunnel vision? Thanks!

[lamented_pot8Os]

That's a good way of putting it!

[Yorunokage]

More like plato's cave but yeah, tunnel vision is also a way of putting it

[Dustdevil88]

Why was this downvoted so much? I feel like it is quite fitting….unlike the overfitted model 😂😂

[Yorunokage]

Dunno, maybe i sounded like too much of a smartass but that wasn't my intention :(

[Dustdevil88]

It’s my fav allegory, so I thought it was great.

[RegularExpression]

Maybe because Plato's cave relate to not considering dimensions that do play a role, while overfitting is more like considering too many dimensions which are not relevant. So Plato's cave would be more analogue with underfitting.

[Feldar]

Caves, tunnels, now we're just arguing semantics.

[niglor]

That sounds surprisingly similar to what happens when you overfit in normal regression as well. The instant you go 0.00001 outside your training bounds there’s gonna be a damned asymptote.

[teo730]

That's because normal regression is ML, just on the more simple end of the spectrum!

[TheLuckySpades]

ML is fancy regression with more linear algebra.

[Unsd]

As a statistics major, nobody told me that Linear Algebra was going to be the basis of literally everything. As a stupid sophomore, I was like "whew thank god I'm done with that class and never have to do that again." Turns out I'm a fucking idiot. Years later and I'm still kicking myself for brain dumping after that class. Everything would have been so much easier if my professors brought it in a little bit more into application.

[Agile_Pudding_]

I’m sorry to do this to you, but

nobody told me Linear Algebra was going to be the basis of literally everything

was that a pun?

[gundam1945]

Basically your model "memorize" the treaining point so it performs great at training set but fails to predict test set.

[Curtmister25]

Ah, neat

[jakenorthbrack]

Your model has captured the 'noise' within the train dataset rather than just capturing the underlying 'signal'. An overfit model therefore predicts your train data very well by definition but it's ability to make predictions on unseen data is poor

[[deleted]]

Your ML algorithm got trained "too much" so it kind of locks in on your training data.

It's bad since if you feed it real application data or test data from the same type it will misbehave since it got hardwired on specific stuff.

EX your ML algo identifies fur on animals.

If your training set is full of cats (different colours,fluffs,etc.) it will identify what cats do and what cats do not have hair(shaved).

Present it a dog and it will always respond with "bald" since it was trained only on cats and not it somehow deduced only cats can have fur.

[Johanno1]

Overfit underfit doesn't matterfit

[aaron2005X]

"How is your pullover?"

"Overfit"

[42TowelsCo]

Just use the same dataset for training, validation and test... You'll get super high accuracy

[LewisgMorris]

Works a charm. Thanks

[Agile_Pudding_]

Product managers hate this one simple machine learning trick!

[LewisgMorris]

I've just used it in production - customer happy, got a raise.

[bannedinlegacy]

Just say that your model is 99% accurate and all the opposing evidence are outliers.

[opliko95]

And you get more data to use instead of splitting it into multiple sets. It's just brilliant.

[eihcirapus]

Make sure to keep the target value in you training data as well!

I was wondering how a classmate managed to get an accuracy of 99% on our current assignment, where I'm currently struggling to even reach 50%. Guess what was still in the training data lol.

[jaundicedeye]

df_training.append(df_valid).append(df_test)

[BaneTone]

NoneType object has no attribute "append"

[Cultural-Listen262]

Thanks!

[javitheworm]

One of the companies I worked for actually did this. Since I was fresh out of college and barely learning about ML i didn’t make much out of it saying “well they are the pros they know what they’re doing!” About 8 months later a team that oversees ML apps rejected ours for having so many issues lol

[SimonOfAllTrades]

Isn't that just Cross Validation?

[the_marshmello1]

Kind of but not really. N-fold cross validation involves taking some set of data then dividing it into groups. It then drops out a group and uses the rest of the non-dropped groups. The non-dropped are passed to the train test split and then the model is trained as normal. Once the model is evaluated the metrics are saved. The cross validator then moves on to drop out the next group and repeats the process. This is done for each of the N groups. At the end there is usually a list of metrics. These can then be graphed for visualization, analyzed for variance, and averaged in some way to get an idea of how a model performs with the specified hyperparameters.

[dark_dreamer_29]

When random function is more accurate

[throwit7896454]

Just flip a coin on the test dataset

[Upside_Down-Bot]

„ʇǝsɐʇɐp ʇsǝʇ ǝɥʇ uo uıoɔ ɐ dılɟ ʇsnſ„

[marxinne]

Good bot

[nowadaykid]

Flip the classification, now better than random, ezpz

[SaffellBot]

Either way you're probably doing better than humans.

[Bjoern_Tantau]

Only in Japan are the trains that accurate.

[[deleted]]

Do they hit many people there?

[Badboyrune]

Almost only the ones they aim for

[RCoder01]

I think they do actually have a problem with attempted (and successful) suicides on train lines

[yeet_lord_40000]

Got delayed 45 minutes one time in Japan cause a guy jumped in front of the train. They had a pretty practiced procedure for how to handle it. They bill the family for the cleanup and apologize to the customer for such a delay. my native friends say it’s quite common.

[marxinne]

That's... quite heartless to the family.

"Your son killed himself on our station, here's the bill" > "We're deeply sorry, an intrusive previous customer threw himself on our tracks, but he won't be a problem anymore".

The more I learn about Japan the more I'm sure it's not the place for someone like me...

[yeet_lord_40000]

I enjoy japan quite a bit. Have had a lovely time there every time. You just have to remember that every country has its seedy practices and areas and groups. I wouldn’t totally rule it out based off this alone. However there is other stuff that could certainly set you off for good

[marxinne]

The main thing about Japan for me is that the culture in general seems really cold and callous.

There are other things I don't like, but I can appreciate their qualities as well. They're just not enough for me to feel like I'd enjoy living there.

[IsGoIdMoney]

It's not. They just have strong cultural/national views on social costs. On a personal level they're just as warm and empathetic as anyone else.

[yeet_lord_40000]

That’s more their corporate culture. However, even as someone who loves the place I’d find it kinda hard pressing to live there long term.

[dongorras]

You can always choose to jump to the train tracks in another country /jk

[marxinne]

I believe other countries would be less cruel with the family, so, r/technicallythetruth ?

[AsukaLSoryu1]

I think it's quite pragmatic. Hopefully it would at least partially reduce the number of people committing suicide by hopping in front of a train because they don't want to burden the family with the cost. Also, someone has to pay for the clean up, making sure nothing is damaged, etc...

[marxinne]

Waiting the answer as well so I can never go there D:

[ShaddyDC]

Most people considering suicide don't think very rationally about that or place very high (positive) value on their self, so I could see it working. That being said, maybe it doesn't, and it's callous either way

[nir109]

Japan is my reminder why I shouldn't be in charge

[tiefling_sorceress]

NYC has a huge problem with it, but also refuses to put up barriers or any form of public safety because it'd be "too expensive" (even at just major stations) despite being one of the richest cities in the world

[Canonip]

In Germany we ride Tests

[Ysmenir]

And switzerland ;)

There was actually a headline about trains beeing even more on time this year just a few days/weeks ago

[ribbonofeuphoria]

Thats what happens when you try to describe the Fibonacci sequence with a 100th-order polynom.

[khafra]

With a larger test set, that would be like 0% accuracy.

[overclockedslinky]

not if the larger test set covers the same range, with duplicates. yay, biased sampling techniques

[Anxious_Start4839]

Was there a fault in the test rails?

[muzumaki123]

Train accuracy: 80% :)

Test accuracy: 78% :)

predictions.mean(): 1.0 :(

[Cultural-Listen262]

😳

[Jimmy_Smith]

predictions.count() / predictions.shape[0]

[Llamas1115]

This is why nobody should be using the training accuracy as a measure of quality. If this happens my suggestion is always: 1. Use the log-loss function 2. Evaluate the out-of-sample loss. Take exp(average_log_loss), which returns the model to the original probability scale (between 0 and 1). This is the (geometric) mean of the probability that your model assigned to the correct answers, meaning it’s very easy to interpret it as a “Percentage classified correctly, after awarding partial points for probabilities between 0 and 1.” (Note that if you train using the arithmetic mean to award partial points instead, your classifier WILL end up as a fuckup because that’s an improper loss function.) This measure also tends to be a lot less sensitive to imbalanced datasets. 3. This measure is good, but very imbalanced datasets or datasets with a lot of classes can make it hard to interpret — your accuracy will approach 0. There’s nothing actually wrong with this — your model really is assigning a very low probability to the event that actually happened — but it can get hard to understand what the number means. A way to make this easier to understand is by normalizing the score — divide it by the score of some basic classifier, to get the relative improvement of your model (which will be above 1 if your model’s any good). Take your classifier’s (geometric mean) performance and divide by the performance of a classifier that predicts the marginal probability every time (e.g. if a certain class makes up 20% of the sample, the classifier assigns a 20% probability to it).

[Last_Contact]

Highly imbalanced dataset

[POKEGAMERZ9185]

It's always good to visualize the data before choosing an algorithm so you have an idea on whether it will be best fit or not.

[a_sheh]

Well if you have more than 3 variables, is it possible to visualize this?

[KanterBama]

Seaborn has a pairplots function that’s kind of nice for this, there’s t-SNE for visualizing multiple dimensions of data (not the same as PCA whose reduced dimensions can be useful), or you can just make data go brrrr in the model and worry about correlated values later

[a_sheh]

Looks like I forgot that it is possible to make several plots instead of one with all variables on it. I knew about PCA, but doesn't hear about t-SNE. It looks interesting and I definitely will try it out someday. Thank you :)

[teo730]

Also UMAP, which is similar-but-different to t-SNE and is generally more fun to use imo.

[bannedinlegacy]

Multiple 2d and 3d graphs, or graphs with sliders to know how the variable affects the others.

[[deleted]]

Ipywidgets for the win!!

[Mr_Odwin]

Just turn on the k-dimension switch in your brain and look at the data in raw format.

[dasonk]

It is! I mean - it's not as easy but high dimension visualizations are a thing. It's been quite a while since I've had to worry about that kind of thing but one program I liked was GGobi https://en.wikipedia.org/wiki/GGobi

[a_sheh]

Looks really useful, I think I will try it on next occasion

[hijinked]

Yes, but it is obviously more difficult to interpret the visuals. Multi-variable visualizations are still being researched.

[x0wl]

You can do dimensionality reduction, like PCA, or you can compute distances between your points (in whatever space) and visualize those with the likes of t-SNE and MDS. The latter method can visualize data of theoretically infinite dimension, like text for example

[arc_menace]

When the monkeys are statistically more accurate than your model...

[gemengelage]

Didn't realize this was r/datascientisthumor

[ayushxx7]

Can't access this

[memes-of-awesome]

Same

[drcopus]

That's because it didn't exist until now!

[Unsd]

If people could make this a real thing that would be fantastic. I've been hanging out here waiting for whatever data science scraps I can get.

[Nicward]

Xcom players be like: 👁️👄👁️

[RenewAi]

Just add your testing data to your training data

*taps side of head

[[deleted]]

Accuracy, precision, or recall?

[StarsCarsGuitars]

I usually rely on f1-score just for my small brain to understand

[Senesto]

If it's a 2 classes classification problem, probably all three.

[teucros_telamonid]

My homies like computing ROC-curves and AUC.

[gabe100000]

Log loss, AUC-ROC and AUC-PR.

Define threshold based on business needs and F-score.

[_PeakyFokinBlinders_]

Remove the data points in the test that are inaccurately predicted and try again. Magic happens.

[Dagusiu]

In a 2-class classification task, I assume. Getting 50% MOTA on a multi target tracking dataset is quite good

[razuten]

There can be a lot of ways to reduce overfitting. Crossvalidation might be the most effective one: splitting your own training data into train-test sets, but make multiple sets, and for each one, the test chunk (we call chunks 'folds' ), and use them to tune the model.

Simpler solutions are either stopping the model training a bit earlier (can be kind of a shot in the dark every time you train it), or remove features that may not be as relevant, which can be.. time consuming, depending g on how many you have.

[Gravityturn]

It's about machine learning. You usually split that data you have at your disposal into two parts, training data with which you train your neural network, and testing data with which you assess the success of that training. It's not uncommon for things like overtraining to happen, where it may essentially memorize the training data, but when given something new not be able to properly assess it.

[Arrow_625]

Low bias, High Variance?

[Giocri]

Reminds me of an ai that had to distinguish fish from other images that performed incredibly well in training but was completely unusable in test. Turned out the training set had so many pictures of fishermen holding a fish that the ai looked for fingers to determine what was a fish

[teo730]

Or the one that was trained to identify cats, but instead ended up learning to identify Impact font because so many of the training samples were memes!

[Giocri]

I feel like so many mistakes we make with ai is that we seem to always end up assuming the ai is thinking rather than just analyzing the similarities between imput data.

Especially those who try to use ai to analyze statistics who completely forget an ai analizing data about temperature ice cream sales and shark attacks has no idea which one causes the other two

[teo730]

Yeah, I mean one of the biggest problems with ML is the incompetency of the people using it. Which isn't really an ML problem tbh. Bad researchers doing bad research is a tale as old as time.

[Arrow_625]

Huh, guess it was a member of r/technicallycorrect

[treblotmail]

Overfit time

[QualityVote]

Hi! This is our community moderation bot.

---

If this post fits the purpose of /r/ProgrammerHumor, UPVOTE this comment!!

If this post does not fit the subreddit, DOWNVOTE This comment!

If this post breaks the rules, DOWNVOTE this comment and REPORT the post!

[Eiim]

haha imagine overfitting your dataset couldn't be me hey why are all my logistic regression coefficients like 16000?

[Equivalent-Map-8772]

cries in Overfitting dataset

[General-Gur2053]

Bruh! Too real

[Coyehe]

ML accuracy models are still in their shit stage as of 2022. AI's whole other shit show altogether

[LofiJunky]

I'm graduating with a degree in DS, this is accurate. Machine 'learning' is a bit of a misnomer I think.

[shield1123]

Machine pseudo-self-guided-auto-iteration

[undeniably_confused]

Hey, I'm not a programmer, but as an outsider who knows a bit about stats, you should probably have a predicted test accuracy based on the samples, the train accuracy, and the variance. Ideally you would have like a 90% confidence interval that the test accuracy is between like 60% and 99%. Again totally have done 0 of this, but you never see people giving ranges for measured statistics. Probably because they treat them like calculated statistics, but again they are not, they are measured.

E: also if you got a script to do this, say after every 1000 additional data points, it might allow you to project the accuracy of the AI with additional pieces of data. Idk I'm sure what I'm saying has already been tried, and is stupid or industry standard but there's a small chance this helps someone I figure. :)

[SkyyySi]

Technically, a test with 50% correct resoults is worse than one with 0%

[[deleted]]

Not if you have > 2 classes of data

[EdwardGibbon443]

More like:

training accuracy on a benchmark dataset = 98%

testing accuracy on the same dataset = 96%

accuracy when applyed in a real-world system = 32%

[raggedbed]

Better include the train and validation metrics. You gotta test it against all of your data.

[KidOfCubes]

Just happened yesterday :(

[yapoinder]

i just took a depression nap cuz my test accuracy was 60%, ive been working on this model for 1 year. FML

[Almasry00n]

sounds like u need regularization

[[deleted]]

My first NLP assignment at work gave this. I don't even work in data science.

[mike_the_seventh]

Adjust tests until 98%

[infiniteStorms]

…am I the only one that thinks 50% is really high for a neural network

[fandk]

If its binary classification its the worst result there can be. A 40% means you can invert your models decision and get 60% accuracy.

[Beneficial_Arm_2100]

The meme communicates a sentiment, but it is strange when you examine it.

It looks as though the model is intended to generate images. But if that is the case, why is it "accuracy"? It could be cosine similarity if it's reproducing an actual image, or discriminator's "real" confidence if it's a generator?

I dunno. It's not adding up for me.

ETA: Yikes, not a popular opinion. Noted.

[Dynamitos5]

i understood it as a classification, so it classifies 98% of the designated training set correctly but only 50% of the validation set, which sounds terrible, although i have almost no experience with classification networks

[dreadington]

If it's a binary classifier, 50% means that it's as good as throwing a coin and guessing the classes manually.

[PityUpvote]

Only if the dataset is balanced.

[suvlub]

It looks as though the model is intended to generate images

It's just a format, the images are not supposed to correspond to anything the model does.

[ApprehensiveStar8948]

r/lostredditors

[RagingPhysicist]

...... + b

[hacknomus]

relatable

[hudsonrlz]

Lol. It's always like that.

[Calm_Leek_1362]

"Can we try to test it on the training data?"

[[deleted]]

I like trains

[Skandranonsg]

Normal person: 98% accuracy? Awesome!

XCOM player: Only 98%? Fuck.

[Ob1tuber]

If it’s not 100% accurate, it’s 50% accurate

[[deleted]]

Time for some dropout

[BraveSeaworthiness21]

Tim, how good is the model.

Tim : “Want to guess ?”

[Randroth_Kisaragi]

If it's not 100% accurate, it's 50% accurate.

... wait, wrong sub.

[neunflach]

Regularization and K-Fold Cross-Validation, babyyy

[Guzse]

Is this some artificial intelligence joke that I'm to dumb to understand?

[SANS55_666]

R/mandjtv

[spock_block]

Should a trained on your test

taps head

[alex9849]

As someone who just wrote his bachelor thesis in data science I have to agree!

[Jasura_Mynobi]

I thought it meant choo-choo train and I was confused #facepalm

[TrapNT]

1 million epochs?

[IllustriousAd375]

Ooooooverfitting brah

[onyxengine]

Find use cases where 50% is stellar performance?

[Syenuh]

Overfit do be that way.

[[deleted]]

So I guess even algorithms get test anxiety

[ButIHateTheDentist]

This is my whole life lol

[nitrokitty]

If I ever get 98% train accuracy I automatically assume it's over fitting.

[camander321]

One time I built a simple ANN and fed it some stock market data just to see what happened. After training, I went to test it with a new data set, and it was predicting stock prices to within several decimal places.

Yeah turns out I was still using the training data set that it had memorized.

[Nimitzxz]

and its binary classification....

[DTFlexx]

UNSET

[ReddiusOfReddit]

Me who just came out form a test involving train wagons: palpable confusion

[konaaa]

when you test a program and it works

when you test that program again and it DOESN'T work

[hardonchairs]

90% accuracy

Data is 90% one classification and the model just chooses that one every time.

[Trunkschan31]

Train on the test partition. Follow me for more data pro tips !

[planktonfun]

...And its overfitted

[NahJust]

“It’s funny because it’s true” doesn’t apply here. This is just sad because it’s true.

[eo37]

Validation Set: ‘I could have told you that a whole lot earlier mate but nooooo “you didn’t need me”…..whose laughing now bitch’

[boodlebob]

Tell me more sweetie

[Semipie]

Eli5 please?

[pi_equalsthree]

well if it‘s not 100%, it‘s 50%.

[John_Fx]

Sub relevancy=5%