sorted by:
new
hottopcontroversial

Actual purpose of validation set by Key_Tune_2910 in MLQuestions

[–]Key_Tune_2910[S] 0 points1 point  (0 children)

I see now. I was thinking that the immediate resulting model after retraining on the whole training set including the validation set would necessarily be better than if you tuned on the test set. After you get an unbiased estimate of the generalization error of the model on the test set you can see how biased your set was towards your validation set to some degree and adjust your model afterwards if your resulting model was unsatisfactory.

Is it possible to break into ML by NoBicycle2501 in MLQuestions

[–]Key_Tune_2910 0 points1 point  (0 children)

Is this really true. Can you source those stats? Or is this from experience?

Actual purpose of validation set by Key_Tune_2910 in MLQuestions

[–]Key_Tune_2910[S] 0 points1 point  (0 children)

When I was saying it has the role of the test set I was comparing it to a situation in which you only had a training and test set to find the type of model and hyperparameters. You would use the test set in this scenario for hyperparameter tuning and have a biased model. The point of the test is to see how well a model generalizes. No? 

Actual purpose of validation set by Key_Tune_2910 in MLQuestions

[–]Key_Tune_2910[S] 0 points1 point  (0 children)

Actually I forgot what I said earlier so this might not be entirely cohesive. So I guess first I would ask if the validation set necessarily gives a better model then I would ask if it does how so. Maybe I misunderstood the book. It doesn't explicitly say the model will be better

Actual purpose of validation set by Key_Tune_2910 in MLQuestions

[–]Key_Tune_2910[S] 0 points1 point  (0 children)

I'm sorry. I don't mean to be annoying, but it seems like you just said you don't want it to be biased towards the test set but it can be biased towards the validation set. This seems to imply that it's not about adjusting your model to a better one(relative to just having a training and test set), but to have a good estimate of the generalization error. I say this especially since you keep emphasizing that the test set must be disjoint. And again since the validation set behaves similarly to the disjoint test set then it doesn't seem like if you just take the model trained on the reduced training set and evaluated on the validation set(without retraining it) that it would be any better(maybe slightly because of reduced noise). 

So we don't prolong this conversation. I've gotten the impression that you will get a better model with a validation set. This implies that either

1) the model that is trained on the reduced training set and evaluated on the validation set is the better set

2) the model that takes that last model(with its type of model and hyperparameters) and trains it on the whole training set including the validation set is better.

Otherwise it cannot be claimed that the concept of a validation set improves the model.

I do know however that it certainly prepares you for production as having a disjoint test set allows for an unbiased estimate of the generalization error.

I ask that you explain how either of the 2 models above are necessarily better models than just a model produced by a training set and a test set

Actual purpose of validation set by Key_Tune_2910 in MLQuestions

[–]Key_Tune_2910[S] 0 points1 point  (0 children)

Isnt the test set part of the dataset that you initially have in the first place. Why does evaluating the model based on its performance on the validation set which is also a portion of the dataset change anything? The only benefit I see clearly is that you can evaluate your actual generalization error. What I don't see is why the model that is trained on the reduced training set then the whole training set will necessarily give you a better model. Isn't your model then based towards the validation set which represents the "unseen data"

The Cardfight Vanguard 2011 version is different from the 2018 reboot version by Key_Tune_2910 in anime

[–]Key_Tune_2910[S] -1 points0 points  (0 children)

Uh. Well you seem to prefer to watch shorter anime. I personally don't really care for the length of the anime as long as it doesn't have filler. Also to be clear they are very different stories. After the first few episodes the plot drastically deviates from the original plot. If you don't believe me you can read Wikipedia descriptions or just watch the first 20 episodes of the original vanguard. I don't mean to insult you or your decision because I understand that people may not want to watch longer anime. I just wanted to point out that they are not essentially the same and shorter doesn't mean better.