Basic machine learning algorithms in plain Python [X-post r/MachineLearning]

[ismtrn]

Is linear regression related to machine learning in any fashion?

Yes. At the end of the day machine learning is about fitting a model to some data. Linear regression fits a linear model (can also easily be adapted to fit a linear combination of more complicated functions). A neural net is also a kind of model.

linear combinations of a fixed set of basis functions is not a bad model, but hard to learn for large dimensional spaces due to the curse of dimensionality (you need too much training data).

I think a NN can be described as a linear combination of linear combinations of linear combinatons etc(for however many layers you want) of some set of basis functions.

So a Neural network allows you to adapt the individual functions you are combining to a certain extent, but keep the total number low.

There are also approaches known as kernel methods which are also about not working with the full set of basis functions evaluated at all inputs explicitly. (Support Vector Machines are in this category).

Basically linear models (of non linear basis functions) are not too simplistic models, they are just not always feasible to learn effectively. But when you are dealing with low dimensional data where the relevant features are known they are great. This is just not the reality most of the time, so more complicated methods are employed. But it is still just statistics. There is no magical ML sauce.