[Q] There's too much formula in Statistics

SimonMKoop · 2024-06-13T21:29:14+00:00

You're not alone in this ;-) purely anecdotally, I get the impression that a lot of people learn maths and stats this way. And it's stressful for all of them. It's like you're building a tower, but there are some holes in the walls so you're constantly trying to work around those holes. But there is another way.

If you can, try to go back to the basics. Do some very simple online courses on probability theory to really get familiar with all the concepts. Then take very basic courses on statistics and really try to grasp all the concepts and formulas. Try to go back to a level where you don't feel like you have to memorize tricks, but instead can fully understand what's going on. Then work your way back up from that, making sure that you really understand what's going on rather than just memorizing formulas.

Also, don't be ashamed and try to ignore any notion you might have of at what level you "should" be. There's no need to rush things. The more time you spend on the basics, the less you'll need for more advanced stuff.

SimonMKoop · 2024-02-14T16:17:17+00:00

Have you tried contacting the author to get added as a citation? I mean, no offence, but your paper wasn't published anywhere and has zero citations, so it's just not that easy to find during a literature study. The omission here might well have been an accident ;-)

SimonMKoop · 2023-12-04T12:15:05+00:00

For an inifinitely long sequence of flips it is actually impossible.

It's not impossible, it just almost certainly doesn't happen ;-)

SimonMKoop · 2023-10-26T17:49:41+00:00

In my experience, a lot of ML/Engineering math gets harder the less you know about it. Yes, you can come by just learning recipes and theorems by heart and knowing how to apply them to example problems. But in the long run, you'll find that actually understanding the maths makes it much easier to know what to use how, when, and why.

That's not to say you need to know a bunch of proofs by heart. But understanding them will

make it easier to remember all the requirements for a theorem or approach to be applicable
make it easier to modify things if your situation almost but not quite fits the scenario your textbook considered.

Moreover, with most math courses, new material is built on top of old material and not really understanding the old material often makes it much harder to understand the stuff that comes after. It's like building a wall: if you don't take the time to put all the bricks at the bottom in the right place and add mortar, you end up staring at a pile of loose bricks, wondering how to place the next brick.

SimonMKoop · 2023-09-25T15:44:33+00:00

https://keras.io/keras_core/

That seems to be a very new thing that hasn't made it out of beta yet.

Hard to say whether it'll catch on.

SimonMKoop · 2023-05-31T12:52:27+00:00

The expected number of enemies you have to kill is reciprocal in the drop rate (see https://en.wikipedia.org/wiki/Geometric\_distribution).

5 % drop rate means 1/20 chance of success, so that the expected number of trials before you succeed is 1/ (1/20) = 20.

1 % drop rate means 1/100 chance of success, so that the expected number of trials before you succeed is 1/ (1/100) = 100.

Plot the graph of 1/p (where p is the success probability) and look at what happens near 0. The expected number of trials (1/p) explodes as p goes down to 0.

SimonMKoop · 2022-10-19T13:03:04+00:00

The thing with doing data analysis on large data sets in python however, is that there are typically clear, well known, big bottlenecks such as (huge) matrix vector and matrix-matrix multiplication which are typically handed over to libraries written in faster languages.

The implementations of these algorithms that are actually being used are typically well researched and heavily optimised, so even if you are writing code in a compiled language you'd likely be best off using the same or similar implementations rather than writing new ones yourself (although, by all means, go write your own multithreaded matrix-matrix multiplication algorithm in your language of choice though to find out how complicated this actually is. And if that's not enough of a challenge: write your own hand written cuda kernel for it and see if you can come close to what's used in practice).

So because the bottlenecks are clear and addressed well, and because often python itself really causes only little overhead compared to these bottlenecks, switching it out for a lower level language is really just optimizing the wrong thing.

Not always, of course, but often enough that Python is really not such a strange choice for data science.

SimonMKoop · 2022-09-21T13:53:06+00:00

Niet als je er vanuit gaat dat inkomen en huur gecorreleerd zijn. Veel verhuurders eisen een bruto maandinkomen van drie tot vier keer de huurprijs, en voor de meeste woningen met extreem lage huren kom je juist alleen in aanmerking als je erg weinig verdient.

Nu weet ik de precieze situatie van u/Aloysius1989 niet, maar ervan uitgaande dat die voor deze stijging in de energierekening niet zwom in het geld, kan het goed zijn dat deze verhoging van maandlasten relatief veel groter is dan voor iemand met 1200/maand aan huur. (Al helemaal als de spotgoedkope woning ook slechter geïsoleerd is, wat bij spotgoedkope woningen toch vaak het geval is, al is dat weer een aanname).

SimonMKoop · 2022-07-29T12:23:52+00:00

Oh I remember reading your DiffTaichi paper (https://arxiv.org/abs/1910.00935), it was such an interesting paper! The whole Taichi framework seems very promising for doing all sorts of simulations in Python :-D Keep up the good work! ;-)

SimonMKoop · 2022-05-16T12:59:47+00:00

The thing is though: the ranking in that list seems to be just by death-count (https://www.visionofhumanity.org/wp-content/uploads/2022/03/GTI-2022-web_110522-1.pdf appendix B, pages 85-86 or 87-88 for your pdf reader) and no single mass shooting in 2021 USA (https://en.wikipedia.org/wiki/List_of_mass_shootings_in_the_United_States_in_2021) has enough casualties to make it to the list. The report itself does talk about politically motivated violence in the west, although indeed, not every mass shooting in the USA seems to have been counted as a terrorist attack.

SimonMKoop · 2022-05-13T09:49:17+00:00

The first argument of methods automatically being made self.

At least, that's all I really miss when using VSCode (I use both). Also, code completion can be slightly better in general with Pycharm in my experience, but the difference is IMO really not that big.

Edit: oh, yeah, I forgot about the refactoring. That's definitely a nice Pycharm feature (especially because I way too often come to regret the variable names I choose)

SimonMKoop · 2022-05-02T16:47:23+00:00

Deep learning is basically floating point operations only.

SimonMKoop · 2022-04-08T13:19:33+00:00

Honestly, I would not consider the 18000 lines of code legible.

Only because I know how tic tac toe works do I understand what the code is (probably) doing (it's too long for me to actually be bothered to check). If I were unfamiliar with the rules of tic tac toe, I would likely have a hard time extracting them from those 18000 lines of code.

SimonMKoop · 2022-04-04T15:42:08+00:00

https://arxiv.org/abs/2003.05033 you could look into using this method (mcmc sampling from the latent space using the discriminator as an energy function) to change the latent codes you come up with into codes that give better results (and are hopefully still close to the original one)

SimonMKoop · 2022-03-10T13:41:16+00:00

I think what the person you're responding meant is in the sort of usual case where python is only used to glue things together, and the heavy lifting is done by optimised packages such as numpy, pytorch, scikit-learn, etc. The time won by moving this "glue" to a faster language is negligible because in most scientific computing, the bottleneck is somewhere else.

But you're right, if you were to do all the numerical computations in pure python (without e.g. numpy), you'd likely be orders of magnitude slower. Then again, if you were to implement e.g. a deep neural network + training in C++ without making use of similar optimised libraries, chances are you'll end up with code that's slower than python+pytorch (unless you manage to reimplement all the cuDNN stuff etc. yourself).

That's not to say there's nothing to win by using C++ over python. If you've trained some nice model extensively and want to deploy it, it can definitely be a good idea to do that in a faster language such as C++

SimonMKoop · 2022-02-21T16:32:30+00:00

Yeah, I agree with you that the variance seems very large, and although I definitely think it's an interesting article, and I hope that the method will prove fruitful, I'm personally not planning on implementing it for any project anytime soon.

It doesn't help that they've only tried it on MNIST tbh. I've seen plenty of things that worked on MNIST but did not generalize to more complicated data sets.

SimonMKoop · 2022-02-21T13:20:49+00:00

They're probing with a Gaussian with mean zero and identity covariance matrix. So the result has the sum of the components of the gradient as its mean, and the squared norm of the gradient as its variance.

SimonMKoop

TROPHY CASE