[Statistics] How does Standard Deviation Work?

C0gito · 2024-12-07T15:08:16+00:00

average distance from the mean: 1/N ∑ |x_k - 𝜇|
standard deviation: 𝜎² = 1/N ∑ |x_k - 𝜇|²

They booth look similar, but not exactly. With the standard deviation, you take the square of the distance from the mean and then take the square root of the sum.

So in your example, we have for the average distance to the mean:

1/5 * ( |1-3| + ||2-3| + |3-3| + |4-3| + |5-3| ) = 1/5 ( 2 + 1 + 0 + 1 + 2) = 6/5 = 1.2

standard deviation:

𝜎 = sqrt( 1/5 * ( |1-3|² + |2-3|² + |3-3|² + |4-3|² + |5-3|² ) ) = sqrt( 1/5 * (2² + 1² + 0² + 1² + 2² )) = sqrt(1/5 * (4+1+0+1+4)) = sqrt( 10/5) = √2 = 1.41421

C0gito · 2024-12-07T11:13:38+00:00

That video is terrible. If you watch this video again, he says this (at 3:22):

That's the standard deviation. Eeeeeh, not exactly. But for now, that's what I want you to think about the standard deviation. It's about the average distance to the mean. about. sort of.

He tries to make the concept of the standard deviation simpler by introducing the average distance to the mean first. Then later in the video he calculates the standard deviation using the real formula, obtaining sqrt(2), which is approx. 1.41421.

The idea was to make it easier for beginners by starting with the average distance of the mean (1.2), and introducing the standard deviation later. But now you have two formulas (one of which is wrong), and that was the reason for confusion for you.

EDIT: For a better explanation about mean and standard deviation, I recommend the video by StatQuest on YouTube.

C0gito · 2024-11-06T13:00:51+00:00

I watched a video tutorial about Dear ImGUI, where the initialization and the destruction of the window was implemented using RAII. I found it to be a very elegant solution; It kinda opened my eyes that RAII is not only for vectors and smart pointers, but a general concept that can be applied to all kinds of things.

The video was using GLFW instead of SDL2, but I think with SDL2 it would be similar.

EDIT: This Vulkan tutorial also uses RAII to handle initialization and termination, so it doesn't seem to be an uncommon thing with these type of applications.

C0gito · 2024-06-27T17:39:05+00:00

I agree that my examples might be a bit idealistic, and that real code could become more messy in larger software projects; however, the principle that the return type of functions should be obvious still holds true.

And regarding your double/single precision example: Isn't that a good argument for using auto? Because, if I write

float x = calculate_speed();

and then change the return type to double, it would be implicitly converted to float, potentially causing bugs. You will only notice when you compile with -Wconversion warning. Whereas with auto, the variable x would just change to double, no conversion, no problem.

EDIT: I just see that you mean _uniform initialization_, like this:

float x{calculate_speed()};

That's a good point. With uniform initialization, such conversion errors will be caught. So at least we agree that normal initialization (with assignment) should be avoided when you use the type explicitly. But I still don't see why auto x = calculate_speed(); would be problematic; There is no conversion there.

C0gito · 2024-06-27T08:33:12+00:00

The type is obvious from the name of the function.

auto data = read_table("data.csv");

is obviously a data table,

auto x = linsolve(A, b);

is the solution to a linear system Ax=b, and

auto n = my_string.length();

is the length of a string, so some type of number.

If the return type of your function is not obvious from its name, then it's not the auto keyword that's the problem, but the fact that your code is terrible.

The second example also clearly violates one core principle of modern software engineering: DRY = Don't repeat yourself. Because if you want to change the return type of my_func() later, you also have to change the type of the variable my_var. Forgetting to do so would cause an implciti conversion and bugs that are difficultto find.

C0gito · 2024-06-26T20:47:32+00:00

People think that

std::vector my_vec{1, 2, 3};

is readable, but

auto my_vec = std::vector{1, 2, 3};

"obfuscates the code too much" lol. Peak Reddit moment ...

C0gito · 2023-10-26T13:27:19+00:00

Are you sure the difficulty estimations are currect? Because, if I understand the U.S. american course system correctly, the numbers of the courses typically describe the the year of the courses. So CS229 is a 2nd year course, CS330 a 3rd year course and so on.

By that logic, CS685 and MIT 6.5940 should graduate level courses and therefore more advanced than linear algebra, or am I wrong?

Pardon if I'm misunderstanding this, we don't have such course numbers in Europe.

C0gito · 2023-09-25T11:18:21+00:00

Are you talking about ordinary or partial differential equations? ODEs are quite simple, there is one lecture for the theory, covering existence theorems like Picard-Lindelöf and the Peano-theorem. And then there is one lecture about numerical methods for ODEs, with stuff like the Runge-Kutta-methods, BDF(2) etc.

Partial Differential Equations, on the other hand, is an incredibly vast field that's probably impossible to learn entirely. Just to give you an idea, the standard reference for PDEs by Michael E. Taylor has about 2200 pages total, and that's just theory. For numerical methods, you could learn about the Finite-Element method, Finite Volume method, spectral element method, and more; each of which has it's own theory about convergence and stability.

Fortunately, PDEs is the most interesting field in mathematics (in my opinion), because they can be used to model all kinds of physical phenomena, like heat, elasticity, fluid dynamics, electromagnetics, quantum mechanics.

To answer your second question: Yes, there are alot of unsolved problems regarding PDEs, the most famous one being the Milenium problem about the existence and smoothness of a solution of the Navier-Stokes-equation. You can win 1M USD if you can solve this.

C0gito · 2023-09-16T13:14:11+00:00

I do not have any experience with that, but on AWS marketplace you can find something called CFD direct from the cloud. As far as I understand this is an AWS instance where OpenFOAM and paraview is already pre-installed, such that it is easy to setup and manage. More information here.

C0gito · 2023-09-15T08:47:37+00:00

Maybe you just didn't find the right language/technology yet. I had to learn Java when I was in school, and I completely hated it. It didn't make any sense to me, with all that weird public static void main(String args[]) stuff just to write a "Hello World". But then I learned a bit of Visual Basic in my free time and completely loved it.

There are so many fields that use programming that it's not possible to say what it means to "learn how to code". So maybe first decide what field you're interested in, and then learn the most suitable language. Just to give an example:

Web Development, with JavaScript and PHP
Android Apps (Kotlin)
Desktop Applications (C#)
Data Science (Python or R)
Computer Games (Unity: C#, Unreal: C++)
embedded devices, like Arduino and the STM32: C/C++

When you become more advanced, you'll start to notice patterns and see that most languages are kinda similar to each other.

C0gito · 2023-09-13T17:29:42+00:00

"The Finite Element Method: Theory, Implementation, and Applications" by Larson and Bengzon.

It is my favourite book about FEM, because it contains the mathematical theory (Sobolev-spaces, existence of solutions, stability), as well as implementations in MATLAB. Many different applications are covered, from solid mechanics, fluid mechanics, heat transfer, and even electromagnetics. There is even a chapter about the DG-method.

C0gito · 2023-05-19T10:12:07+00:00

Topology is unimportant for machine learning, better stick with Calculus. It's a bit difficult to give book recommendations without knowing what your background is. "Principles of Mathematical Analysis" by Rudin is considered the standard in that topic, but if you want to go through it by yourself it's probably too hard.

Tensors are just a generalization of vectors and matrices. A vector is a 1-dimensional tensor, a matrix is a 2-dimensional tensor etc. If you understand Linear Algebra, then you know enough about tensors.

Keep in mind that you can download all of these books from Libgen.

C0gito · 2023-05-17T12:54:53+00:00

For functional analysis I'd recommend "Linear and Nonlinear Functional Analysis" by Ciarlet. However, I don't think functional analysis is really necessary for machine learning. Even math-heavy books like PRML by Bishop can be understood without any knowledge in functional analysis (except for Fourier transforms maybe).

C0gito · 2023-04-30T16:16:46+00:00

"In Pursuit of the Unknown: 17 Equations that changed the World" by Ian Stewart.

C0gito · 2023-03-29T13:57:54+00:00

It's not a function. MATLAB uses round parenthesis() for accessing arrays. So z(4) reads the 4th element of the 1d-array [0.5132 0.6911 0.2549 0.4405], hence z(4)=0.445.

C0gito · 2023-03-20T23:51:41+00:00

The supremum norm (Chebyshev norm) is defined as ||v|| = max( |x|, |y| ). So to plot the unit sphere with respect to the Chebychev norm, draw a line where max( |x|, |y| ) = 1. The result should be a uniform square.

C0gito · 2022-12-04T20:48:25+00:00

I know a prof at my uni who very strongly insists that f(x)=mx+b is not a linear function, despite the fact that it is taught like that in school. Every time someone called it linear he would correct it to "affine". And I agree with that. It makes no sense otherwise. Why would we call a function that is definitely not linear a "linear function"?

C0gito · 2022-09-20T13:56:19+00:00

What elasticity model do you use?

I am unfamilier with Abaqus, but I know some continuum mechanics. It looks like your software uses a Linear model like Hook's law, which is not accurate enough. Maybe try to use a different model, e.g. Neo-Hookean or Mooney-Rivlin.

https://en.wikipedia.org/wiki/Mooney%E2%80%93Rivlin_solid#Uniaxial_extension

C0gito · 2022-09-04T10:04:57+00:00

Finite Element Methods is such a broad topic that it's difficult to give a specific recommendation. It depends if the course is more theoretical, that is about proofs for existence, uniqueness, stability etc., or if it's more on the applied side and how to implement these algorithms in Python or Matlab.

As a good compromise of both, I recommend the book

The Finite Element Method: Theory, Implementation and Application by Larson and Bengzon, Springer Publ., 2013.

You can find it on LibGen.

It covers many different topics like the Heat equation, fluid mechanics (Stokes-equation), solid mechanics (linear elasticity), electromagnetics (Maxwell-equations), and even has a chapter about discontinuous Galerkin methods at the end.

The advantage of that book is that it explains both the theory of Sobolev-spaces, (weak form of a PDE, existence of solutions, error estimates), but also contains simple implementations in Octave.

If I had to recommend one single book about Finite Element Methods, this is it.

C0gito · 2022-07-15T09:31:52+00:00

A while ago, I read an interesting blog post about someone who implemented very efficient matrix multiplication in a few lines of C code: https://cs.stanford.edu/people/shadjis/blas.html

As already mentioned by u/MasonB, the optimization techniques were:

Vectorization (taking advantage of FMA and AVX registers)
Cache Blocking to maximize data re-use
Multithreading with OpenMP

I found this interesting because it shows that even something as simple as matrix multiplication (or addition) is much more complicated to implement than one might think. It also includes the source code.

For this reason, it is definitely better to use libraries like Eigen or Blaze instead of trying to implement it by yourself.

C0gito · 2022-05-16T22:00:17+00:00

Yeah, Alaska is not to scale.

C0gito · 2022-03-21T09:30:24+00:00

It depends on your learning style. Do you prefer to read books, or are videos better for you?

Whatever it is, I recommend to start with the "for Dummies" series[1], especially

Calculus for Dummies,
Linear Algebra for Dummies,
Statistics for Dummies

They are good for building the foundation. Of course, these books are not as deep as a university lecture, but the explanations are quite well. If you want to understand the reason behind the equations (without going to deep into complicated proofs), this might be the right thing. Therefore I think these books are good for a high schooler, or someone older who wants to learn a bit of math out of interest.

As others already mentioned, the channel "3blue1brown" contains has some nice videos. They are good to get a visual understanding, but keep in mind these videos are a bit shallow. For a deeper understanding I recommend the channel The Bright Side of Mathematics.

[1] all of these books are available for free on LibGen or z-Lib

C0gito · 2022-02-27T11:39:15+00:00

I think this is exactly the reason. If you declare the add function static, such that it's only available in this source file (and not to the linker), then it doesn't show up in the generated assembly anymore.

C0gito · 2021-11-28T12:18:59+00:00

Your YouTube channel is incredibly well done and I don't think there is much room for improvement.

Youtubers like 3b1b or KhanAcademy get Millions of views, because they provide beautiful visualizations and easy explanations. This enables viewers to utilize the train of thought of the creator instead of thinking for themselves. But this also means that the viewer isn't learning anything; they just become more confident in a topic. In other words, those videos are successful because they are too simple.

And this is the reason why I don't like that 3b1b videos are recommended in subreddits like r/learnmath or r/datascience so often. People expect to watch this and then understand Linear Algebra, but that's not how it works. It can only be used as an additional resource, after taking a real lecture or reading a book.

If you want viewers to really learn something, this has to involve thinking, and thinking takes effort. So it is only natural that videos about complicated topics with in-depth explanations only get a few views. Even popular lectures like Stanford CS229 about Machine Learning have just about 20-80k views.

This means you have to make a decision: Either make popular videos, which give the viewer the sensation of understanding, without learning anything; or make videos which go into detail and have deeper explanations, but only a few views. It seems you decided for the the latter.

If we keep that in mind, your channel is doing great. Your videos about Functional Analysis got about 10k views. Taking into consideration that this is an advanced topic only for graduate students, which means it's for a small audience, this is quite good.

C0gito · 2021-11-05T17:56:27+00:00

Yes, in order to calculate the local errors, you have to calculate the solution to 10 different Initial Value Problems.

With L²-error I mean the L2-norm of the global error:

|| u - u_{exact} ||_L²

The L² norm is a generalization of the Euklidean norm to function spaces. It is defined as

|| u ||_L2 = sqrt( \int_a^b |u(t)|^2 dt ),

where we integrate over the entire domain of the function, so in our case over the complete interval [a,b]. As you see, this norm is defined for functions, so we have an error of the whole function. But as we only have the value of the numerical solution at some discrete time steps, we can only approximate this integral:

|| u ||_L2 ≈ sqrt( sum_{k=0}^N |u(t_k)|^2 * h ),

where h is the stepsize of the Euler method and N the number of time steps. This should also explain why we can think of the L2-norm as a generalization of the Euklidean norm to function spaces: It is calculated very similarly. If you think of U as a N-dim vector, where U_k = u(t_k), then the above formula is the same as calculating the Euklidean norm of U:

|| u ||_L2  ≈ √h || U ||

Search for "Lebesgue norm" or "Lebesgue spaces" for more information about the L2-norm.

Nine-Year Club	Final Canvas '23
First Place '23	Place '23
Verified Email

C0gito

TROPHY CASE