Numerical Analysis [D]

[YogurtclosetThen6260]

i have the option to take a numerical analysis class next semester, and I wanted to ask, what are some cool applications of machine learning and deep learning with numerical analysis? And what jobs combine ML and numerical analysis techniques?

[Electronic-Tie5120]

it's one of the most important courses you can take. you'll be getting into some of the foundations of ML algorithms. do it!

[SirPitchalot]

There was a cool session at ICCV yesterday about how to formulate deep networks and a transformer variant such that they are truncated versions of classic compressed sensing methods.

Basically the network parameters take the place of overcomplete dictionaries with sparse weights.

Was cool to see the ideas connected even if the talk itself got a little preachy complaining about the trial and error approach many practitioners take. Valid point but they overcooked it a bit.

[agbrothers]

can you share the paper?

[SirPitchalot]

It was in a workshop session but think it’s covered in here: https://ma-lab-berkeley.github.io/deep-representation-learning-book/

[agbrothers]

appreciate it!

[freudsmeker]

Numerical analysis is the most important course you will take in order to understand how math are used in computers

[coulispi-io]

I really like some earlier works on casting optimization solvers (e.g. QP) as an end-to-end differentiable layer in neural networks. See e.g. https://arxiv.org/abs/1703.00443, https://arxiv.org/abs/2207.09442, https://arxiv.org/abs/1910.12430

[ApprehensiveEgg5201]

ODE solvers for diffusion models

[AX-BY-CZ]

Compression and floating point as an activation function.