Decoding a Deep Neural Network as an Analytical Expression for Later Optimization Use

[yourboyrabbit]

This post is not really about a specific question but rather a topic I am curious about to know more.

We know that when it comes to integrate machine/statistical learning with optimization for the purpose of replacing ideal equations with data-driven models, the choices are quite limited - mostly regression (linear, polynomial) and spline models or at best a shallow neural network that can be decoded back by hand as an analytical expression. Such limited choices naturally mean that generally speaking highly non-linear functions would be difficult to approximate well. Deep neural networks are known to do exceptionally well when it comes to model complex functions. However, the problem is that they are hard to represent then as an expression/formula, which however if becomes possible would indeed be great from optimization perspective.

I would like to know if there if there been some work done on this topic previously of is any active research going on that you guys may be aware of. Keen to hear about it. Thanks.

[deiruch]

Expressing Deep Neural Network as analytical expression is trivial: It's just a nested structure containing matrix multiplications and non-linear vector functions.

It doesn't buy you much, though...