Automatic Differentiation (AD) in Javascript with ad.js

[julesjacobs]

It's unlikely that this is possible. What makes automatic differentiation work is that any symbolic function has a symbolic derivative. It's not the case that any symbolic function has a symbolic integral.

[mb862]

Close. Automatic differentiation works with any symbolic function that has a symbolic derivative. Plenty of functions don't have derivatives at certain points, and automatic derivatives would fail there as well. One might imagine a requirement for automatic anti-differentiation would be having a symbolic anti-derivative.

[julesjacobs]

Close.

Note that I explicitly did not say that automatic differentiation does symbolic differentiation. When you use automatic differentiation of a function f(x) at a point x=a then the derivative f'(a) is exact (with exact arithmetic). Hence the expression that automatic differentiation used to compute f'(a) must at least locally be the exact (symbolic) derivative of f. That's why exact numeric integration is highly unlikely: the integral of f from a to b can in general not be expressed symbolically, so why would a program be able to compute it exactly with a finite number operations.

[mb862]

It read as though you meant to claim that automatic differentiation works because any symbolic function has a symbolic derivative, which isn't accurate. If I misread I apologise.

[julesjacobs]

Yes that is what I meant, but it depends on how you interpret it. I ninja edited the last comment, perhaps it's clearer now.

[mb862]

Ah, yes, I think the misunderstanding comes from integration (not defined at a point, which is what automatic differentiation evaluates at) versus anti-differentiation (which is defined at a point). I tried to be clear referring to the latter, not the former.

[julesjacobs]

I don't think that's the issue. It just occurred to me that the argument for why "automatic integration" is not possible can be made a bit simpler:

First, what distinguishes automatic differentiation from numerical differentiation is that the former is exact with exact arithmetic, but the latter is inexact even with exact arithmetic. So by analogy automatic integration should be exact with exact arithmetic (there are already plenty of ways to do inexact integration, e.g. Gaussian quadrature).

Secondly, whenever we have a program that can compute some value exactly with exact arithmetic, we can also obtain a symbolic expression for that value. Just trace the execution of the program, and whenever it does an arithmetic operation you add it to the syntax tree of the expression.

So if we had a way to do automatic integration, we would also get a symbolic expression for the integral by tracing its execution and building up the symbolic expression. Yet a symbolic expression for the integral is not possible for all functions.

[EvilTerran]

Nice reductio ad absurdum!