Numerical Analysis
The process of using a
black box program to automatically differentiate every line of a source code using repeated applications of the
derivative chain rule. The result is a transformed program that accomplishes the original computation and computes derivatives as well.
This is extremely useful when dealing with
legacy programs, as one can quickly ascertain a gradient vector or even a
Jacobian Matrix without resorting to
finite difference techniques, or the tedium of manual differentiation of a 60000 line program.
AD codes like
ADIFOR or ADOL-C operate in both
forward and
backward modes. The backward mode requires more memory, but is faster.