Numerical Methods

See also: Numerical Analysis

Numerical Methods are used to obtain an approximate solution to equations which are either computationally costly to compute or for which a closed form/analytical solution can not be found. These days, this means build a computer program to compute an approximate solution.

Numerical methods are used very commonly in the hard sciences, engineering, and any other place where the math gets hard (your fancy graphing calculator uses numerical methods when you do things like finding where some equation crosses the axis.)

Numerical methods can be found for any hard problem. Specifically there are numerical methods for integration (commonly called numerical quadrature), differentiation (commonly named finite difference methods), interpolation, and extrapolation.

Numerical methods are concerned with 3 attributes: Consistency, Stability, and Accuracy.

Consistency is basically the question of whether or not you are solving the right problem. If the numerical scheme gives the same answer as you would get in the case that you could solve the problem exactly, it is consistent with that equation. For partial differential equations for instance, consistency is verified by doing taylor expansions on the different terms in the scheme in question and showing that as the grid size approaches zero, the computational scheme approaches the partial differential equation.

Stability concerns the question of whether or not small errors in the calculations due to a variety of reasons will diminish or get larger and larger until the solution is completely obscured by them. For schemes where the solution is computed one time step at a time or one space step at a time, stability is a key factor as to whether or not a scheme can be used. Typically there is an "radius of convergence" or range of step sizes within which the scheme is stable but outside of which it is not. This is fairly intuitive: if you take very large time steps for a function that oscillates very rapidly for instance, it's unlikely to return the correct result.

Accuracy is how quickly a given scheme approaches the "correct" value. Two schemes may both be consistent and stable but one may be more accurate than the other, meaning that it will take fewer "steps" to bring the error down to the same level. Accuracy is often measured in terms of the 'step size' of the variables (accurate of the order of h³ for instance if the variable is h.)

Most numerical methods have their base in finite difference methods (the fundamentals of calculus have the same ideas as their base).