The ancient Babylonians knew of the quadratic formula to solve quadratic equations as long ago as 2000 BC, and it is today familiar to everyone who has taken a course in high school mathematics. It gives the roots of such a quadratic equation as a formula in terms of its coefficients, using only a finite number of additions, subtractions, multiplications, divisions, and the extraction of roots. In 1545 Girolamo Cardano published solutions to the cubic equation developed by himself, Scipione del Ferro and Niccolo Fontana Tartaglia as well as a solution to the quartic equation by one of his students Lodovico Ferrari, that also used only a finite number of those basic operations (the cubic formula and quartic formula are, however, quite complex compared to the quadratic). In the centuries that followed, mathematicians struggled in vain to find such a similar formula that would work for an equation of the fifth degree, until Joseph Louis Lagrange began work in 1770, unifying the various tricks used to solve polynomial equations known at the time, that hinted that this was not possible. Paolo Ruffini in 1799 then came up with a proof that purported to show that such a quintic formula did not exist, but his proof had missing parts and ultimately it was Niels Henrik Abel in papers published in 1824 and 1826 who closed all of Ruffini's gaps and once and for all settled the centuries-old question, and hence it is known as Abel's Impossibility Theorem. The original proof by Ruffini and Abel only worked for the quintic, and it was Evariste Galois who eventually came up with a method of proof that would generalize to equations of higher degree. The particular proof given here is thus based on Galois theory.
First of all, we need to define what it means for an equation to be solvable by radicals. We say that an extension K of a field F is an extension by radicals if there exist α1, ..., αr in K and positive integers n1, ... nr such that K = F(α1, ..., αr) and α1n1 is in F and αini is in F(α1, ..., αi-1). We say that a polynomial in F[x] is solvable by radicals over F if the splitting field of the polynomial is contained in an extension of F by radicals. This is the same as saying that a polynomial f(x) in F[x] is solvable by radicals if, starting from elements of F every zero of f(x) may be obtained by using a finite sequence of additions, subtractions, multiplications, divisions, and the extraction of nith roots.
An important theorem in Galois theory establishes the relationship between solvable groups and extensions by radicals. It may be shown that for any field of characteristic 0 a polynomial is solvable by radicals if and only if its splitting field has a solvable Galois group. Given this fact, the proof of Abel's theorem basically shows that for some equations of the fifth degree the splitting field won't generate a solvable Galois group.
Let y1 be a real number transcendental over the field of rational numbers Q, and y2 be transcendental over Q(y1) and so on until we get y5 transcendental over Q(y1, ..., y4). These are called independent transcendental elements over Q. Let E = Q(y1, ..., y5), and let f(x) = (x - y1)(x - y2)...(x - y5). Multiplying this out gives the elementary symmetric functions of yi:
s1 = y1 + y2 + ... + y5
s2 = y1y2 + y1y3 + ... y4y5
and so on up to
s5 = y1y2y3y4y5
The coefficient of xi in the expansion of f(x) is thus ±s5-i. Let F = Q(s1, s2, ..., s5) (i.e. the finite extension of Q in the coefficients of our polynomial). Then E is the splitting field in F of f(x). Since our independent transcendentals yi act as if they were indeterminates over Q, each permutation σ in S5, the symmetric group on five letters, induces an automorphism σ' of E that leaves Q fixed and permutes the elements yi. Since arbitrarily rearranging the order of the roots in the product form still produces the same polynomial, e.g.:
(x - y1)(x - y2)(x - y3)(x - y4)(x - y5)
is still the same polynomial as
(x - y4)(x - y2)(x - y1)(x - y5)(x - y3)
for instance, it also leaves F fixed, so each σ' is an automorphism in G(E/F). Now, since S5 has order 5!, |G(E/F)| ≥ 5!, since there could be some automorphisms in G(E/F) that are not permutations of the form σ'. However the splitting field of a quintic polynomial has at most 5! elements, it must be that |G(E/F)| = 5!, and all of the automorphisms σ' must make up the entire Galois group. Thus G(E/F) must be isomorphic to S5.
And what of S5? The only composition series of S5 is S5 ≥ A5 ≥ {e}, but the quotient group A5/{e} (isomorphic to A5 itself) is not abelian, so S5 is not solvable, hence, there exist quintic polynomials which are not solvable by radicals. Generalizing the argument, it may also be easily shown that the Galois group of an nth degree polynomial generated in a similar fashion is isomorphic to Sn, and it is well known that the alternating groups An for n ≥ 5 are all simple and non-abelian, and hence not solvable.
Note that this theorem doesn't say that every fifth degree polynomial is insoluble by radicals. For instance x5 - 1 is solvable by radicals because its splitting field K is generated over Q by a 5th primitive root of unity, making its Galois group abelian. As previously mentioned, under what conditions equations are thus solvable by radicals is one of the applications of Galois theory.
So in the absence of a quintic formula, how are the roots of general quintic equations obtained? There are many numerical methods that work with great effectiveness in finding the numerical roots of any polynomial, certainly, such as Laguerre's method or the Jenkins-Traub method, that are guaranteed to effectively converge on some root of a polynomial no matter where you start. This is the practical way of doing it, but for some applications, an analytical solution might be more enlightening.
Naturally, all of this work on Galois theory shows that it is necessary to go beyond the ordinary arithmetic operations and extractions of roots. There are more ways to kill a quintic than choke it with radicals, as Ian Stewart puts it. George Jerrard, who initially--and mistakenly--announced that he had found a solution to the general quintic in radicals, later wrote in papers about 1835 that a general quintic would be solvable using only the basic arithmetic operations, the extraction of roots, and something he called a "hyperradical", that he defined as the solution to the Bring-Jerrard form of the quintic x5 - x + p = 0, which he had shown any quintic may be reduced via judicious application of Tschirnhausen transformations.
Later mathematicians realized that Jerrard's hyperradical could actually be characterized by functions already known to mathematical analysis at the time. In 1858, Charles Hermite developed a solution to the Bring-Jerrard quintic in terms of the elliptic Jacobi theta functions and their associated elliptic modular functions, using an approach similar to the more familiar solution to the cubic equation involving trigonometric functions. Felix Klein developed in 1877 a particularly elegant method for solving quintics based on the symmetries of the icosahedron (gee, so the d20 much beloved of D&Ders does have some interesting properties!). His actual solution is the topic for another node, but basically he came up with it by noting that the group of rotations of the icosahedron can be shown to be isomorphic to A5 which group played such a starring role in the above proof. Klein's result wound up expressing the roots of a quintic in terms of generalized hypergeometric functions.
For equations of arbitrarily high degree, Henri Poincare among others further generalized Klein's approach and found that a special class of automorphic functions known as Siegel modular functions, natural generalizations of the elliptic modular functions that feature in Hermite's approach, can be used to express their roots. Perhaps these scary-sounding arcane functions are actually much more fundamental than anyone ever realized...
Sources:
John B. Fraleigh, A First Course in Abstract Algebra, 5th ed.
Felix Klein. Lectures on the Icosahedron and the Solution of Equations of the Fifth Degree.
Charles Hermite. Sur la Résolution de L'Equation Du Cinquème Degré, Comptes Rendus de l'Académie des Sciences t. XLVI, 1858 (I), pp. 508ff.
Ian Stewart, Galois Theory
"Solving the Quintic With Mathematica" poster. http://library.wolfram.com/examples/quintic/