inner product

Given a real vector space V, an inner product is a bilinear form V*V → R, commonly denoted as (x,y), which satisfies:

bilinearity

(x,y+z) = (x,y)+(x,z); (x,by) = b(x,y); (x+y,z) = (x,z)+(y,z); (ax,y) = a(x,y).

symmetry

(x,y)=(y,x).

nonnegativity

(x,x) ≥ 0, and (x,x)=0 only for x=0.

These properties are enough to prove the Cauchy-Schwarz inequality, from which it follows that ||x||² = (x,x) is a norm (Cauchy-Schwarz is required for the triangle inequality). Note, however, that not every norm is derived in this way!

For complex vector spaces, the analogue of an inner product is a hermitian product. Confusingly, physicists will use the bra-ket notation (invented for the Dirac formalism     of quantum mechanics), denoting it as <a|b>. The linear functionals y → <a|y> and x → <x|b> (for fixed a and b) are accordingly denoted <a| and |b>.

As a clarification of ariels' writeup above, one defines an inner product on a vector space over a finite field as an object that satisfies the bilinearity and symmetry properties. We run into some problems however with the nonnegativity condition which, over finite fields, we interpret as <a,a> = 0 if and only if a = 0.

While the inner products we use over finite fields do serve their purpose beautifully, we are going to investigate just how badly the nonnegativity property can fail over finite fields. We shall show using elementary techniques that the nonnegativity property fails for all symmetric bilinear forms (i.e. objects satisfying properties the bilinearity and symmetry properties stated above) on vector spaces of dimension greater than 1 over any field of characteristic 2.

Let us begin by noting that if V is a vector space of dimension 1 over the field F where F has characteristic 2, then V is isomorphic as a vector space to the field F itself. Taking V = F for convenience of notation, we define <a,b> := ab for a,b ∈ F. The reader can verify that this is a legitimate inner product as defined above (although Cauchy-Schwarz is kind of meaningless in this setting).

Now suppose F = GF(2) and V = F² with the "inner product" <(a₁, a₂),(b₁,b₂)> := a₁b₁ + a₂b₂. Then <(1,1),(1,1)> = 1 + 1 = 0 with the arithmetic taking place in GF(2). This is just an example of how nonnegativity fails, but we shall prove the following more general result:

Proposition. Let F be any field of characteristic 2 and let V be any vector space over F of dimension greater than 1 (which we write dim(V) > 1). Then, if <.,.> is a symmetric bilinear form on V, there exists an a ≠ 0 in V with <a,a> = 0.

Proof. Let a,b ∈ V. Note that <a+b,a+b> = <a,a> + 2<a,b> + <b,b> = <a,a> + <b,b> since 2x = 0 for all x ∈ F. Therefore the functional f(a) := <a,a> is linear.

Suppose <.,.> satisfies the nonnegativity of the definition of inner product. Then since V ≠ {0}, we have that there is a vector v in V with <v,v> ≠ 0. Thus f must in fact have rank 1. We now apply the Rank + Nullity theorem which in fact extends to vector spaces of any dimension to get that the nullity of f must be equal to dim(V) - 1 > 0 since dim(V) > 1. In other words there is a nonzero subspace of V which goes to 0 under f, i.e. the kernel of f is nontrivial. Thus there is a vector a ≠ 0 in V with <a,a> = f(a) = 0. QED.

And we're done! Another seemingly difficult problem settled using elementary techniques. In fact, I believe that the technique of proof here is more insightful that the actual result.

An inner product is a linear operator often used to test constituents of a vector subspace for orthogonality. The inner product, while applying to geometric, real, and complex vectors, and functions, still abides by four general rules. They are as listed by ariels:

Linearity 1: <x|y+z> = <x|y> + <x|z>

Linearity 2: <x|ky> = < kx|y> = k<x|y> for all k real

Symmetry: <x|y> = <y|x>

Non-negativity: <x|x> = ||x||||x|| >= 0 (where ||x|| denotes the norm of x which is zero iff x is a zero function or the zero vector)

The inner product as defined for geometric, real vectors is also known as the dot product or scalar product. For two geometric vector, real vectors, x and y, the inner product can be written: <x|y> = ||x||||y||cos(a) or = x1y1 + x2y2 + . . . + xnyn

In other words, the inner product or the vectors x and y is the product of the magnitudes of the vectors times the cosine of the non-reflexive (<=180 degrees) angle between them. Or the inner product of x and y is the sum of the products of each component of the vectors.

For real or complex n-tuples, the definition is changed slightly. For two real or complex n-tuples x and y, < x|y > = < x*|y > = ||x||*||y||cosa = x1*y1 + x2*y2 + . . . + xn*yn (here * denotes the complex conjugate not multiplication which is implied anyways.)

In other words, the inner product of two real or complex n-tuples is the product of the magnitude of the second n-tuple with the magnitude of the complex conjugate of the first n-tuple. Or the inner product of two real or complex n-tuples x and y is the sum of the products of the complex conjugates of the components of the first n-tuple and the product of the components of the second n-tuple. The definition is pretty much the same for n-tuples and geometric vectors except that in the complex case one n-tuple gets evaluated as its complex conjugate. Because of the symmetry principle, it doesn't matter which n-tuple, but by convention the first one is chosen. This need to use the complex conjugate for only one n-tuple arises from the need to preserve the non-negativity of the norm. If x is complex, ||x||||x|| = ||x||^2 may be negative.

Finally, for functions, the inner product is defined much differently. Note also that the definition is only valid for piecewise continuous functions.

For two piecewise continuous functions f and g on a closed interval a to b, = ∫fgdx (the limits of integration are from a to b).

The norm a function is also defined as ||f|| = >= 0. Hence we can define the inner product redundantly as = ||f||||g||cosa. It is difficult to find a geometric analogy for this definition but it should be stressed that the norm of a function does not represent its arc length and the angle a does not represent the angle between the functions.

The best use for the inner product is to define and test for orthogonality. For geometric vectors, orthogonality and perpendicularity may be considered the same thing but this definition does not hold for complex n-tuples or functions. A better definition is that two geometric vectors, real, or complex n-tuples, or functions are orthogonal if and only if their inner product is zero. This is extremely important as it makes the inner product the only condition for orthogonality, a very important and useful concept throughout linear algebra.