A point of inflection is, as Ryouga describes, a point on a graph where concavity changes (making it the second derivative equivalent of a local extremum). He's also correct that most points of inflection are zeroes of the second derivative. However, not every zero of the second derivative is an inflection point.
Take, for example, the function f(x)=x4. The first derivative is f'(x)=4x3, the second derivative is f''(x)=12x2. Finding the zeroes of the second derivative, 12x2=0, gives x=0. However, x=0 is not the location of an inflection point.
The second derivative, f''(x), is 12x2. The graph is concave up where the second derivative is positive - 12x2 is positive everywhere except at zero, so the graph is concave up on both sides of x=0. Thus, the concavity does not change, so it's a zero of the second derivative but it's not a point of inflection.
There are also inflection points where the second derivative is not zero. Take, for example, f(x)=x1/3. The first derivative is f'(x)=(1/3)x-2/3, the second derivative is f''(x)=(-2/9)x-5/3. The second derivative has no zeroes. However, for negative values of x, the graph is concave up, and for positive values of x, the graph is concave down. x=0 is the location of an inflection point (f''(0) is undefined). So, inflection points can not only occur where the second derivative is zero - they can also occur where the second derivative is undefined.
tdent informs me that you can tell whether a point which has f''(x)=0 is an infleciton point or not by looking at subsequent derivatives (third derivative, fourth derivative, etc.). If the next nonzero derivative (evaluated for that particular value of x) is an odd derivative (third, fifth, etc.), then the point is an inflection point - if the next nonzero derivative at that x is an even derivative (fourth, sixth, etc.), then it is not.