The
IEEE 754 binary floating point arithmetic standard states that the default value
1 for an invalid
operation is a NaN or a Not-a-Number.
SIDEBAR:
IEEE 754 also states that if the result of a computation is too large to be represented in the precision being used then it is to be replaced by an infinity and that the sign of the infinity (i.e. +infinity or -infinity) will be the same as the sign of the "correct" result).
One useful way of thinking of these infinity values is that they preserve as much information about the final result as possible (i.e. that it was very large and negative or that it was very large and positive).
For example, both e1000000 and 10200 * 10200 result in +infinity as both results are positive yet too large to be represented in either single (32-bit) or double (64-bit) IEEE floating point.
On the other hand, 10200 * -10200 would yield -infinity.
Quiet NaNs and Signaling NaNs
There are two types of NaNs - quiet NaNs and signaling NaNs
(the abbreviations NaNQ and NaNS are often used for the terms quiet NaN and signaling NaN respectively).
Each of the following operations will result in a quiet NaN:
- any operation having one or more quiet NaNs (or signaling NaNs if the IEEE 754 INVALID exception is disabled) as input values unless the operation has a single defined value if all of the input NaNs are replaced by any possible finite or infinite value2
- real square root of a negative number
- 0.0 * infinity
- 0.0 / 0.0
- infinity / infinity
- REMAINDER(anything,0.0)
- REMAINDER(infinity,anything)
- infinity - infinity (when the signs of the two infinities are the same)
or infinity + infinity (when the signs of the two infinities are different)
With the exception of the first case, each of these operations results in a runtime exception if the IEEE 754 INVALID exception is enabled (i.e. the program dies with an ugly geek-readable error message).
Operations involving quiet NaNs don't result in exceptions - i.e. they're
quiet.
The key result of these rules is that once you get a NaN during a computation, the NaN has a strong tendency to propagate itself throughout the rest of the computation (as essentially any usage of the NaN results in a new NaN).
It can't possibly be emphasized too much that this is a good thing even though it may very well be a very annoying thing!
The reason that this is a good thing is that every NaN that you get in your final output represents a value which could not be computed (i.e. if a non-NaN had appeared instead of the NaN then you might have been fooled into believing that the non-NaN was a correct value).
IMPORTANT: a division by zero does NOT result in a NaN unless the numerator is zero or a NaN.
Dividing a positive value or +infinity by zero results in +infinity and dividing a negative value or -infinity by zero results in -infinity3.
This is, without a doubt, the most common misconception regarding IEEE 754 NaNs.
Note that multiplying a NaN by zero does NOT result in zero but, according to the first rule, results in a NaN.
SIDEBAR:
Some processors, like the IBM POWER4, store values into memory faster if the memory location is already in the L1 cache when the store operation occurs.
Consequently, some software developers have been known to use the following sequence to clear an array quickly:
DO I = 1, N
A(I) = A(I) * 0.0
ENDDO
This is a really bad idea as any NaNs in the array before the loop will still be NaNs after the loop as the multiplication of a NaN by 0.0 yields a NaN!
A signaling NaN is never generated by a floating point operation.
Consequently, the only way to get a signaling NaN in a program is to either be unlucky (or lucky, depending on your point of view) regarding the
value of an uninitialized
variable or by explicitly initializing a variable's value to be a signaling NaN.
If a signaling NaN is used in a floating point operation and the IEEE 754 INVALID exception is enabled then the operation generates an INVALID exception.
Comparing NaNs
NaNs have one other very important
characteristic: every NaN is neither equal to, less than or greater than any other conceivable value (including the NaN itself, an infinity or any other NaN).
For example, if X is a NaN and Y is any conceivable value then the following comparisons are all false:
X equals Y
X > Y
X >= Y
X < Y
X <= Y
On the other hand, if
X is a NaN and
Y is any conceivable value then the following comparison is true:
X not equals Y
Two possible comparisons are particularily interesting:
Y equals Y
Y not equals Y
The first of these is false if
Y is a NaN and true otherwise
and the second is true if
Y is a NaN and false otherwise.
In other words, comparing a value with itself is one way to determine if the value is a NaN.
Note that comparing a value with itself in order to determine if it is a NaN is an unsafe programming practice as an optimizer which doesn't properly take into account IEEE 754 NaNs might eliminate the comparison (since "any value is equal to itself so why bother actually doing the comparison?").
Fortunately, most programming language implementations on systems which support IEEE 754 floating point have a library routine which can be used to determine if a value is a NaN (the C library routine is called isnan).
Performance considerations
Enabling the IEEE 754 INVALID exception (or any of the other IEEE 754 exceptions) can have a
DRAMATICALLY negative impact on a program's performance.
Consequently, the IEEE 754 standard specifies that all exceptions are to be disabled by default.
If IEEE NaNs and infinities didn't exist then it would be quite difficult to debug a program which does a significant amount of floating point arithmetic without enabling exceptions.
Since this could have a VERY negative impact on performance, enabling exceptions for a program which runs for many hours when exceptions are disabled might be totally impractical.
On the other hand, the fact that NaNs (and infinities) have a strong tendency to propagate themselves from where they are created through to where the output is generated makes it considerably easier to debug large floating point intensive programs by simply checking for the presence of NaNs at certain key points.
For example, something as simple as summing all of the elements in a matrix will tell you if any of the elements are NaN or infinities (you'll get a NaN if any of the elements are NaNs or if you've got a mixture of positive and negative infinities, and you'll get an infinity if you've got one or more infinities of the same sign (there is a small chance that you'll get an infinity if you've got some VERY large values in the matrix although this is almost certainly a bug anyways)).
Representations of NaNs
The 32-bit IEEE 754 representations of these values are:
Positive infinity: 0x7f800000
Negative infinity: 0xff800000
Signaling NaN: any bit pattern between
0x7f800001 and 0x7fbfffff
or any bit pattern between
0xff800001 and 0xffbfffff
Quiet NaN: any bit pattern between
0x7fc00000 and 0x7fffffff
or any bit pattern between
0xffc00000 and 0xffffffff
The 64-bit IEEE 754 representations of these values are:
Positive infinity: 0x7ff0000000000000
Negative infinity: 0xfff0000000000000
Signaling NaN: any bit pattern between
0x7ff0000000000001 and 0x7ff7ffffffffffff
or any bit pattern between
0xfff0000000000001 and 0xfff7ffffffffffff
Quiet NaN: any bit pattern between
0x7ff8000000000000 and 0x7fffffffffffffff
or any bit pattern between
0xfff8000000000000 and 0xffffffffffffffff
Note that there are many different bit patterns which represent NaNs.
I've heard that some implementations of the IEEE 754 standard use the extra bits towards the end of the 64-bit NaN representations to encode the location in the program where the NaN was created.
This scheme makes it easier for a programmer to debug a program which is generating NaNs by pinpointing the statement that generated the NaN.
References
1The
default value is the value that you get if the operation doesn't result in a
runtime exception (i.e. if the program doesn't terminate abnormally).
Whether or not an operation results in a NaN or a runtime exception depends upon whether the IEEE 754 INVALID exception is disabled or enabled respectively.
2the hypot(x,y) function defined as
sqrt(x*x + y*y) yields +infinity if x is infinity regardless of the value of y.
Consequently, hypot(infinity,y) should yield +infinity regardless of whether or not y is a NaN.
3this isn't quite true as the IEEE 754 standard has a rather strange concept known as -0 (i.e.
negative zero).
Dividing a positive
finite value by +0 yields +infinity whereas dividing a positive finite value by -0 yields -infinity (similarily negative finite divided by -0 yields +infinity and negative finite divided by +0 yields -infinity).