One of the reasons that there is no nobel prize in Mathematics might be the fact that the leading Swedish mathematician at the time, Gösta Mittag-Lefler, was a personal enemy of Nobel's.
Mittag-Lefler, a brilliant mathematician and the founder of the Mittag-Lefler Institute, was also a controversial and sometimes feared person. He would often participate in inflamed public debates, and he was a conservative in political matters. If a Nobel Prize in Mathematics had been created, Gösta Mittag-Lefler would have had a very good chance of receiving one of the first ones. Nobel would not have wanted this to happen.

It seems that the Nobel Archive at almaz.com/nobel has a long page on this subject, on http://almaz.com/nobel/why_no_math.html. They also focus heavily on Gösta Mittag-Lefler, but they have some evidence that Nobel was not so averse to him. They also mention ariels patently false story, noting in passing that Nobel had no wife.