> I don't know of free programs, but I believe Mathematica does support > arbitrary precision numbers.
I have tried MATHEMATICA, but my general impression is that the error control is entirely unreliable. Even if one requests AccuracyGoal->12 and PrecisionGoal->12, the program returns at most 3-4 accurate digits. Requesting higher precision produces failures due to the lack of memory, etc. The above is probably easy to predict, since MATHEMATICA seems to use rather standard non-adaptive finite-differences, and these usually cannot be more accurate. So, it seems that the arbitrary precision arithmetic is not really of advantage in MATHEMATICA, at least in the case of the PDE solving.
(or maybe I am wrong and someone will tell me how to overcome these barriers).