> The other difference is that we never said that all mathematics > is based on geometry, can be expressed in terms of geometry, > can be justified on the basis of geometry.
That's pretty well Euclid's approach. Numbers, including number theory, are grounded in geometry. Had Euclid been shown the proof of the independence of the parallel postulate, my guess is that he's still assume there was one correct geometry, and hyperbolic and elliptic geometries are just things we can define in terms of that geometry.
We do hear that all > of mathematics can be expressed in ZFC (or at any rate all the > mathematics that isn't specifically designed to be done outside > ZFC), and this makes the independence results of set theory > a bit more worrisome than those of geometry.
Isn't this a contradiction? The independence results show that mathematics can't all be reduced to ZFC. Of course, the independence results themselves involve inner models and forcing models, but it seems to me the obvious way to interpret it all is that there are reasonable ways to add axioms to ZFC.