Your use of the phrase in this instance, however, is interpreted according to the following remark from the link above. For what it is worth, the burden of proof is upon you. You claim that it must be possible to formalize mathematics differently from how it is formalized. You have committed yourself to a "social responsibility" of demonstrating the alternative. That is your choice. So, it is your burden and it has nothing to do with your dislike of Cantor's transfinite arithmetic.
"Within philosophy, Occam's Razor (OR) is often wielded against metaphysical theories which involve allegedly superfluous ontological apparatus. Thus materialists about the mind may use OR against dualism, on the grounds that dualism postulates an extra ontological category for mental phenomena. Similarly, nominalists about abstract objects may use OR against their platonist opponents, taking them to task for committing to an uncountably vast realm of abstract mathematical entities. The aim of appeals to simplicity in such contexts seem to be more about shifting the burden of proof, and less about refuting the less simple theory outright."