<< Don't most of us have mixed feelings about what the subject matter of mathematics is? >>
Yes, most of us do. But a few hardy souls (no pun intended, but there it is) have speculated about the nature of mathematics. Some of them were even mathematicians, like Brouwer (intuitionism) and Hilbert (formalism) and R L Wilder (not sure how to classify him). I take it that Hardy didn't go as extensively into such foreign fields as these two, which may be why he called his little book "A Mathematician's Apology". I suppose Bertrand Russell, and Schroeder before him would usually be classified as more logicians than mathematicians, or simply as philosophers (no slight intended). Same for Plato, Aristotle, Nicholas of Cusa, and so on. And there are people like Isaac Newton, who occasionally philosophized a little, for example, in his Principia mathematica _philosophiae_ naturalis (general scholium to Book III), or Albert Einstein, or Erwin Schroedinger (e.g., What is Life?) or Werner Heisenberg (I forget the title of his philosophical work), or a whole host of physicists, especially when they get old.
I guess most mathematicians, in our current usages of the term, don't give much of a hoot about the ontological status of numbers or the nature of the existence of mathematical objects and processes, nor where proofs come from (storks, maybe?), except as the sort of thing one might talk about over coffee, beer, wine, brandy, or on a long airplane flight. Now in the times when the term "mathematician" ("mathematicus") was still compounded with the term "astrologer" ("astrologus"), there was apt to have been more concern about foundations of mathematics, especially as related to theological matters. This last reflection has no doubt been brought on by the fact that I am currently scouring the Mysterium Cosmographicum (Secret of the Universe), written by the young Johannes Kepler, who loved to philosophize about the foundations not only of mathematics, but of mathematical foundations of the whole universe.