Bottom line: student's common-sensible mathematical encounters with mathematical quantities are their only (conceptual understanding) means of personally deriving arithmetic and functional "numeracy." Without quantitative derivations, they are forced to play the (risky) game of tying to play scholastic "conventions."
I hope you meant to say that the only path in children to understanding numbers is to leverage their sense of physical quantity. But the ultimate and achievable goal is to break past that and have them thinking about numbers.
I think the most brilliant thing I have said on this forum in all of the years I have posted is that teachers need to teach the same set of students for a longer period of time, like for all of elementary school. I would have math teachers in elementary school teach one period of first graders, one of second, and so on, all the way through fifth grade. We'll leave kindergarten and sixth grade as they are. Kindergarten is mostly just getting used to having to go to school and sixth grade can be a weening period for middle school. But each math teacher should teach the same kids from 2nd through 5th grade. That would be the end of most of the nonsense. Oh, there will still be the kids failing and there will still be snake oil salesmen seeing an opportunity, but I think the impact on teachers will be enormous, and likewise on the students.
The biggest advantage I (and similar parents) have with teaching our children, that I can put my finger on, is that we get to continue on year after year. They grow and we learn. I cannot imagine anymore, teaching each child just one year, and passing them on to someone else while I move on to the next child. That is like 6 artists taking turns painting the same picture. Yes, there are kids that are quite bright and can integrate this stuff almost on their own, but their are a lot more kids that just need a little more consistent attention. And those really bright kids, just bump them up a grade. Easily done since we aren't talking about changing teachers.