That's the way Gauss understood it. However, prevailing sentiment against actual infinities started to change with Cantor (not that there were not earlier proponents) and its now status quo to accept both kinds.
I think that is tied into the arguments (context) you are in. I mean, when we say that 0.999... equals 1 we are actually saying that the limit of 0.999... equals 1 and limits add and multiply just like exact quantities, in fact limits of exact quantities are exact quantities and also add and multiply the same as well. So from that standpoint, there is no difference.
Can you give an example of "infinity as a number" (I am using the phrase loosely).