One way to define infinite sets is: Removing finite sets from infinite sets does not modify the cardinality of these infinite sets.
Another way to define infinite sets is: There are proper subsets of infinite sets that have the the same cardinality of these supersets.
The axiom of infinity defines that infinite sets exist in math that is established under acception of this axiom.
In this math infinite sets not only are not required to be constructed from finitely (or infinitely) many operations on finite sets but this is also not possible.
This includes in the reverse way that infinite sets cannot be modified in their cardinality by applying finitely many finite operations on these infinite sets.
In this way finite and infinite sets are cleanly and consistently separated from each another so there are no contradictions.