In statistics text books it is proposed that sampling from a finite population with replacement is equivalent to sampling from an infinite population. I find this somewhat misleading.
Suppose that we have a population of size N generated by random variable Normal(MeanM, StdDevM). Then take samples of size n < N from this population and calculate average (let's call it MeanS).
MeanS = (1/n)*sum of samples
There is no way you can estimate MeanM in an unbiased fashion. You can only estimate population mean (let's call it MeanP) which is not equal to MeanM, the mean of random variable that generated the population.
Is my thinking flawed? Or do we always infer about an hypothetical infinite population?