First question. If the AP curriculum does not included type I and type II errors, then how would one describe significance levels? Given that alpha is the probability of a type I error, it is possible to discuss alpha without bringing errors into the discussion?
Second. I get the impression that often tests of significance are done without setting an alpha level. If that is the case, how does one really make a decision based on the P-value? No matter how small P may be, there is still a chance that we will be wrong if we conclude a significant effect is present.
Last. I would like comments on the following multiple-choice question. The question assumes no preset significance level.
Q: A significance test gives a P-value of .04. From this we can say:
A. We will fail to reject H0 4% of the time B. We will reject H0 4% of the time C. If we reject H0, our decision has a 4% chance of being incorrect D. If we reject H0, our decision has a 4% chance of being correct E. None of the above are true