Date: Mar 29, 2013 5:11 AM
Author: Luis A. Afonso
Subject: A rarely shown fact about p-values

A rarely shown fact about p-values

Note that because we are performing a *point-wise* test, H0: mu= mu0, and only with *mu0 data* the p-values are uniformly distributed, it follows that in real world the probability to observe a *significant* value is effectively larger than alpha when we are dealing with a right one-tailed test. In fact the p-values Distribution is, even slightly, left skewed and we stand at an *at most * alpha level. Preferably they join together near 1.


............................ <--real alpha--<
____________||_______________|_______________|1.0
....... Alpha=0.05 __________P
............||false positives|


By real alpha we note the interval inside which fall the 0.05 proportion of test values concerning the real distribution of p-values.. Note that from the supposed critical value when the p-values are uniformly distributed and the [ || , 1.0 ] interval containing the alpha proportion of them is the false positive region. They are classified wrongly positive in spite to be situated at the 5% interval [P, 1.0] left bound.

Simulation results

A test statistics is supposed to follow a Normal Standard z, therefore the p-values are u= phi(z), with mu0=0. When mu=1, 0.5, 0.2 , the fractions concerning u=0.95 (alpha=5%) are 0.254, 0.134, and 0.079 , all larger than 0.05. The false positive amounts to 24.9, 8.9 and 2.9% respectively.

Luis A. Afonso