Hypothesis Testing

Date: 04/22/99 at 08:19:41
From: Kimberly Long
Subject: p-value

I have a question regarding a p-value relative to where the null 
hypothesis value falls in the Confidence Interval. The first example 
below has a p-value (0.99) with a null hypothesis value that barely 
falls within the upper limit of the 95% Confidence Interval (0.161185, 
0.500078) and an alternate that falls outside the CI. The second 
example has a lower p-value (0.070) when the null hypothesis and 
alternative fall well within the CI (0.035131, 0.2899948). Would you 
please explain this? (Note: Minitab output below)


Test and Confidence Interval for One Proportion
Test of p = 0.5 vs p > 0.5
Success = M
Exact
Variable          X      N  Sample p        95.0 % CI       P-Value 
Sex              10     32  0.312500  (0.161185, 0.500078)    0.990

Test and Confidence Interval for One Proportion
Test of p = 0.25 vs p < 0.25
Success = Y
Exact
Variable          X      N  Sample p        95.0 % CI       P-Value 
Smoke             4     32  0.125000  (0.035131, 0.289948)    0.070

Date: 04/22/99 at 19:11:39
From: Doctor Pat
Subject: Re: p-value

Kimberly,

P values and confidence intervals are two different ways of 
approaching the decision of hypothesis testing. Let's take them one at 
a time. Keep in mind that in both cases you are testing whether or not 
it is reasonable to believe that the NULL might be true. You are not 
proving it, but asking if it is a reasonable number.  

The p value does this by telling you the probability that you would 
get a sample proportion this far (whatever happened in this sample) or 
farther from the null IF THE NULL HYPOTHESIS WERE TRUE. If the number 
is so small you can't believe it happened by chance, then you reject 
the null and accept that the alternative is more likely to be true.  
Usually this is a pre-established alpha value .05 or .01 etc. 
  
The confidence interval is a prediction model... it says "given the 
results of our sample, our best estimate of the true proportion is 
somewhere in the interval specified" and the level of confidence is 
1- the alpha level of the p value approach. Now if we are confident 
that the truth is inside this interval, then we can accept the null as 
a possible proportion if it falls inside that interval, and reject the 
null if it is outside the interval.  

In both cases these two tests are the same. It's two different ways of 
saying the same thing.. 

Hope that helps.

- Doctor Pat, The Math Forum
  http://mathforum.org/dr.math/