Search All of the Math Forum:
Views expressed in these public forums are not endorsed by
NCTM or The Math Forum.



Re: For nonrejection of H0, don't we want high signifance?
Posted:
Apr 26, 2013 12:11 PM


On 20130426, Jeff Miller <milleratotago@yahoo.com> wrote: > On Wednesday, April 24, 2013 11:31:11 AM UTC+12, Paul wrote:
>> from a conceptual standpoint, say H0 was some generic hypothesis >> rather than residual normality. My question about whether the >> significance should be small or large depending on whether I'm >> interested resoundingly showing the resonableness of rejecting or not >> rejecting H0 still seems to hold.
> I think your basic intuition is right: Other things being equal, a _larger_ p value tends to indicate that the data are _less discrepant_ from what Ho would predict.
> But this view is so oversimplified that it can easily be misleading. For example, the p value might be large because there isn't much data or because there is lots of random error, and in neither case should you conclude that the data provide good support for Ho.
> When the research goal is to show that Ho is true to a good approximation, Ho testing is just not a good method. The confidence interval approach is much more informative in most such situations, because they allow conclusions of the form "Ho is not wrong by more than X units". CIs are not easily adapted to your original question of checking for normality, though, AFAIK.
This is what the research goal should be, but I do not know of any work on this other than my paper, which leaves much to be dessired. I considered the Bayesian testing of a slightly spread out null against a much more spread out alternative. What I found is that if the spread of the region in which one would want to accept is small relative to the standard error of estimate, one can treat it as a point null. If it is large relative to the standard error, just look at the value. In between, it depends on the nature of the spread of the weight measure for acceptance, and I have not in the 40+ years since I go this result found a good way of approximating the questions to the user to get robustness.
 This address is for information only. I do not claim that these views are those of the Statistics Department or of Purdue University. Herman Rubin, Department of Statistics, Purdue University hrubin@stat.purdue.edu Phone: (765)4946054 FAX: (765)4940558



