Search All of the Math Forum:
Views expressed in these public forums are not endorsed by
NCTM or The Math Forum.



Re: The power of a test
Posted:
Jun 9, 2013 8:28 AM


On 09/06/2013 5:26, Rich Ulrich wrote: > They do a Monte Carlo randomization of 1000 trials under > the null condition; they use cutoffs for a test; they report > the observed "size" of the test for this test. Since they were > looking at a 5% tail, they expected 50, and this set of 1000 > trials happened to give them 38  The other tests were > computed on the same set of 1000 and gave the same or > off by 1 in the count of rejections. They were all the same. > > You need to study, read papers, contemplate, whathave > you, to get used to this terminology and set of expectations.
I should start from the English language, but unfortunately I don't have much time. I do what I can and I always hope that someone could explain things without being too rude.
> For instance, it is technically a BAD performance for a RNG > if it does not produce "random variation" in every criterion > that you measure it against. And, 5 times out of 100, a > set of 1000 trials will result in less than/ greater than the > number of rejections specified by the CI  if you have a > decent RNG.
Using a good generator, I also see 0.034, 0.036 (occasionally), but it seemed strange to me that the authors reported in that table a single value. Now, thanks to you, I know that I was wrong; good!
> The later columns in the table are called "power" because > (a) every sample is nonnormal, by design, and (b) they > represent how often that nonnormality is detected. > > I hope that you recognize that this is a peculiar paper, in > a way. Most of the time, authors are proposing ways to > *detect* differences. These authors are proposing > "robust measures" that will NOT report samples as non > normal when they are "merely" contaminated by outliers > of one sort or another. Thus, they are happy and proud > to point to the lack of power for detecting the specified > sorts of contamination, for certain "robust" tests.
Yes, it's a peculiar paper; for that reason I'm trying to understand "in deep" the exact meaning of the tables, formulas and phrases.
> If you want a test to detect nonnormality in the form > of those contaminations ... the JB does fine. The power > of the socalled robust tests is going to be concentrated > elsewhere. Presumably.
I usually use the KS and the AD tests for the normal distribution (mainly because there is a procedure to calculate the pvalues).
Now I'm interested in the JB test. I'll try to use the quantiles calculated with a Monte Carlo simulation and I'll try to use the Omnibus K^2 statistic as explained here: http://hj.se/download/18.3bf8114412e804c78638000150/1299244445855/WP20108.pdf formula 2.18.
> I'm not sure I see much value in their paper. If the JB > rejects severely, and their robust tests do not, you might > conclude, "Well, the sample would be pretty normal if it > were not for 1%/5% contamination." > > But then... I've never, ever, paid much attention to any > formal tests of normality. Maybe it is a lot more useful > to someone who has had data where other tests of normality > were useful.
Thank you Cristiano



