Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: The power of a test
Replies: 5   Last Post: Jun 13, 2013 6:29 PM

 Messages: [ Previous | Next ]
 Cristiano Posts: 60 Registered: 12/7/12
Re: The power of a test
Posted: Jun 9, 2013 8:28 AM

On 09/06/2013 5:26, Rich Ulrich wrote:
> They do a Monte Carlo randomization of 1000 trials under
> the null condition; they use cutoffs for a test; they report
> the observed "size" of the test for this test. Since they were
> looking at a 5% tail, they expected 50, and this set of 1000
> trials happened to give them 38 -- The other tests were
> computed on the same set of 1000 and gave the same or
> off by 1 in the count of rejections. They were all the same.
>
> You need to study, read papers, contemplate, what-have-
> you, to get used to this terminology and set of expectations.

I should start from the English language, but unfortunately I don't have
much time.
I do what I can and I always hope that someone could explain things
without being too rude.

> For instance, it is technically a BAD performance for a RNG
> if it does not produce "random variation" in every criterion
> that you measure it against. And, 5 times out of 100, a
> set of 1000 trials will result in less than/ greater than the
> number of rejections specified by the CI -- if you have a
> decent RNG.

Using a good generator, I also see 0.034, 0.036 (occasionally), but it
seemed strange to me that the authors reported in that table a single value.
Now, thanks to you, I know that I was wrong; good!

> The later columns in the table are called "power" because
> (a) every sample is non-normal, by design, and (b) they
> represent how often that non-normality is detected.
>
> I hope that you recognize that this is a peculiar paper, in
> a way. Most of the time, authors are proposing ways to
> *detect* differences. These authors are proposing
> "robust measures" that will NOT report samples as non-
> normal when they are "merely" contaminated by outliers
> of one sort or another. Thus, they are happy and proud
> to point to the lack of power for detecting the specified
> sorts of contamination, for certain "robust" tests.

Yes, it's a peculiar paper; for that reason I'm trying to understand "in
deep" the exact meaning of the tables, formulas and phrases.

> If you want a test to detect non-normality in the form
> of those contaminations ... the JB does fine. The power
> of the so-called robust tests is going to be concentrated
> elsewhere. Presumably.

I usually use the KS and the AD tests for the normal distribution
(mainly because there is a procedure to calculate the p-values).

Now I'm interested in the JB test.
I'll try to use the quantiles calculated with a Monte Carlo simulation
and I'll try to use the Omnibus K^2 statistic as explained here:
formula 2.18.

> I'm not sure I see much value in their paper. If the JB
> rejects severely, and their robust tests do not, you might
> conclude, "Well, the sample would be pretty normal if it
> were not for 1%/5% contamination."
>
> But then... I've never, ever, paid much attention to any
> formal tests of normality. Maybe it is a lot more useful
> to someone who has had data where other tests of normality
> were useful.

Thank you
Cristiano

Date Subject Author
6/7/13 Cristiano
6/7/13 Richard Ulrich
6/8/13 Cristiano
6/8/13 Richard Ulrich
6/9/13 Cristiano
6/13/13 Luis A. Afonso