On Sun, 30 Dec 2012 13:12:36 -0800 (PST), firstname.lastname@example.org wrote: [snip, before and after] > >Making the story short, I am missing detailed cookbook description of test saying clearly what are the assumptions and what are the null and alternative hypothesis. >
It would be nice if such a thing exists. I don't remember ever seeing any collection like that. Maybe someone else knows of something.
I do remember the comment from one of my first stat teachers, that you don't understand a test until you know why it rejects when its competitors do not, and vice-versa. I usuallly learned those details by looking at several forms of the computation formulas for the tests, and noting where the difference exists. - You probably need some background in statistical estimation theory to have it all make best sense.
You do not mention the more subtle points that can arise with the tests you named.
COMPUTATION. The Wilcoxon is exactly the same as the Kruskal-Wallis when the latter is applied to two groups: If you see any difference in their reported p-values, it will usually be because the two algorithms have not made precisely the same (approximate) adjustment for the correction for ties. Or else, the two are using different algorithms for either the small-sample, exact value, or for the large-sample approximation.
CRITERION. The KS test also starts with ranks, but it uses a *single* point of extreme difference for its test. And the usual tables do not apply exactly when the data features ties. So this test differs from the other two because it has a different criterion. With a bit more generality, I think we might say that it has a different "loss function" for measuring departure the null.
For some other tests:
It is common to see t-test presented with tests for pooled vs separate variances. Can you tell by looking at SDs and Ns which test will be "more powerful" for given comparison? [assumption]
It is common (SPSS, say) to see a contingency table with both the Pearson chisquared test and the Likelihood test. Do you know which test is more sensitive to which kind of difference? [criterion]
Can you construct a set of paired data for which the paired t-test is less powerful than the separate-groups t-test? [assumption] (Is it ever fair to ignore the knowledge that these data are correlated?)
The Spearman and Kendall coefficients for rank-correlation do not have the same rejection area. Do you have a reason for selecting one over the other? [criterion]