I recently used ANOVA to see if there was a significant interaction in a before-and-after treatment-vs-control experiment. (Two levels within each of two factors, with a single dependent variable.)
However, upon presenting my results it was suggested that instead of using ANOVA, I should use a simple Z-test, as follows:
Difference of differences = (t2-t1) - (c2-c1)
where t2 = "After" treatment mean t1 = "Before" treatment mean c2 = "After" control mean c1 = "Before" control mean
and the standard error of this mean would be equal to the square root of the four individual variances summed.
Something about using this Z-test procedure instead of ANOVA doesn't sit well with me, though I'm having trouble pinpointing why. It seems like there's information not being captured, but I'm having difficulty explaining what exactly is wrong, if anything, with Z-tests instead of ANOVA.