Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Topic: Testing the significance of difference in set means at different times
Replies: 9   Last Post: Nov 7, 2012 7:53 PM

 Messages: [ Previous | Next ]
 Stuart.Palmer@deakin.edu.au Posts: 6 Registered: 11/1/12
Re: Testing the significance of difference in set means at different times
Posted: Nov 1, 2012 6:10 PM

On Friday, 2 November 2012 01:13:08 UTC+11, Bruce Weaver wrote:
> On 01/11/2012 6:27 AM, Stua...@deakin.edu.au wrote:
>

> > On Thursday, 1 November 2012 17:47:30 UTC+11, Ray Koopman wrote:
>
> >> On Oct 31, 9:16 pm, Stuart.Pal...@deakin.edu.au wrote:
>
> >>
>
> >>> I have data from two large sets of samples (100s in one set [a1] and 1000s in the other set [b1]). a1 has a mean score Ma1. b1 has a mean score Mb1. I can evaluate the significance associated with the difference in mean scores [Ma1-Mb1] observed between these two sets.
>
> >>
>
> >>>
>
> >>
>
> >>> I also have two similar sets of data collected at a later period - a2 and b2, with means Ma2 and Mb2. Again, I can evaluate the significance associated with the difference in mean scores [Ma2-Mb2] observed between these two sets.
>
> >>
>
> >>>
>
> >>
>
> >>> I'm interested in whether there is a way to evaluate the significance of the observed change across the two time periods between the difference observed in the set means? That is, an appropriate method to evaluate the significance of the observed difference in (Ma1-Mb1)-(Ma2-Mb2).
>
> >>
>
> >>>
>
> >>
>
> >>> Any sugestions would be greatly appreciated.
>
> >>
>
> >>
>
> >>
>
> >> For each person, get the algebraic difference between their score
>
> >>
>
> >> at time 1 and their score at time 2. Then do an independent-groups
>
> >>
>
> >> t-test of the difference between the means of the two sets of
>
> >>
>
> >> difference scores.
>
> >
>
> >
>
> > Thanks for the reply.
>
> >
>
> > I omitted an important piece of information sorry, each of the sets
>
> is a cross-sectional sample (sub-set) of their respective populations,
>
> and the data in a1 and a2 (and b1 and b2) are not paired.
>
>
>
>
>
> When you say the observations are not paired, that could mean that each
>
> person (or unit of observation, if not people) appeared in one and only
>
> one of the 4 samples. Or it could mean that people could appear in more
>
> than one sample, but you don't have the information needed to pair up
>
> their observations. Which is it? If the former, you have 4 independent
>
> groups in a 2x2 layout, and you can use two-way ANOVA (Condition x
>
> Time). The null hypothesis for the interaction term is that the
>
> difference between A and B is the same at both time points. (It could
>
> also be stated that the difference between times 1 and 2 is the same for
>
> conditions A and B.)
>
>
>
> You can find some nice introductory notes on two-way (or factorial)
>
> ANOVA here: http://davidmlane.com/hyperstat/ -- see chapter 13.
>
>
>
> HTH.
>
>
>
> --
>
> Bruce Weaver
>
>
>
> "When all else fails, RTFM."

Cheers Bruce, thanks for that.

Sets A and B are cross-sectional (representative) samples of two different populations. The members of the sets at time 1 and 2 are different, though still representative. The principal measure of interest is the mean value (of a rating given by) the respective sets.

I have used one-way ANOVA to explore the significance of the difference in mean score between A1 and A2, B1 and B2, A1 and B1 (ie, [Ma1-Mb1]), and, A2 and B2 (ie, [Ma2-Mb2]).

My interest/question was about testing the significance of the 'difference of the differences' (Ma1-Mb1)-(Ma2-Mb2).

I had considered a two-way ANOVA using all of the data and looking at the significance of interaction term, but was unsure. I will look at this.

Thanks again.