I have a bit of a problem in my research and I was wondering if any could help.
The problem is: 12 Subjects rated 20 items on 2 different scales. I believe that on one of these scales subjects will agree in general on what ratings to give and on the other there will be more variability between subjects' patterns.
To get a measure of subject agreement I have correlated each subject's scores with each other subject's score on that scale. This give me two set of r values one of 0.09 and one of 0.01. I want to know if a t test on these r values would be a legitimate test to conduct to see if there is significantly greater agreement between subjects and one scale than on the other. If so does this test have a particular name?
If the procedure is not appropriate then can anyone suggest a test that can compare inter subject reliability.
I would prefer if replies are e-mailed and I will post a summary of replies if I receive them