Search All of the Math Forum:
Views expressed in these public forums are not endorsed by
NCTM or The Math Forum.



_pragmatic_ item/scale analysis
Posted:
Jun 20, 2004 10:01 AM


Hello to all!
A client to our university stats consulting has produced a questionaire with over 250 items that he considers to be associated to 12 scales (with 630 items for a scale). There is consent that the number of items is far too high in a clinical context, and has asked us to reduce the item count in a "pragmatic" way. Hm. He objects to the idea of re evaluating content validity ("they're all reasonable"). Nevertheless, I've been asked to cut 30% of the items while retaining the 12 scales.
Now, I am not really proficient in this kind of pragmatics. Having read through the usual references I have a devised a list of "selection criteria" to eliminate items.
The first consideration _after_ this was: any procedure meeting the demand of "no more than x items" is likely to miss the "optimal" scale. I might _have_ to eliminate items although their withdrawal will lower Cronbachs alpha and deteriorate other desirable properties.
For the time being, I would kindly ask you to comment on the following list. Please recall that it is not about _sense_ / validity: I have to come up with some formal or quantitative algorithmstyle "kick'emoutnoquestionsasked" thing. I am _not_ proud of it.
So here is what I have so far: exclude (yes/no)item if (1) solution probability is outside range 0.2<p<0.8. The item does not help differ between people and does not contribute substantially to scale variance. (2) item correlates negative with total and mostly negative with other items of that scale. Assuming correct coding scheme, the item measures sth. else. (3) reliability measure Cronbachs alpha rises if item deleted, this is implemented in both SAS and SPSS routines. (4) item correlates r>.90 with some (13) other items and exclusion does not lower reliability too much. Then it is considered "semantically redundant" (5) the distribution of solution probabilities has a lower dispersion if item excluded (propto [1], I guess) (6) all else being equal, switch to linguistics and drop item with higher lexical and syntactical complexity.
_Technically_, I am confident that I can write a SAS macro that does some sort of stepwise selection using indices of (1)(4). I am not sure what to think of its output.
So it would be of enormous help if you could comment on that list or provide other hints on "reducing questionaires".
Thanks for your consideration.
HansChristian Waldmann
 PD Dr. Hans C Waldmann Methodology & Applied Statistics in Psychology & the Health Sciences located: ZKPR, University of Bremen, Grazer Str.6, D28359 Bremen mail: waldmann@samson.fire.unibremen.de web: http://samson.fire.unibremen.de/waldmann friend of: IBMAIX  PERL  POSTGRESQL  SAS  LaTeX  VIM



