Analyzing Survey ResultsDate: 05/04/2000 at 15:34:05 From: Diane Davis Subject: Survey statistics A satisfaction survey was given that asked people to rank, in order of importance, seven factors. I need to determine the meaning of the results (e.g. what is the most important factor, second most important factor, etc.) I've tabulated the results (see below) for each category, and determined the percentage it represents of the ranking and of the category as a whole. 1st 2nd 3rd 4th 5th 6th 7th 8th Factor 1 15 36 13 9 12 4 9 0 Factor 2 26 9 23 13 12 9 1 4 Factor 3 3 4 9 18 20 21 17 2 Factor 4 3 8 5 16 16 20 16 9 Factor 5 7 6 12 8 12 11 19 21 Factor 6 7 14 9 18 12 11 19 6 Factor 7 16 16 16 12 10 17 9 0 How do you make a determination of priority when, for example, Factor 2 received the greatest number of responses for the first and third ranks? Any help you give is appreciated. Thank you. Date: 05/04/2000 at 20:50:58 From: Doctor Schwa Subject: Re: survey statistics In general, this problem is impossible to solve. Otherwise we'd have voters rank the candidates in an election and then use the rankings to pick an unambiguous overall favorite. But there's a nice little proof out there that, according to a few simple rules, there is no possible system that will always give the "best" answer. So, we have to make do somehow. Statistically the easiest thing to work with would be the average rank assigned to each factor, so why not use that? By that criterion, Factor 1 edges out 2 slightly; 7 is a distant third, followed by 3, 4, 5, and 6. By the way, why are there 8 places when there are only 7 factors? And if people had to rank each thing, why isn't there the same number of people total each time? Did everybody leave one rank spot blank? Also each factor didn't get ranked by the same number of people... - Doctor Schwa, The Math Forum http://mathforum.org/dr.math/ Date: 05/09/2000 at 15:00:57 From: Diane Davis Subject: Re: survey statistics Thank you for your help. The reason there were 8 places is that there was an "other" response where an answer could be written in. Some chose to use this, but there weren't more than 2 of the same response. Also, some respondents may have only indicated their 1st through 5th choices, etc. which is why the total responses were not equal for each factor. This was clearly a very flawed instrument; however, it was never really intended to be used statistically. That issue came up after the fact. How would you recommend designing a more useful "attitude" survey tool in which results could be more accurately tabulated? If you could recommend some resource books, Web sites, etc. I would appreciate it. Thanks again for all of your help and quick response. Date: 05/11/2000 at 13:16:49 From: Doctor Schwa Subject: Re: survey statistics >The reason there were 8 places is that there was an "other" response >where an answer could be written in. Some chose to use this, but >there weren't more than 2 of the same response. "Other" is always dangerous if you want to get consistent data that you can compare from person to person. But "other" is really useful if you want to get qualitative data; people will write in things that are very useful anecdotally, but very hard to analyze quantitatively. >Also, some respondents may have only indicated their 1st through 5th >choices, etc. which is why the total responses were not equal for >each factor. Yes, so I recommend that you ask people, instead of ranking them in order of importance, to rate each one on a scale. For instance: "choose a rating from 1 to 10, with 1 being totally unimportant and 10 being vitally important." Then taking the average clearly makes sense. Ranks are much harder to work with. Ranks also don't show so much when people have a clear favorite or when there are two or three that are about equally important to them. >This was clearly a very flawed instrument; however, it was never >really intended to be used statistically. That issue came up after >the fact. > >How would you recommend designing a more useful "attitude" survey >tool in which results could be more accurately tabulated? If you >could recommend some resource books, websites, etc. I would >appreciate it. Survey design is a difficult topic. It's not taught particularly well in most college stat courses. I haven't seen any particularly great books on survey design. I recommend visiting the apstat-l discussion list, archived by the Math Forum: http://mathforum.org/kb/forum.jspa?forumID=67 and searching the archives of the list for words like "survey design" - if you don't find anything good there, post a message asking for a suggested reference. We may be the math doctors, but they are the real experts on statistics. >Thanks again for all of your help and quick response. No problem! - Doctor Schwa, The Math Forum http://mathforum.org/dr.math/ |
Search the Dr. Math Library: |
[Privacy Policy] [Terms of Use]
Ask Dr. Math^{TM}
© 1994- The Math Forum at NCTM. All rights reserved.
http://mathforum.org/dr.math/