Associated Topics || Dr. Math Home || Search Dr. Math

### Analyzing Survey Results

```
Date: 05/04/2000 at 15:34:05
From: Diane Davis
Subject: Survey statistics

A satisfaction survey was given that asked people to rank, in order of
importance, seven factors. I need to determine the meaning of the
results (e.g. what is the most important factor, second most important
factor, etc.) I've tabulated the results (see below) for each
category, and determined the percentage it represents of the ranking
and of the category as a whole.

1st  2nd  3rd  4th  5th  6th  7th  8th
Factor 1   15   36   13    9   12    4    9    0
Factor 2   26    9   23   13   12    9    1    4
Factor 3    3    4    9   18   20   21   17    2
Factor 4    3    8    5   16   16   20   16    9
Factor 5    7    6   12    8   12   11   19   21
Factor 6    7   14    9   18   12   11   19    6
Factor 7   16   16   16   12   10   17    9    0

How do you make a determination of priority when, for example, Factor
2 received the greatest number of responses for the first and third
ranks?

```

```
Date: 05/04/2000 at 20:50:58
From: Doctor Schwa
Subject: Re: survey statistics

In general, this problem is impossible to solve. Otherwise we'd have
voters rank the candidates in an election and then use the rankings to
pick an unambiguous overall favorite. But there's a nice little proof
out there that, according to a few simple rules, there is no possible
system that will always give the "best" answer.

So, we have to make do somehow. Statistically the easiest thing to
work with would be the average rank assigned to each factor, so why
not use that?

By that criterion, Factor 1 edges out 2 slightly; 7 is a distant
third, followed by 3, 4, 5, and 6.

By the way, why are there 8 places when there are only 7 factors? And
if people had to rank each thing, why isn't there the same number of
people total each time? Did everybody leave one rank spot blank? Also
each factor didn't get ranked by the same number of people...

- Doctor Schwa, The Math Forum
http://mathforum.org/dr.math/
```

```
Date: 05/09/2000 at 15:00:57
From: Diane Davis
Subject: Re: survey statistics

The reason there were 8 places is that there was an "other" response
where an answer could be written in. Some chose to use this, but there
weren't more than 2 of the same response.

Also, some respondents may have only indicated their 1st through 5th
choices, etc. which is why the total responses were not equal for each
factor.

This was clearly a very flawed instrument; however, it was never
really intended to be used statistically. That issue came up after the
fact.

How would you recommend designing a more useful "attitude" survey tool
in which results could be more accurately tabulated? If you could
recommend some resource books, Web sites, etc. I would appreciate it.

Thanks again for all of your help and quick response.
```

```
Date: 05/11/2000 at 13:16:49
From: Doctor Schwa
Subject: Re: survey statistics

>The reason there were 8 places is that there was an "other" response
>where an answer could be written in. Some chose to use this, but
>there weren't more than 2 of the same response.

"Other" is always dangerous if you want to get consistent data that
you can compare from person to person. But "other" is really useful if
you want to get qualitative data; people will write in things that are
very useful anecdotally, but very hard to analyze quantitatively.

>Also, some respondents may have only indicated their 1st through 5th
>choices, etc. which is why the total responses were not equal for
>each factor.

Yes, so I recommend that you ask people, instead of ranking them in
order of importance, to rate each one on a scale. For instance:
"choose a rating from 1 to 10, with 1 being totally unimportant and 10
being vitally important."

Then taking the average clearly makes sense. Ranks are much harder to
work with. Ranks also don't show so much when people have a clear
favorite or when there are two or three that are about equally
important to them.

>This was clearly a very flawed instrument; however, it was never
>really intended to be used statistically. That issue came up after
>the fact.
>
>How would you recommend designing a more useful "attitude" survey
>tool in which results could be more accurately tabulated? If you
>could recommend some resource books, websites, etc. I would
>appreciate it.

Survey design is a difficult topic. It's not taught particularly well
in most college stat courses. I haven't seen any particularly great
books on survey design. I recommend visiting the apstat-l discussion
list, archived by the Math Forum:

http://mathforum.org/kb/forum.jspa?forumID=67

and searching the archives of the list for words like "survey design"
- if you don't find anything good there, post a message asking for a
suggested reference. We may be the math doctors, but they are the real
experts on statistics.

>Thanks again for all of your help and quick response.

No problem!

- Doctor Schwa, The Math Forum
http://mathforum.org/dr.math/
```
Associated Topics:
College Statistics
High School Statistics

Search the Dr. Math Library:

 Find items containing (put spaces between keywords):   Click only once for faster results: [ Choose "whole words" when searching for a word like age.] all keywords, in any order at least one, that exact phrase parts of words whole words

Submit your own question to Dr. Math
Math Forum Home || Math Library || Quick Reference || Math Forum Search