Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: Is there a way to calculate an average ranking from uneven lists?
Replies: 15   Last Post: Oct 30, 2013 12:18 PM

 Messages: [ Previous | Next ]
 JohnF Posts: 219 Registered: 5/27/08
Re: Is there a way to calculate an average ranking from uneven lists?
Posted: Oct 28, 2013 6:26 AM

Jennifer Murphy <JenMurphy@jm.invalid> wrote:
> James Waldby <not@valid.invalid> wrote:
>>
>>For the given problem, averages of ranks probably aren't a statistically
>>sound approach. For example, see the "Qualitative description" section
>>of article <http://en.wikipedia.org/wiki/Rating_scale>, which says:
>>"User ratings are at best ordinal categorizations. While it is not
>>uncommon to calculate averages or means for such data, doing so
>>cannot be justified because in calculating averages, equal intervals
>>are required to represent the same difference between levels of perceived
>>quality. The key issues with aggregate data based on the kinds of rating
>>scales commonly used online are as follow: Averages should not be
>>calculated for data of the kind collected." (etc.)

>
> Yes, I did feel a little uneasy about averaging numbers that are not
> really numerical in the usual sense.

Yes, I think that approach would be wrong, based on the following
extreme case counterexample:
Suppose you have 100 different lists, 99 of which identically contain
the same two books, and only those two books, in the same ranking,
Lists 1-99 contain: Book#1=The Bernie Madoff Story, #2=The Ken Lay Story
Finally, the 100th list contains, say 100 different books, including
our above two losers, but ranked
List 100 contains: Book#1=The complete works of Shakespeare,
#2=..., #3=..., ..., and finally,
#99=The Bernie Madoff Story, #100=The Ken Lay Story
Clearly, Bernie and Ken suck, but when they're the only two
books on a list, then they have to rank #1 and #2.
So you need a methodology that avoids a combined ranking giving
Bernie: 99 #1-scores and one #99-score, and giving
Ken: 99 #2-scores and one #100-score.
That would significantly overestimate them.
--
John Forkosh ( mailto: j@f.com where j=john and f=forkosh )

Date Subject Author
10/27/13 Ben Bacarisse
10/28/13 Jennifer Murphy
10/30/13 Jennifer Murphy
10/27/13 James Waldby
10/28/13 Jennifer Murphy
10/28/13 Graham Cooper
10/28/13 Graham Cooper
10/28/13 Graham Cooper
10/28/13 Graham Cooper
10/28/13 Graham Cooper
10/28/13 Jennifer Murphy
10/28/13 JohnF
10/28/13 Jennifer Murphy
10/29/13 JohnF
10/30/13 Jennifer Murphy