The Math Forum



Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.


Math Forum » Discussions » sci.math.* » sci.math

Topic: Is there a way to calculate an average ranking from uneven lists?
Replies: 15   Last Post: Oct 30, 2013 12:18 PM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
JohnF

Posts: 219
Registered: 5/27/08
Re: Is there a way to calculate an average ranking from uneven lists?
Posted: Oct 28, 2013 6:26 AM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

Jennifer Murphy <JenMurphy@jm.invalid> wrote:
> James Waldby <not@valid.invalid> wrote:
>>
>>For the given problem, averages of ranks probably aren't a statistically
>>sound approach. For example, see the "Qualitative description" section
>>of article <http://en.wikipedia.org/wiki/Rating_scale>, which says:
>>"User ratings are at best ordinal categorizations. While it is not
>>uncommon to calculate averages or means for such data, doing so
>>cannot be justified because in calculating averages, equal intervals
>>are required to represent the same difference between levels of perceived
>>quality. The key issues with aggregate data based on the kinds of rating
>>scales commonly used online are as follow: Averages should not be
>>calculated for data of the kind collected." (etc.)

>
> Yes, I did feel a little uneasy about averaging numbers that are not
> really numerical in the usual sense.


Yes, I think that approach would be wrong, based on the following
extreme case counterexample:
Suppose you have 100 different lists, 99 of which identically contain
the same two books, and only those two books, in the same ranking,
Lists 1-99 contain: Book#1=The Bernie Madoff Story, #2=The Ken Lay Story
Finally, the 100th list contains, say 100 different books, including
our above two losers, but ranked
List 100 contains: Book#1=The complete works of Shakespeare,
#2=..., #3=..., ..., and finally,
#99=The Bernie Madoff Story, #100=The Ken Lay Story
Clearly, Bernie and Ken suck, but when they're the only two
books on a list, then they have to rank #1 and #2.
So you need a methodology that avoids a combined ranking giving
Bernie: 99 #1-scores and one #99-score, and giving
Ken: 99 #2-scores and one #100-score.
That would significantly overestimate them.
--
John Forkosh ( mailto: j@f.com where j=john and f=forkosh )



Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© The Math Forum at NCTM 1994-2017. All Rights Reserved.