Jennifer Murphy <JenMurphy@jm.invalid> wrote: > James Waldby <email@example.com> wrote: >> >>For the given problem, averages of ranks probably aren't a statistically >>sound approach. For example, see the "Qualitative description" section >>of article <http://en.wikipedia.org/wiki/Rating_scale>, which says: >>"User ratings are at best ordinal categorizations. While it is not >>uncommon to calculate averages or means for such data, doing so >>cannot be justified because in calculating averages, equal intervals >>are required to represent the same difference between levels of perceived >>quality. The key issues with aggregate data based on the kinds of rating >>scales commonly used online are as follow: Averages should not be >>calculated for data of the kind collected." (etc.) > > Yes, I did feel a little uneasy about averaging numbers that are not > really numerical in the usual sense.
Yes, I think that approach would be wrong, based on the following extreme case counterexample: Suppose you have 100 different lists, 99 of which identically contain the same two books, and only those two books, in the same ranking, Lists 1-99 contain: Book#1=The Bernie Madoff Story, #2=The Ken Lay Story Finally, the 100th list contains, say 100 different books, including our above two losers, but ranked List 100 contains: Book#1=The complete works of Shakespeare, #2=..., #3=..., ..., and finally, #99=The Bernie Madoff Story, #100=The Ken Lay Story Clearly, Bernie and Ken suck, but when they're the only two books on a list, then they have to rank #1 and #2. So you need a methodology that avoids a combined ranking giving Bernie: 99 #1-scores and one #99-score, and giving Ken: 99 #2-scores and one #100-score. That would significantly overestimate them. -- John Forkosh ( mailto: firstname.lastname@example.org where j=john and f=forkosh )