Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: Please critique my scheme for re-weighting source data
Replies: 7   Last Post: May 27, 2012 11:57 AM

 Messages: [ Previous | Next ]
Re: Please critique my scheme for re-weighting source data
Posted: Feb 23, 2012 4:07 PM

On Feb 23, 3:20 pm, Jennifer Murphy <JenMur...@jm.invalid> wrote:
> On Thu, 23 Feb 2012 13:56:58 -0500, Rich Ulrich
>
> <rich.ulr...@comcast.net> wrote:

> >You give no hint, that I notice, of what it is that you
> >are trying to accomplish.

>
> >For most purposes of inference that come to my mind,
> >the extreme cases -- the ones that you seem to propose
> >to drop -- are the most informative and most interesting.
> >So I conclude that your interests are probably the opposite
> >(in some fashion) from what my naive interests would be.

>
> >I repeat-- What are you trying to do?
>
> I am trying to calculate for each word the relative likeliness that it
> would be encountered by an average well-educated person in their daily
> activities: reading the paper, listening to the news, attending classes,
> talking to other people, reading books, etc.
>
> The raw scores that I have already do that, but I question the
> weighting.I do not think that the average person encounters the types of
> words typically found in academic journals at the same frequency as they
> would those found in newspapers or magazines. Therefore, I want to
> re-weight the five sources to reflect a more average experience.

Seems to me what you are looking for is a kind of `basket of
information` that the `average well-educated person` would encounter
in their `daily activities`. So, I guess, those should be your
weights. In other words, you need to assess the volume of spoken
sources, etc. the average person you are interested in is exposed to
in each of the five groups. You could use that as a basis for your
weighting. Whether it beats the raw (equal) weights is not immediately
clear.

Date Subject Author
2/23/12 Jennifer Murphy
2/23/12 Richard Ulrich
2/23/12 Jennifer Murphy
2/23/12 RGVickson@shaw.ca
2/24/12 JohnF
5/24/12 James Beck
5/27/12 aruzinsky