On 22 May 2006 09:48:40 -0700, "dantimatter" <firstname.lastname@example.org> wrote:
> Hello all, > > I'm trying to fit a curve to correlation data, but the problem is that > I don't know what weighting I should apply to get a good chi-squared > fit. The data fed to the correlator is sampled logarithmically, so > that the sampling time for the data at the low end of the correlation > curve is very short compared to the times for the data at the high end. > This of course means that the data at the low end has been averaged > over more times than the data at the high end. > > Is there a good way to weight the residuals for the best possible fit?
At first, I thought that I should be able to understand this, but I finally conclude that I don't even know for sure that "correlation data" means "paired values" to be correlated.
From those of us practicing more generic statistics -- I think we are not familiar with the phrase, "data fed to the correlator."
Google gives 20 hits on that phrase. It seems most often to occur in articles about processing photons. You can probably get a better answer if you spell out more details for us.
The latter description which *seems* intelligible at first glance is contradictory to me on closer inspection.
That is, "sampling time is shorter at the low" end seems to my naive reading to be contradicted by "averaged over more times" at the low end -- "Shorter sampling time" would imply less accuracy for individual measurements; "averaged over more times: would imply greater accuracy.
In any case, a "good chi-squared fit" depends on what you are fitting, and "best possible fit" should be the one that gives the sensible model, and not the one that gives the smallest conceivable test value -- You would probably get that from putting all your weight on just a few points.
Assuming that the data selection represent the universe that you want to predict to, you possibly want to weight each score by the inverse of its variance.