Math Forum Discussions

paulvonhippel at yahoo

Posts: 72
Registered: 7/13/05

Re: Reducing bias of a Bayesian point estimator
Posted: Oct 18, 2012 12:39 AM

Plain Text

On Monday, October 15, 2012 8:55:57 AM UTC-5, David Jones wrote:
>
> news:97c2645d-3a4e-45dc-9d31-1a797fa6d364@googlegroups.com...
>
> On Sunday, October 14, 2012 3:58:58 PM UTC-5, David Jones wrote:
>
> > "Paul" wrote in message
>
> >
>
> > news:f18a81ba-069f-45fc-ad7c-2da6931ee06a@googlegroups.com...
>
> >
>
> >
>
> >
>
> >
>
> >
>
> > On Saturday, October 13, 2012 5:23:14 PM UTC-5, David Jones wrote:
>
> >
>
> > > "Paul" wrote in message
>
> >
>
> > >
>
> >
>
> > > I am interested in ways of reducing the bias of a point estimator when
>
> > > the
>
> >
>
> > >
>
> >
>
> > > true parameter is near the boundary of the parameter space.
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > > Suppose g = gamma U / (N-1), where U ~ chisq(N-1), N is a known small
>
> >
>
> > > sample
>
> >
>
> > >
>
> >
>
> > > size, and gamma is an unknown parameter. A priori we know that 0 <=
>
> > > gamma
>
> >
>
> > > <
>
> >
>
> > >
>
> >
>
> > > 1. Notice that the upper inequality is strict; that is, gamma cannot
>
> > > have
>
> >
>
> > > a
>
> >
>
> > >
>
> >
>
> > > value of 1.
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > > One approach to estimation is to assign gamma a prior distribution that
>
> > > is
>
> >
>
> > >
>
> >
>
> > > uniform on (0,1). Then the posterior distribution of gamma is a scaled
>
> >
>
> > >
>
> >
>
> > > inverse chi-square, truncated on the right at 1. Now the obvious point
>
> >
>
> > >
>
> >
>
> > > estimators are the posterior mean and median. (I canï¿½t use the mode
>
> >
>
> > > because
>
> >
>
> > >
>
> >
>
> > > it can take a value of 1.) The trouble with the posterior mean and
>
> > > median
>
> >
>
> > > is
>
> >
>
> > >
>
> >
>
> > > that they have large negative biases if the true value of gamma is
>
> >
>
> > > actually
>
> >
>
> > >
>
> >
>
> > > close to 1.
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > > Iï¿½d be grateful for ideas on how to reduce this bias. One idea Iï¿½ve
>
> > > been
>
> >
>
> > >
>
> >
>
> > > toying with is to use a posterior quantile greater than the median ï¿½
>
> > > i.e.,
>
> >
>
> > >
>
> >
>
> > > quantile p where p>1/2. Maybe I would use a larger p when I had a larger
>
> >
>
> > > g.
>
> >
>
> > >
>
> >
>
> > > This isnï¿½t an idea that Iï¿½ve seen discussed elsewhere. Many thanks
>
> > > for any
>
> >
>
> > >
>
> >
>
> > > references on this or other possibilities.
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > > -------------------------------
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > > (1) Why do you think "bias" is important?
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > >
>
> >
>
> > > (2) If you want to define a point estimate in a Bayesian context, it
>
> >
>
> > > would
>
> >
>
> > >
>
> >
>
> > > be best to define a realistic loss function for the actual situation and
>
> >
>
> > > to
>
> >
>
> > >
>
> >
>
> > > use this to derive the corresponding "best" point estimate.
>
> >
>
> > -----------------------------------------
>
> >
>
> >
>
> >
>
> > Bias is important. The quantity that I am estimating, gamma, is a variance
>
> >
>
> > that will be used to calculate confidence intervals. If the estimate of
>
> >
>
> > gamma is negatively biased, then the coverage of the confidence intervals
>
> >
>
> > will be too low.
>
> >
>
> >
>
> >
>
> > What is an appropriate loss function to use under these circumstances?
>
> >
>
> >
>
> >
>
> > ----------------------------------------------------------------
>
> >
>
> >
>
> >
>
> > "Bias" is most often used in the context of an arithmetic mean. It is
>
> > clear
>
> >
>
> > from these extra details that you are not actually concerned with
>
> > estimating
>
> >
>
> > gamma. It is also somewhat confusing that you are contemplating mixing the
>
> >
>
> > classical and Bayesian paradigms. The phrase "is a variance that will be
>
> >
>
> > used to calculate confidence intervals" indicates that there are other
>
> >
>
> > statistics around, possibly statistically dependent on g, and these
>
> >
>
> > dependencies would need to be taken into account
>
> >
>
> >
>
> >
>
> > For a purely classical approach, you would need to evaluate the sampling
>
> >
>
> > distribution of some combination of a sample statistic and the selected
>
> >
>
> > estimate of gamma. Supposed bias in the estimate of gamma is unimportant
>
> >
>
> > because any such effects are eliminated by correctly evaluating the
>
> > sampling
>
> >
>
> > distribution of the combined statistic, and in using this to derive the
>
> >
>
> > confidence interval. Clearly you wouldn't be expecting to use a Student's
>
> > t
>
> >
>
> > distribution here. Of course using different combinations of sample
>
> >
>
> > statistics and different selected estimates of gamma would typically lead
>
> > to
>
> >
>
> > different confidence intervals with different properties. If evaluation of
>
> >
>
> > the distribution can't be done analytically, then simulation or
>
> >
>
> > bootstrapping may be useful routes to a practical procedure.
>
> >
>
> >
>
> >
>
> > For a sensible Bayesian approach you would want to evaluating a credible
>
> >
>
> > interval, not a confidence interval, for your other parameter of interest.
>
> >
>
> > This would involve integrating the joint posterior distribution of the
>
> >
>
> > parameters with respect to gamma. Of course the answers here would depend
>
> > a
>
> >
>
> > lot on the joint prior distribution of all the parameters and you would
>
> > need
>
> >
>
> > to have good reasons for any assumptions. You didn't seem particularly
>
> >
>
> > convinced of the "uniform on (0,1)" distribution for just one of the
>
> >
>
> > parameters, and you would also need to consider dependence in the joint
>
> >
>
> > prior distribution.
>
>
>
> ---------------------------------------------------
>
>
>
> I am in fact interested in estimating gamma. I am using a Bayesian approach
>
> to account for the fact that gamma cannot exceed 1, but I would like the
>
> resulting estimate to have minimal bias in a frequentist sense.
>
>
>
> The estimation of gamma is embedded in a more complicated problem, but I
>
> believe progress can be made by addressing the estimation of gamma alone.
>
> Many thanks for any suggestions regarding the problem as I originally posed
>
> it.
>
>
>
> ---------------------------------------------------
>
>
>
> Many of the usual frequentist techniques for bias adsjustment would lead to
>
> estimates not obeying your bound. If you were not insisting on not allowing
>
> a value of gamma exactly equal to one then the obvious thing to start from
>
> would be a simple truncation at 1. You might consider truncating at a value
>
> slightly smaller than one if an exact-1 causes problems. Otherwise you
>
> might be able to base something on he following....
>
>
>
> (a) let T be the non-truncated version of the estimated for gamma;
>
>
>
> (b) the transformed estimate S=T/(1+T) should have good properties if the
>
> true value of gamma is small, and is forced to lie in the required interval;
>
>
>
> (c) consider the extended set of estimates S=T/(1+T^a)^(1/a), which are
>
> again forced to lie in 0 to 1, and choose a for good properties
>
>
>
> Other families of transformed estimates might be considered.

This is an intriguing possibility except that what I need is an estimator that performs well if the true value of gamma is large (close to 1) not small.