Drexel dragonThe Math ForumDonate to the Math Forum



Search All of the Math Forum:

Views expressed in these public forums are not endorsed by Drexel University or The Math Forum.


Math Forum » Discussions » sci.math.* » sci.stat.math.independent

Topic: Data Mining, Data Partitioning, Prediction and Inference
Replies: 2   Last Post: Mar 11, 2011 4:39 PM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
root

Posts: 50
Registered: 8/26/09
Re: Data Mining, Data Partitioning, Prediction and Inference
Posted: Mar 11, 2011 11:44 AM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

slutsky_fan <matttbogard@gmail.com> wrote:
> Typically to deal with generalization error, when building forecasts
> or predictive models, data will be partitioned at least into training
> and validation data sets. What if I'm primarily just concerned with
> making inferences about certain variables in the model (evaluating
> directions of co-efficients, odds ratios etc.)? Wouldn't I actually be
> better off NOT PARTITIONING the data and using the whole data set to
> get better co-efficient estimates?
>
> In other words, If I've just developed a predictive model, and all I
> want from it are predicted probabilities, I should probably partition
> the data and validate my results. But then, when it comes to making
> actual inferences about the relationships between the variables I
> should probably re-build the model on a complete non-partitioned data
> set. Is this a good or bad methodology?
>
> Thanks.


In my experience the data are grouped into three classes:
1. those known to have some property
2. those known not to have that property
3. all the rest for which having/not-having is not known.

The "training" is done using groups 1,2 and is then
applied to group 3.



Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© Drexel University 1994-2014. All Rights Reserved.
The Math Forum is a research and educational enterprise of the Drexel University School of Education.