This is just a quick recommendation for a holiday gift or holiday reading - Nate Silver's "The signal and the noise." It is subtitled, Why so many predictions fail and some of them don't.
Nate Silver was popular in this last election season because he had the best record of prediction in 2012. And this year, he hit all 50 states correctly, predicting Presidential outcome. - His method is, mainly, to weight other people's polls by their previous success. (But - this is not strongly spelled out - he also makes subjective choices of whose polls to attend to.)
This is a pretty easy read, I hope, even for people who are not mathematicians, and it does have a number of good teaching examples. He goes into some detail on "over-fitting."
Silver made his first fortune in sports betting, and some fame by improving on predictions for Who will make it among minor league baseball players. And he added to his fortune, for a while, playing on-line poker. These contribute to his examples.
He does spend some time explaining why you can't really beat the stock market. (Transaction fees, on top of all else.)
The chapter on climate change includes a nice historical recap of the science, some of which I was not famililar with. He does endorse the usual conclusion.
A big reason "why predictions fail", in addition to technical ones, is that the prognosticators tend to pick-and-choose among the plentiful, new data, so that they ignore what doesn't fit some previous model -- instead of learning from it. In terms of the title -- they confuse signal and noise, erring in both directions.
I particularly liked the introduction. The original "information explosion" is the one that followed the invention of the printing press in the 15th century. What it led to, before the Industrial Revolution really took off 300 years later, was a centuries of religious wars. - It took a long time for people to sort out what they could agree on