
Re: information theory?
Posted:
Nov 2, 2011 4:47 AM


On 01/11/2011 17:29, RichD wrote: > On Nov 1, Chris<chris.sant...@gmail.com> wrote: >> Do you have a specific question? Or just want to lurk and learn? > > I'd like to lurk someplace, where there are discussions > of current topics. It's such a rich field, there are always > new ideas. > >> Information Theory was the hardest class I took in grad school. So >> abstract, and so much probability.... but it's a cool topic >> nonetheless. > > I read Shannon's paper long go, and found it fascinating > ever since. In the most general sense, it's applicable > to so many areas. For instance, people have recast > thermodynamics as information processing. Or, you > could model the entire universe as a computer, and > information theory applies. It even describes human > nature  manipulating symbols is what separates us > from the apes. > >>>> Are there any boards, or private blogs, dedicated >>>> to discussion of information theory?
You don't really say which part of information theory you are interested in  but some of the more interesting modern developments stem from Bayes theorem and the principle of maximum entropy.
You could do worse than look for the collected works of Ed Jaynes  it used to be available online at one time but now it is published.
http://www.amazon.com/TJaynesProbabilityStatisticsStatistical/dp/0792302133
David MacKays book Information Theory, Inference and Learning might be a cheaper alternative.
Not an easy read and assumes higher mathematics as a starting point.
 Regards, Martin Brown

