Can you recommend me a list of good Probability Books for self-studying, with good explanations and introductions for Information Theory and not for the typical statistical subjects?
Probability books useful for Information Theory?
3 Answers
Most of the references above are basic texts on Information Theory and not necessary probability theory based. I'm guessing that you're looking for probability theory texts with some emphasis on information theory in preparation for delving more deeply into information theory. For that I'd recommend taking a look at Paul Pfeiffer's Concepts of Probability Theory or either of Alfred Renyi's two books Probability Theory or Foundations of Probability. All either mention information theory specifically or have presentations influenced or working toward the subject in general. All three are reprints that can be had fairly cheaply from Dover Publications.
Alternately, for a bit more money, you might consider Alfredo Leon-Garcia's fairly standard text Probability, Statistics, and Random Processes for Electrical Engineers. As he's also subsequently written texts for communication engineering, he certainly has a very information theory/comm theory flavored presentation. The most recent edition is the 3rd, but the 2nd edition is substantially the same for less money on the used market.
I think all four above are relatively good for self-study, though Renyi's presentations are a tad more sophisticated mathematically and may seem more dense to the beginner.
There is the book "Information Theory, Inference, and Learning Algorithms" by David Mackay and available for free on this website
http://www.cs.toronto.edu/~mackay/itila/p0.html
However, I recommend that you complement it by a more traditional approach such as
Texts
For an introductory text, try Applebaum's Probability and Information: an Integrated Approach.
You might be interested in a classic text by Woodward: eg. Probability and Information Theory with Applications to Radar.
A good and "cheap" book ($3.99 USD) book by Dover Publications provides a great An Introduction to Information Theory. Another book from Dover Publications is here, (same title).. Both include discussion of probability in the context of information theory.
Online resources addressing probability in the context of information theory:
You can download Information Theory Primer, available in pdf, as well as in LaTeX from the author's (Tom Schneider) website.
-
0The first four links have expired: could you post the last names of the authors so that it could be possible to find the references elsewhere? – 2016-04-30