0
$\begingroup$

I'm an engineering student who has taken one undergraduate course in probability theory, but that's all my exposure so far. I'm trying to get into machine learning and need to develop more of a background for this purpose.

I hear about different types of probability theory and where it builds from, but the connections between these are extremely hazy to me. For example, I've heard of probability theory being grounded on top of measure theory, but have also heard of it based on logic, have heard the terms Kolmogorov and de Finetti probability theory, but can't make sense of what's going on here. Is there different factions of probability theorists who disagree with each other or do they all play a part in some coherent framework? Would anyone give me a beginner's synopsis of the relationships and layout between these things?

  • 1
    I think "Kolmogorov probability theory" means the measure theory framework. You don't need any of that to have an understanding of machine learning. What you really need is "naive" probability and statistics using elementary calculus. – 2017-02-10
  • 0
    @mathematician I picked this book up by Jaynes called probability theory the logic of science and was also wondering if this is compatible or incompatible with measure theory? – 2017-02-10
  • 2
    @Jake It seems that Jaynes' book is introduction to probability + mathematical statistics. The Measure theoretic framework is mentioned only in the appendix. – 2017-02-10
  • 1
    Most people dealing with probability foundations are using measure theory as their foundation. A basic philosophical tenet of the subject is that it should not really matter what your foundation actually is, pretty much everything should go through the same way. A merit of the measure theoretic approach is that you need an extremely small axiomatic system to get off the ground. – 2017-02-10
  • 1
    A drawback of the measure theoretic approach is that the measure spaces studied in probability theory are for the most part not "concrete": in other words, their elements don't really *mean* anything. This makes the whole proceeding have some additional, arguably unnecessary, abstraction. Still, probability theory went through a veritable renaissance upon the widespread adoption of the Kolmogorov axioms in the early 20th century. – 2017-02-10

1 Answers 1

1

Kolmogorov probability theory defines the probability measure on $\sigma$-algebras, and the measure is countable additive. This is what pretty much everybody uses.

De Finetti defines the the probability measure on algebras, and the measure is only finitely additive. It is an alternative theory, but beyond you would actually need.

My suggestion would be to start with basic books, such as Sheldon and Ross, and upgrade to more advanced things (Grimmet is good for engineering students, Billingsley is good too) only after you have become confortable with conditioning, independence, joint pdf's,etc.