Is there a mathematically rigorous book giving an introduction to boosting, etc. A book that is rigorous like "A Course in Probability Theory" by Kai Lai Chung.
Rigorous book on bootstrapping, boosting, bagging, etc.
3 Answers
Shapire's Boosting: Foundations and Algorithms is IMHO a very didatic and rigorous book about the subject.
Elements of Statistical Learning is available online.
There is a chapter on Boosting (chapter 10).
Does it satisfies your definition of rigorous? If by "rigorous" you mean "measure theoretic," then no.
Along with the text that @William proposed (it's a great reference), for bootstrapping it's hard to beat:
Efron 1987, The Jackknife, the Bootstrap, and Other Resampling Plans.
Efron and Tibshirani 1994, An Introduction to the Bootstrap.
Hall 1995, The Bootstrap and Edgeworth Expansion is more rigorous than the above. Upon a cursory glance, it appears to dive pretty deep into the theoretical details of the bootstrap.
Specifically for boosting, Robert Schapire has a list of references to read found here. And for bagging, Martin Sewell has a list of references here. His site has a bunch of reference lists for a number of machine learning topics (see here).
-
0Could you elaborate on which of these many sources are mathematically rigorous? I don't like it when I read about some results and don't understand every little detail about why it's true. – 2012-03-06
-
0@user782220 For the boosting list, I'd start with the 'overview' papers he lists. Schapire's papers seem to be on the more rigorous end of the spectrum. For bagging, Breiman's papers are the place to start. Also, the bagging list has a few *Annals of Statistics* papers. These are (obviously) from a statistical point of view, but are quite 'rigorous.' As for Efron's books, I don't know how 'rigorous' you're hoping to get, but you really can't go wrong with the 1994 text (IMO). Another good bootstrap reference is [Davison & Hinkley](http://tinyurl.com/72a9nvg). – 2012-03-06
-
0@user782220 If you're still not satisfied, the [Stats SE](http://stats.stackexchange.com/) has a good number of machine learning folks who may offer a different perspective. – 2012-03-06
-
0@user782220 Upon more searching, I would surmise that Peter Hall's [bootstrap book](http://www.amazon.com/Bootstrap-Edgeworth-Expansion-Springer-Statistics/dp/0387945083) is probably the most rigorous one out there. – 2012-03-06