The only explanations I've seen of the bias/variance tradeoff rely on rewriting the squared error of an estimator as the sum of bias and variance terms. How does the bias/variance tradeoff work if the loss function is not squared error? Thanks in advance!
Connection between bias/variance tradeoff and squared error
1
$\begingroup$
probability
statistics
2 Answers
0
MSE=bias$^2$ +variance So the tradeoff is obvious for MSE. If your loss function is some other function of bias and variance then there will be a variance-bias tradeoff for it too. Otherwise there is none for varying the expected loss.
-
0Typically, we can't minimize bias and variance at the same time, can we? – 2013-12-24
0
@ loganecolss
Yes, we can not reduce both bias and variance simultaneously. Decrease in the one will lead to increase in the other, that's why we call it 'bias-variance trade-off'