Bias Variance Tradeoff#
Bias : Error due in the hypothesis or model to fit the data.
Variance : Error due to fitting random noise in the data. How the model varies with different training set.
Bias Variance Decomposition#
\[
y = y^* + \epsilon
\]
\[\begin{split}
\begin{align*}
E\big[ L(y, \hat y) \big] &= E\big[ (y - \hat y)^2 \big]\\
&= \left(E\big[ \hat y \big] - y^*\right)^2 + \text{Var}\big[ \hat y \big] + \text{Var}\big[ y \big]
\end{align*}
\end{split}\]
Bias
Variance
Irreducible Error