Concept - Bias-Variance Tradeoff

• Bias (s.345): difference between the average prediction of our model and the correct value we are trying to predict.
• Variance (s.345): variability of model prediction for a given data point, i.e. the spread.
• Formula (s.346): $\text{Prediction Error}={\text{Bias}}^2+\text{Variance}+\text{Irreducible Error}$
• Bullseye figure (s.345): high bias + low variance = underfitting; low bias + high variance = overfitting; low bias + low variance = ideal.
• Complexity plot (s.346): bias falls, variance rises with model complexity; test error is U-shaped with the minimum at the optimal balance (left = underfitting/high bias, right = overfitting/high variance).
• Why ensembles (s.346): reduce bias (boosting) and reduce variance (bagging).

Related

Concept - Bias and Variance (Naive Bayes example, earlier chunk) | Concept - Overfitting | Concept - Underfitting | Concept - Ensemble Learning

Source chunk: 03 Text Classification - Chunk 12 (s.331-360)