Concept - Bias and Variance
Slide-based statement (s.145, 03 Text Classification - Chunk 05 (s.121-150)): “In Machine Learning lingo, we’d say that by ignoring relationships among words, Naive Bayes has high bias… but because it works well in practice, Naive Bayes has low variance.”
- Naive Bayes ignores word order, grammar and phrases (s.141-143), so it makes strong simplifying assumptions: high bias.
- Yet it separates normal messages from spam surprisingly well (s.144): low variance.
Related
Formal definitions and tradeoff (s.345-346, 03 Text Classification - Chunk 12 (s.331-360))
Bias = difference between the average prediction and the correct value; variance = variability of the prediction for a given point. Error = Bias² + Variance + Irreducible Error. See Concept - Bias-Variance Tradeoff.