Concept - Stable and Unstable Learners
| Stable | Unstable | |
|---|---|---|
| Examples (s.341-342) | kNN, linear regression, SVM | decision trees, neural networks |
| Reaction to slightly modified training set | very similar results | drastically different boundaries/predictions |
| Variance | low | high |
| Benefit from ensembles (bagging) | marginal | large (averaging reduces variance and stabilizes predictions) |
Takeaway (s.342): unstable learners (trees, NNs) -> ensembles stabilize them and boost accuracy; stable learners (kNN, linear models) -> only marginal improvements.
Note: s.341 names kNN as the stable example; s.342 adds linear regression and SVM.
Related
Concept - Ensemble Learning | Concept - Bagging | Concept - Bias-Variance Tradeoff | Concept - Weak and Strong Learners
Source chunk: 03 Text Classification - Chunk 12 (s.331-360)