Khaled Notes

Concept - Weak and Strong Learners

Properties1
tagsmidterm, text-web-mining

1 min read

Concept - Weak and Strong Learners

  • Weak learner (s.343): performs only slightly better than random guessing. Example: a decision stump (1-level tree) with 55-60% accuracy on a binary task (vs 50% random). Weak on their own, but combined in an ensemble they can form a strong learner -> basis of Concept - Boosting (AdaBoost, Gradient Boosting): train many weak learners sequentially, each correcting the mistakes of the previous one.
  • Strong learner (s.343): already performs well alone, low error (random forest, deep neural network, SVM with a good kernel). Can still be ensembled (e.g. stacking in Kaggle competitions) for extra performance.
  • Summary (s.344): weak learner -> Boosting; unstable strong learners -> Bagging/Stacking; for strong learners ensembles give diminishing returns.

Concept - Ensemble Learning | Concept - Stable and Unstable Learners | Concept - Boosting | Concept - Bagging | Concept - Stacking

Source chunk: 03 Text Classification - Chunk 12 (s.331-360)

Graph View

Backlinks