Concept - Ensemble Learning

Definition (s.341): techniques that generate a group of base learners which, when combined, have higher accuracy.

Intuition (s.339): “Three kids can defeat a master”.
Evidence (s.340): on the SQuAD2.0 leaderboard, top systems are ensembles; ALBERT (ensemble) EM 89.731 / F1 92.215 beats ALBERT (single) 88.107 / 90.902 and human performance 86.831 / 89.452.
Distinctions: Concept - Weak and Strong Learners; Concept - Stable and Unstable Learners.
Why (s.346): reduce bias, reduce variance; prediction error = Bias² + Variance + Irreducible error (Concept - Bias-Variance Tradeoff).
Common techniques (s.348):

Technique	Goal	Examples
Concept - Bagging	reduce variance	Random Forest
Concept - Boosting	reduce bias	AdaBoost, XGBoost, Gradient Boosted Decision Trees
Concept - Stacking	increase prediction accuracy	Mlxtend library

Khaled Notes