Concept - Bagging

Bootstrap aggregation (s.350).

1. Draw $m$ bootstrap replications $T_1..T_m$ of the training set (Concept - Bootstrapping).
2. Train $m$ classifiers $C_1..C_m$, one per bootstrap sample.
3. Each predicts $P_i$ for new data; combine outputs by voting (Concept - Majority Voting) -> final prediction $P_f$.

• Decreases error by decreasing the variance (s.348, s.350); works best for unstable learners (Concept - Stable and Unstable Learners).
• Variants: Concept - Random Forest (randomly select features) and ExtraTrees (randomized top-down split) (s.350).

flowchart LR
    S[Training set] --> T[Bootstrap samples T1..Tm] --> C[Classifiers C1..Cm] --> P[Predictions P1..Pm] --> V{Voting} --> F[Final prediction Pf]

Related

Concept - Ensemble Learning | Concept - Boosting | Concept - Stacking

Source chunk: 03 Text Classification - Chunk 12 (s.331-360)