Concept - Gain Ratio

Gain Ratio

Problem (s.321): Concept - Information Gain is biased toward attributes with many values. Using Day as attribute splits the data perfectly into subsets of size 1 (D1…D14) but won’t work for new data.

Fix: use GainRatio as criterion:

$GainRatio(S,A)=\frac{Gain(S,A)}{SplitInformation(S,A)}$ $SplitInformation(S,A)=-{\sum }_{i\in Values(A)}\frac{|S_i|}{|S|}{\log }_2\frac{|S_i|}{|S|}$

SplitInformation penalizes attributes with many values. Notation: $A$ candidate attribute; $i$ possible values of $A$; $S$ set of examples $\{X\}$; $S_i$ subset where $X_A=i$.

Related

Concept - Information Gain | Concept - ID3 | Concept - Decision Tree

Appears in

• 03 Text Classification - Chunk 11 (s.301-330)