Entropy

$E (X) = - \sum_{i = 1}^{K} p (X = X_{i}) lo g_{2} p (X = X_{i})$

Intuition (s.311): uncertainty of X, information contained in X, expected information bits required to represent X.

Conditional entropy (s.311)

$E (X ∣ Y) = \sum_{i} p (Y = Y_{i}) E (X ∣ Y = Y_{i})$ - given y, how much uncertainty remains in X.

Root $S = [9 +, 5 -]$ : $E = - \frac{9}{14} lo g_{2} \frac{9}{14} - \frac{5}{14} lo g_{2} \frac{5}{14} = 0.940$ .
Humidity high $[3 +, 4 -]$ : 0.985; normal $[6 +, 1 -]$ : 0.592.
Outlook sunny $[2 +, 3 -]$ : 0.971; overcast $[4 +, 0 -]$ : 0.0 (pure); rainy $[3 +, 2 -]$ : 0.971.