Concept - Posterior Probability

Definition

The probability that a message belongs to a class given the words it contains, e.g. $p(N\mid \text{Dear Friend})$ (s.79).

Details

• The Naive Bayes score is only proportional to it: $0.67\times 0.47\times 0.29=0.09\propto p(N\mid \text{Dear Friend})$ (s.79-80).
• The spam score $0.33\times 0.29\times 0.14=0.01$ is likewise proportional to $p(S\mid \text{Dear Friend})$ (s.90, by the same procedure).
• The two scores (0.09 and 0.01) are compared to decide the class; they do not sum to 1, so they are not themselves probabilities.

Added context

The name “posterior probability” is not printed on s.61-90; the slides only write $p(N\mid \text{Dear Friend})$ and “proportional to”.

Related

Concept - Prior Probability, Concept - Likelihood, Concept - Naive Bayes

Source chunk: 03 Text Classification - Chunk 03 (s.61-90)

Turning scores into posteriors (s.161; 03 Text Classification - Chunk 06 (s.151-180))

Divide each class score by the sum over classes: see Concept - Posterior Normalization. Example: MNB 0.818 spam / 0.182 ham; BNB 0.953 / 0.047.

From chunk 01 (s.1-30)

$P(h|D)$ = posterior probability of $h$ given evidence $D$ (s.13), $=P(D|h)P(h)/P(D)$ (Concept - Bayes Rule). Maximizing it gives Concept - Maximum A Posteriori (s.18-19). Bar charts s.15-17 show it sharpening as data arrives.

Source chunk: 03 Text Classification - Chunk 01 (s.1-30)