Concept - Posterior Normalization

Definition

Converting the raw (unnormalized) Naive Bayes scores into proper posterior probabilities by dividing each class score by the sum of the scores over all classes (s.161):

$P(c\mid x)=\frac{score(c)}{{\sum }_{c^{\prime }}score(c^{\prime })}$

Worked values (s.161)

Model	Spam score	Ham score	Total	$P(spam\mid x)$	$P(ham\mid x)$
Concept - Multinomial Naive Bayes	0.00133	0.000296	0.001626	0.818 (~82%)	0.182 (~18%)
Concept - Bernoulli Naive Bayes	0.00555	0.000275	0.005825	0.953 (~95%)	0.047 (~5%)

When to use

• Without normalization we only compare magnitudes (ratios ~4.5 for MNB vs ~20 for BNB) - enough to pick the most likely class.
• With normalization we get actual posterior probabilities - needed for interpretable statements such as “spam with 95% certainty” (s.161).
• Both models predict spam, but BNB is more confident (95% vs 82%).

Related

Concept - Posterior Probability, Concept - Naive Bayes, Concept - Likelihood, Concept - Prior Probability

Source chunk: 03 Text Classification - Chunk 06 (s.151-180)