Concept - Additive Smoothing

Problem (s.29, “New Word Problem”): if a word never occurs in category $h$, $p(d_i=\text{newword}|h)=0$ and the whole product $p(\mathbf{d}|h_i)=p(d_1|h_i)\cdots 0\cdots p(d_n|h_i)=0$.

Solution (s.30): $p(d_i=w|h_j)=\frac{count(w,h_j)+\alpha }{{\sum }_{d\in \mathbb{V}}count(d_i,h_j)+\alpha V}$ $\alpha$ = smoothing parameter; $V$ = vocabulary size.

Interpretation (s.30): a weighted estimation of the relative frequency $\frac{count(w,h_j)}{\sum count(d_i,h_j)}$ and the uniform probability $\frac{1}{V}$.

See also the later treatment in Concept - Laplace Smoothing (the $\alpha =1$ case; this link is the course’s later terminology). Used in Concept - Naive Bayes.

Related

Concept - Maximum Likelihood Estimation, Concept - Naive Bayes, Concept - Laplace Smoothing

Source: 03 Text Classification - Chunk 01 (s.1-30)