Concept - Likelihood

Definition

In Gaussian Naive Bayes, the likelihood $L(x\mid \text{class})$ of a feature value is “the y-axis coordinate on the curve that corresponds to the x-axis coordinate” (s.198), i.e. the height of the class’s Gaussian at the observed value.

Example (s.197-206)

• $L(\text{popcorn}=20\mid \text{Loves})=0.06$ (s.202)
• $L(\text{soda pop}=500\mid \text{Loves})=0.004$ (s.203)
• $L(\text{candy}=25\mid \text{Loves})$ = a really, really small number (s.206)
• Class score = prior $\times$ product of these (s.195-201).

Related

Concept - Prior Probability, Concept - Gaussian Naive Bayes, Concept - Underflow

Source chunk: 03 Text Classification - Chunk 07 (s.181-210)

Discrete word likelihoods in the spam example (s.61-64)

• Source: 03 Text Classification - Chunk 03 (s.61-90).
• Word probabilities per class: $p(\text{Dear}\mid N)=0.47$, $p(\text{Friend}\mid N)=0.29$, $p(\text{Lunch}\mid N)=0.18$, $p(\text{Money}\mid N)=0.06$; $p(\text{Dear}\mid S)=0.29$, $p(\text{Friend}\mid S)=0.14$, $p(\text{Lunch}\mid S)=0.00$, $p(\text{Money}\mid S)=0.57$ (s.62).
• Terminology Alert (s.63): because these are probabilities of discrete, individual words (not something continuous like weight or height), they are also called Likelihoods. s.64: the two terms are interchangeable in this case (“don’t sweat it”).
• Used by multiplying with the prior: $p(N)p(\text{Dear}\mid N)p(\text{Friend}\mid N)=0.09$; $p(S)p(\text{Dear}\mid S)p(\text{Friend}\mid S)=0.01$ (s.77, s.90). See Concept - Prior Probability, Concept - Posterior Probability.

Likelihoods in the No-Love score (s.225-231)

$L(\text{popcorn}=20\mid \text{No Love})$ = a tiny number (value not given; log = -33.6), $L(\text{soda pop}=500\mid \text{No Love})=0.000079$ (log -9.45), $L(\text{candy}=25\mid \text{No Love})=0.02$ (log -3.91) (s.225-231, s.235). Read from the red Gaussian curves at the X marks. See 03 Text Classification - Chunk 08 (s.211-240).

Word likelihoods in the spam example (03 Text Classification - Chunk 04 (s.91-120))

• $p(\text{Money}\mid S)=0.57$ vs $p(\text{Money}\mid N)=0.06$ makes Money a spam indicator (s.100). $p(\text{Lunch}\mid S)=0.00$ (unseen in spam) zeroes the whole spam score (s.109); see Concept - Zero Probability Problem.

From chunk 02 (s.36-60)

For text, the likelihood of each word is a count ratio per class: see Concept - Word Likelihood (e.g. $p(\text{Dear}\mid \text{Normal})=8/17=0.47$).

From chunk 01 (s.1-30)

Likelihood = probability of observing the sample under a distribution; with independence $\mathrm{Pr}[x_1..x_m]=\prod p(x_i)$ (s.9). In Bayes’ rule $P(D|h)$ is “likelihood of $D$ given $h$” (s.13). Concept - Maximum Likelihood Estimation maximizes it; Concept - Maximum A Posteriori multiplies it by the prior.

Source chunk: 03 Text Classification - Chunk 01 (s.1-30)