Concept - Bernoulli Naive Bayes (BNB)

Input: binary indicators (word present = 1, word absent = 0) (s.148).
Assumption: each word is a binary feature (s.148).
Effect: only presence/absence of a word matters, not how many times it appears (s.148).
Typical use cases: short texts (tweets, product reviews); when word repetition is less informative (s.148).
Example: “cheap meds online” -> $[0, 1, 1, 1, 0, 0, 0, 0, 0]$ ; identical to the multinomial vector here because no word repeats, but “in longer documents the difference matters”: MNB counts how many times, BNB only whether at least once (s.150).

Worked example and formulas (s.156-161, from 03 Text Classification - Chunk 06 (s.151-180))

Smoothed likelihood: $P (w = 1 ∣ c) = \frac{N _{c, w} + 1}{N _{c} + 2}$ where $N_{c, w}$ = number of documents of class $c$ containing $w$ and $N_{c}$ = documents in class (=2, denominator 4); absence: $P (w = 0 ∣ c) = 1 - P (w = 1 ∣ c)$ (s.156-157).
Document likelihood over the whole vocabulary: $P (x ∣ c) = \prod_{w \in V} P (w = 1 ∣ c)^{x_{w}} [1 - P (w = 1 ∣ c)]^{1 - x_{w}}$ (s.158).
For “cheap project meds”: $P (x ∣ s p am) = \frac{9}{64} \cdot \frac{81}{1024} = \frac{729}{65536} \approx 0.0111$ ; $P (x ∣ ham) = \frac{1}{32} \cdot \frac{9}{512} = \frac{9}{16384} \approx 0.000549$ (s.159).
Scores with priors 0.5: spam 0.00555, ham 0.000275 -> spam, ratio about 20x; normalized 0.953 vs 0.047 (s.159-161). More confident than MNB (95% vs 82%).
Key difference from MNB: absent words are explicitly used in the product (s.158).