03 Text Classification - Chunk 07 (s.181-210)

Overview

This chunk is a worked, step-by-step example of Concept - Gaussian Naive Bayes using the “Loves Troll 2 / Does not Love Troll 2” movie-fan example (the slides carry a “StatQuest”-style cartoon logo; the example is about people’s daily intake of popcorn, soda pop and candy). The training data (16 people, 8 per class) is summarised by one Gaussian (normal) curve per feature per class (s.181). A new person arrives (20 g popcorn, 500 ml soda pop, 25 g candy) (s.182-185) and must be classified (s.186). The classifier starts from Concept - Prior Probability guesses (s.187-193), then multiplies by the Concept - Likelihood of each feature value read off the class’s Gaussian curve (s.194-203). Because the product of many small numbers becomes tiny, the slides introduce the log trick to avoid Concept - Underflow (s.204-210). The Loves Troll 2 score is built up here; the comparison with the “Does not love” score continues in the next chunk.

Detailed notes

1. Setting: training data and Gaussian curves (s.181)

• Two classes of people: Loves Troll 2 (green) and Does not Love Troll 2 (red) (s.181).
• Training data tables have three numeric features: Popcorn (grams), Soda Pop (ml), Candy (grams) (s.181).
  • Loves Troll 2 rows shown: 24.3 / 750.7 / 0.2; 28.2 / 533.2 / 50.5; “etc.” (s.181).
  • Does not Love Troll 2 rows shown: 2.1 / 120.5 / 90.7; 4.8 / 110.9 / 102.3; “etc.” (s.181).
• For each feature there is one plot with two curves: a green Gaussian (Loves) and a red Gaussian (Does not love) (s.181).
  • Popcorn: red narrow curve at low values, green wide curve at high values (clearly separated) (s.181).
  • Soda Pop: red and green curves are wide and overlap (s.181).
  • Candy: green is a tall narrow peak at low values, red a very flat wide curve at higher values (s.181).
• Definition (s.181): “Gaussian Naive Bayes is named after the Gaussian distributions that represent the data in the Training Dataset.”

2. The new person to classify (s.182-186)

• “Now someone new shows up…” (s.182).
• They say they eat 20 grams of popcorn (s.183, an X is marked on the Popcorn axis), drink 500 ml of soda pop (s.184, X on Soda Pop axis, near the right tail of the red curve and at the peak of the green curve) and eat 25 grams of candy every day (s.185, X on Candy axis, where the green curve is essentially flat at ~0 while the red curve has a little height).
• Question (s.186): “Let’s use Gaussian Naive Bayes to decide if they love Troll 2 or not.” Two arrows with ”???” point to the two classes.

3. Step 1: prior probabilities (s.187-193)

• “The first thing we do is make an initial guess that they Love Troll 2.” (s.187)
• “This guess can be any probability that we want, but a common guess is estimated from the training data.” Written as $p(\text{Loves Troll 2})$ (s.188).
• Example (s.189-190): 8 of the 16 people in the training data loved Troll 2, so $p(\text{Loves Troll 2})=\frac{8}{8+8}=0.5$ (s.189; the 0.5 is boxed in red on s.190).
• The value is stored beside the class label: $p(\text{Loves Troll 2})=0.5$ “so we don’t forget” (s.191).
• Likewise $p(\text{No Love Troll 2})=\frac{8}{8+8}=0.5$ (s.192), stored beside the “Does not Love” group (s.193).
• Terminology Alert (s.194): “The initial guesses are called Prior Probabilities.” See Concept - Prior Probability.

4. Step 2: building the Loves Troll 2 score (s.195-203)

• Slide 195: “Now, the score for Loves Troll 2 is…” (heading “Loves Troll 2 Score =”).
• It starts with the initial guess $p(\text{Loves})$ (s.196).
• “…times the Likelihood that they eat 10 grams of popcorn, given that they Love Troll 2.” The formula on the slide reads $\times L(\text{popcorn}=20\mid \text{Loves})$ (s.197).

  Added context

  The slide text says “10 grams” but the formula and the earlier slides (s.183) say 20 grams; this is an inconsistency in the slide wording. UNCLEAR (slide 197) which was intended; the formula and s.183 use 20 g.

• NOTE (s.198): “The Likelihood is the y-axis coordinate on the curve that corresponds to the x-axis coordinate.” (The figure draws a horizontal arrow from the point on the green curve above the X to the y-axis; a dotted red vertical line joins the X on the axis to the curve. A “View correction” overlay appears in the top right of this slide: UNCLEAR (slide 198) what correction it refers to.)
• Multiply by the likelihood that they drink 500 ml of soda pop given Loves: $\times L(\text{soda pop}=500\mid \text{Loves})$ (s.199). The point on the green soda-pop curve is at its peak.
• …times the likelihood that they eat 25 g candy given Loves: $\times L(\text{candy}=25\mid \text{Loves})$ (s.200). The point on the green candy curve is far out in the flat tail, so the y-value is nearly 0.
• Summary of the score structure (s.201): initial guess 0.5, then three likelihood factors.
• Substituting numbers:
  • Likelihood for popcorn = 0.06 (s.202).
  • Likelihood for soda pop = 0.004 (s.203, s.204).
  • Likelihood for candy = ”…”, then ”× a really, really small number” (s.205-206; the slide does not give the number).
• Running product on the slides: $0.5\times 0.06\times 0.004\times (\text{a really, really small number})$ (s.206).

5. Underflow and the log trick (s.207-210)

• NOTE (s.207): “When we get really, really small numbers, it’s a good idea to take the log() of everything to prevent something called Underflow.”
• “The general idea of Underflow is…” (s.208) “…every very computer has a limit to how close a number can get to 0 before it can no longer accurately keep track of that number.” (s.209)
• “When a number gets smaller than that limit, we run into Underflow problems and errors occur.” (s.210)
• See Concept - Underflow (the continuation, how the log is applied to the score, comes in the next chunk; not shown in these slides).

Figures & diagrams

• s.181 Two tables (green = Loves, red = Does not Love) next to three stacked Gaussian plots (Popcorn, Soda Pop, Candy), each with a green and a red curve. Takeaway: each class’s data per feature is modelled by a Gaussian; features differ in how well they separate the classes (popcorn well, soda pop overlapping, candy very different widths).
• s.182-185 The same three plots with a grey person silhouette and an X marking the new person’s value on each feature axis (popcorn 20 g, soda pop 500 ml, candy 25 g), highlighted one feature at a time.
• s.186 Group icons (8 green people, 8 red people) with two ”???” arrows from the new person: which class?
• s.187-194 Group icons plus the priors $p(\text{Loves Troll 2})=0.5$ and $p(\text{No Love Troll 2})=0.5$ with the fraction $\frac{8}{8+8}$ (8 in green, 8 in red).
• s.195-206 Score being built on the left (prior $\times$ likelihoods), curves on the right with horizontal arrows from the curve point to the y-axis showing the likelihood (y-coordinate) for popcorn, soda pop and candy; vertical dotted red lines connect the X on the axis to the curve (s.198-205).
• s.207-210 Same background; text boxes explaining underflow.

flowchart LR
    N[New person<br/>popcorn 20 g, soda 500 ml, candy 25 g] --> P["Prior p(Loves) = 8/16 = 0.5"]
    P --> L1["x L(popcorn=20 | Loves) = 0.06"]
    L1 --> L2["x L(soda pop=500 | Loves) = 0.004"]
    L2 --> L3["x L(candy=25 | Loves) = really small"]
    L3 --> S["Loves Troll 2 score"]
    S -. tiny numbers .-> U["Underflow risk -> take log()"]

Key terms

• Concept - Gaussian Naive Bayes
• Concept - Prior Probability
• Concept - Likelihood
• Concept - Underflow
• Concept - Naive Bayes (general method; also see the previous chunks of this deck)

Formulas

• Prior estimated from training data (s.189): $p(\text{Loves Troll 2})=\frac{8}{8+8}=0.5$; $p(\text{No Love Troll 2})=\frac{8}{8+8}=0.5$ (s.192).
• Score for a class (s.195-201): $\text{Score(Loves)}=p(\text{Loves})\times L(\text{popcorn}=20\mid \text{Loves})\times L(\text{soda pop}=500\mid \text{Loves})\times L(\text{candy}=25\mid \text{Loves})$
• Numeric values: $0.5\times 0.06\times 0.004\times (\text{very small number})$ (s.202-206).
• Likelihood = y-axis value of the class’s Gaussian curve at the observed x value (s.198).

Likely exam questions

1. Why is it called Gaussian Naive Bayes? (s.181) It is named after the Gaussian distributions that represent the training data (one curve per feature per class).
2. What is a prior probability and how is it estimated? (s.188-194) The initial guess for a class before looking at the features; commonly estimated from the training data as the fraction of training examples in the class, e.g. 8/(8+8) = 0.5.
3. What is the likelihood in Gaussian Naive Bayes? (s.198) The y-axis coordinate (curve height) of the class’s Gaussian at the observed feature value.
4. Describe the steps to compute the “Loves Troll 2” score. (s.195-201) Start with the prior, multiply by the likelihood of each feature value (popcorn, soda pop, candy) given the class.
5. Compute the score for the example. (s.202-206) $0.5\times 0.06\times 0.004\times$ (a really small number for candy).
6. What is underflow, and how do we avoid it? (s.207-210) Computers cannot represent numbers closer to 0 than a limit; products of many small likelihoods fall below it and cause errors; take the log() of everything.
7. Why is the candy likelihood so small for the Loves class? (s.200, s.206) The new person’s 25 g sits in the far tail of the green candy curve, which is a narrow peak at low values.

Common confusions

• Likelihood vs probability: here the likelihood is read as the curve height (y-coordinate) at the x value, not an area (s.198).
• Prior vs score: the prior (0.5) is only the first factor; the score multiplies it by the three likelihoods (s.195-201).
• Slide inconsistency: s.197 says “10 grams of popcorn” in the text but the formula and s.183 use 20 g.
• Log trick timing: the slides only introduce the need to take the log (s.207-210); the applied calculation follows in the next chunk.

Slide coverage

Slide	Content	Covered in
181	Training tables + Gaussian curves; definition of Gaussian Naive Bayes	Detailed notes 1; Figures
182	”Now someone new shows up…”	Detailed notes 2; Figures
183	New person eats 20 g popcorn (X on popcorn plot)	Detailed notes 2; Figures
184	Drinks 500 ml soda pop	Detailed notes 2; Figures
185	Eats 25 g candy daily	Detailed notes 2; Figures
186	Use Gaussian NB to decide: loves Troll 2 or not?	Detailed notes 2
187	First: make an initial guess that they love Troll 2	Detailed notes 3
188	Guess = any probability, commonly estimated from training data	Detailed notes 3
189	8 of 16 loved Troll 2, so p = 0.5	Detailed notes 3; Formulas
190	Same, 0.5 highlighted	Detailed notes 3
191	Store p(Loves Troll 2) = 0.5	Detailed notes 3
192	p(No Love Troll 2) = 8/(8+8) = 0.5	Detailed notes 3
193	Store p(No Love Troll 2) = 0.5	Detailed notes 3
194	Terminology alert: prior probabilities	Detailed notes 3
195	Score for Loves Troll 2 is…	Detailed notes 4
196	…the initial guess p(Loves)…	Detailed notes 4
197	…times likelihood of popcorn (20 / “10 grams” text)	Detailed notes 4
198	Note: likelihood = y-axis coordinate (View correction overlay)	Detailed notes 4
199	Times likelihood of 500 ml soda pop	Detailed notes 4
200	Times likelihood of 25 g candy	Detailed notes 4
201	Initial guess 0.5 highlighted	Detailed notes 4
202	Popcorn likelihood = 0.06	Detailed notes 4
203	Soda pop likelihood = 0.004	Detailed notes 4
204	Soda pop likelihood 0.004 (product 0.5 x 0.06 x 0.004)	Detailed notes 4
205	Likelihood for candy is…	Detailed notes 4
206	x a really, really small number	Detailed notes 4
207	Note: take log() to prevent Underflow	Detailed notes 5
208	The general idea of Underflow is…	Detailed notes 5
209	Computers have a limit on closeness to 0	Detailed notes 5
210	Below that limit: underflow problems and errors	Detailed notes 5

Previous: 03 Text Classification - Chunk 06 (s.151-180) | Next: 03 Text Classification - Chunk 08 (s.211-240) | Up: 03 Text Classification - Index