Concept - Generative vs Discriminative Models

	Generative (s.293)	Discriminative (s.293)
Learns	joint probability $P(X,Y)$ - “how was the data generated?“	conditional probability $P(Y\mid X)$
Classification	get $P(Y	X)$from$P(X,Y)$, then categorize
Character	less direct, more probabilistic	more direct, less probabilistic
Examples (s.294)	Concept - Naive Bayes	Concept - Logistic Regression

• s.294 poses it as a question (“Generative and Discriminative? Naive Bayes, Logistic Regression”); the pairing above follows from s.293 and the NB text-generation slide (s.295).
• Naive Bayes as generator (s.295): for word positions 1..T sample category $h_i$ from $p(h)$ (or fixed), sample the word from $p(\mathbf{d}|h_i)$. Limitation: ignores word dependency (P(“read the paper”) > P(“read the movie”)); a Hidden Markov Model partially solves this.

Related

Concept - Naive Bayes | Concept - Logistic Regression

Source chunk: 03 Text Classification - Chunk 10 (s.271-300)