Concept - Logistic Regression

From chunk 10 (s.271-295)

• With one feature there is one weight $w$ and bias $b$; in practice each sample is an n-dimensional vector and the model has n weights and one bias (s.271). For text mining the inputs are Concept - Bag of Words vectors.
• Output example (s.271): $\sigma (-1.2\cdot 10+1.4\cdot 5+2.2\cdot 3+0.6\cdot 5+0.2)$.
• Threshold on the output probability controls the precision/recall trade-off (s.278): see Concept - Precision and Recall.
• Multiclass: Concept - One-vs-All Classification (one LR per class, arg max) or Concept - Softmax Classifier.
• Discriminative model: learns $P(Y|X)$ directly (s.293-294) - see Concept - Generative vs Discriminative Models; contrast with generative Concept - Naive Bayes.
• Evaluated with Concept - Confusion Matrix, Concept - F1 Score, Concept - ROC Curve and AUC.

(Earlier chunks of this deck may append the sigmoid, loss and training details above/below.)

Source chunk: 03 Text Classification - Chunk 10 (s.271-300)

Related

Concept - Sigmoid Function, Concept - Cross-Entropy Loss, Concept - Gradient Descent, Concept - Decision Boundary, Concept - Linear Regression, Concept - Softmax Classifier, Concept - Naive Bayes. Up: 03 Text Classification - Index