Concept - Precision and Recall

  • Precision - how accurate the positive prediction is (s.275). Based on the Predicted Positive column of the Concept - Confusion Matrix.
  • Recall - how many positive cases are detected (s.275). Based on the Positive Label row. Same formula as TPR in Concept - ROC Curve and AUC.
  • Example (s.276): TP=27, FN=4, FP=1, TN=18 -> precision 27/28 = 0.964, recall 27/31 = 0.871, accuracy 0.9.
  • Trade-off (s.278-279): with a logistic-regression probability threshold, a high threshold (about 0.8 in the slide) = high precision, low recall; a low threshold (about 0.2) = low precision, high recall. In non-trivial situations they cannot both be optimized; which to favour depends on the use case. The precision-recall curve falls as recall rises (s.279).
  • Combine them with Concept - F1 Score (s.280).
  • s.277 asks for examples where accuracy is high but recall (or precision) is low; no answers on the slide. Imbalanced data (s.280) is one such case.

Concept - Confusion Matrix | Concept - F1 Score | Concept - ROC Curve and AUC | Concept - Logistic Regression

Source chunk: 03 Text Classification - Chunk 10 (s.271-300)