Concept - Document-Term Matrix

Definition

The matrix $C$ of the Concept - Bag of Words representation (entries may be counts, binary values or Concept - TF-IDF weights). Rows = documents, columns = vocabulary terms. Shape $n$ by $m$: $n$ = number of documents, $m$ = vocabulary size (s.171).

Example (s.170)

Toy corpus of six documents: “car road gas”, “truck road”, “car”, “gas oil”, “gas”, “oil” -> 6x5 binary matrix with columns car, gas, oil, road, truck.

Problems (s.171-172)

• High-dimensional and sparse.
• Just counting, no semantics: “gas” (doc 4) and “oil” (doc 5) have Concept - Cosine Similarity 0 although related (s.172).

Remedies

Concept - Feature Selection (keep top-K columns; still sparse, loses information, s.173) or Concept - Dimensionality Reduction (project to $n\times d$ document-”topic” matrix, s.176) via Concept - Singular Value Decomposition (SVD) / Concept - Latent Semantic Analysis (LSA).

Related

Concept - Bag of Words | Concept - TF-IDF | Concept - Cosine Similarity

Source: 02 Pre-processing - Chunk 06 (s.151-180)

From 02 Pre-processing s.63-70, s.89

Rows = documents, columns = features (vocab words); a cell measures a feature for a document (s.64). Entries: presence, frequency, thresholded frequency (0/1/2+) (s.66). Stored efficiently as Concept - Sparse Matrix (CSR) pairs (s.70). Section “Document-term Matrix & Cosine Similarity” opens at s.89, with the Troll 2 sentences (s.90). See 02 Pre-processing - Chunk 03 (s.61-90).