Concept - Synonymy and Semantic Relatedness
Definition
Linguistic characteristics that plain Concept - Bag of Words vectors miss: different words can mean the same (synonymy) or be about related things (semantic relatedness), so documents on the same topic may share no words (s.190).
How LSA handles it (s.191)
- Dimensionality reduction forces ignoring many details.
- Different words in the original space are mapped to the same dimension in the reduced space.
- The “cost” of mapping synonyms together is much less than collapsing unrelated words.
- SVD selects the “least costly” mapping (Eckart-Young theorem).
Related
Concept - Latent Semantic Analysis (LSA) | Concept - Truncated SVD | Concept - Word Embeddings (also handle synonyms)