Concept - Dimensionality Reduction

Definition (s.174)

New features are learned by combining old features: a map ${\mathbb{R}}^M\to {\mathbb{R}}^d$ with $d<m$.

Versus feature selection

Concept - Feature Selection merely keeps a subset of the existing columns (still sparse, discards information, s.173); dimensionality reduction builds new, combined features.

Algorithms listed (s.174)

PCA, NMF (Concept - Non-negative Matrix Factorization (NMF)), auto-encoder, t-SNE, kernel PCA, manifold learning, …

Linear vs non-linear (s.175)

• Linear (PCA): new axes e1, e2 are linear directions of the point cloud (axes f1, f2).
• Non-linear (manifold learning): a curved sheet (Swiss roll) is “unrolled” into a flat strip.

Linear projection of the term-document matrix (s.176-177)

$(n\times m)\cdot (m\times d)=n\times d$ document-”topic” matrix. The projection matrix ($m\times d$) is found by matrix decomposition: Concept - Latent Semantic Analysis (LSA) / Concept - Singular Value Decomposition (SVD), NMF, …

Related

Concept - Document-Term Matrix | Concept - Truncated SVD | Concept - Feature Selection

Source: 02 Pre-processing - Chunk 06 (s.151-180)