02 Pre-processing - Index

Lecture: “Text Preprocessing: From textual information to numerical vector” (193 slides). Raw text → tokens → normalized words → bag-of-words matrix → feature selection / reduction → LSA topic space.

Canvas: 02 Pre-processing - Canvas

Study aids: 02 Pre-processing - Cheat Sheet | 02 Pre-processing - Mind Map | Past exam style: Past Exam - Question Bank & Style Guide

Chunk notes (in lecture order)

1. 02 Pre-processing - Chunk 01 (s.1-30) - feature extraction and linear models, corpus, normalization, tokenization, Zipf’s law, tokens vs types, stopwords, case folding.
2. 02 Pre-processing - Chunk 02 (s.31-60) - stemming vs lemmatization, morphology, sentence boundary detection, POS tagging, WSD (Lesk), parsing, Universal Dependencies, WordNet.
3. 02 Pre-processing - Chunk 03 (s.61-90) - WordNet relations, bag of words, tf-idf, multiword features, CSR, dictionary reduction, entropy and information gain.
4. 02 Pre-processing - Chunk 04 (s.91-120) - cosine similarity intro (Troll 2 tweets, “Hello, World!” example).
5. 02 Pre-processing - Chunk 05 (s.121-150) - cosine cases (1 / 0 / 0.71), manual procedure, decoding the formula.
6. 02 Pre-processing - Chunk 06 (s.151-180) - finished cosine example, document-term matrix, feature selection vs dimensionality reduction, SVD.
7. 02 Pre-processing - Chunk 07 (s.181-193) - LSA example, truncated SVD, synonymy, implementation recipe, other representations.

Concept notes in this deck

Features and text basics: Concept - Feature Extraction, Concept - Linear Model, Concept - Corpus, Concept - Text Normalization, Concept - Tokens and Types, Concept - Zipf’s Law, Concept - Case Folding, Concept - Morphology, Concept - Sentence Boundary Detection

Syntax and semantics: Concept - Penn Treebank, Concept - Parsing, Concept - Universal Dependencies, Concept - Word Sense Disambiguation, Concept - Lesk Algorithm, Concept - WordNet, Concept - WordNet Relations, Concept - Word Sense Relations, Concept - Synonymy and Semantic Relatedness

Representation and selection: Concept - Multiword Features, Concept - Sparse Matrix (CSR), Concept - Dictionary Reduction, Concept - Entropy (TWM), Concept - Information Gain (TWM), Concept - Document-Term Matrix, Concept - Cosine Similarity, Concept - Curse of Dimensionality

Dimensionality reduction: Concept - Dimensionality Reduction, Concept - Singular Value Decomposition (SVD), Concept - Truncated SVD, Concept - Latent Semantic Analysis (LSA), Concept - Non-negative Matrix Factorization (NMF), Concept - Other Document Representation Approaches

Defined in 01 Intro, reused here: Concept - Text Preprocessing, Concept - Tokenization, Concept - Stopword Removal, Concept - Stemming, Concept - Lemmatization, Concept - POS Tagging, Concept - Bag of Words, Concept - TF-IDF, Concept - Feature Selection