01 Intro - Index

DAT402 Text and Web Mining, Introduction (48 slides). Canvas: 01 Intro - Canvas

Chunk notes

1. 01 Intro - Chunk 01 (s.1-30): motivation (90% unstructured data), definition and applications, search vs discovery, the text mining process, preprocessing (tokenization, stopwords, stemming/lemmatization, POS), feature generation (bag of words, TF-IDF, word embeddings), start of feature selection (s.1-30).
2. 01 Intro - Chunk 02 (s.31-48): pruning and POS filtering, pattern discovery, document clustering (Jaccard, cosine + TF-IDF, embeddings), document classification and sentiment analysis, why text mining is hard, summary (s.31-48).

Overview pages

• 01 Intro - Cheat Sheet
• 01 Intro - Mind Map

Concept notes

Process and basics: Concept - Text Mining, Concept - Text Mining Process, Concept - Text Mining Challenges

Preprocessing: Concept - Text Preprocessing, Concept - Tokenization, Concept - Stopword Removal, Concept - Stemming, Concept - Lemmatization, Concept - POS Tagging

Features: Concept - Bag of Words, Concept - TF-IDF, Concept - Word Embeddings, Concept - Feature Selection, Concept - Pruning

Pattern discovery: Concept - Pattern Discovery, Concept - Document Clustering, Concept - Jaccard Coefficient, Concept - Cosine Similarity (note lives in 02 Pre-processing), Concept - Document Classification

Applications: Concept - Sentiment Analysis, Concept - Sentiment Lexicon, Concept - Information Extraction

Past exam style: Past Exam - Question Bank & Style Guide