Feature Selection (text mining)
Stage 3 of the Concept - Text Mining Process: not all features help; learners might have difficulty with high-dimensional data (01 Intro s.30). Reduce the (large) number of features before data mining (s.11). Generic ML version: Concept - Feature Selection is also covered in the Machine Learning course.
Related
Concept - Bag of Words | Concept - TF-IDF
Appears in
From chunk 02 (s.31-32) (01 Intro - Chunk 02 (s.31-48))
- Pruning document vectors by document frequency (percentual or absolute;
min_df/max_df): see Concept - Pruning (s.31). - Filter tokens by POS tags (s.32): keep adjectives (JJ.) for sentiment analysis (good, bad, great), nouns (N.) for clustering. RapidMiner supports PENN (English) and STTS (German) tag systems, filters as regular expressions; NLTK supports PENN. See Concept - POS Tagging.
- Next stage: Concept - Pattern Discovery (s.33).
From 02 Pre-processing chunk 06 (s.173)
Based on measures such as mutual information, keep the top-ranked K features (“choose a subset of the features”). Example: the 5-column toy matrix is reduced to columns car and gas. Drawback: still sparse and discards lots of information. Contrast with Concept - Dimensionality Reduction, which creates new combined features.
Source: 02 Pre-processing - Chunk 06 (s.151-180)
From 02 Pre-processing s.72-88
Part of Concept - Dictionary Reduction (s.73). Attribute ranking (s.74): score each feature (word) with a measure on individual features (document frequency, information gain, mutual information, CHI), sort by score, keep the first k, learn the model; the kept words form a local dictionary. Information gain is developed with the weather example: Concept - Entropy (TWM), Concept - Information Gain (TWM) (select high IG features). See 02 Pre-processing - Chunk 03 (s.61-90).