Corpus
A generic collection of documents used for research and development of text-mining techniques (02 Pre-processing s.13-14). Collecting documents is the first step of text mining; applications may need a collection process: a Web crawler for Web applications, or an email audit application logging all incoming/outgoing messages at a mail server (s.13).
Example corpora / sources:
- Reuters-21578 and RCV1 (Reuters Corpus Volume 1; about 810,000 English-language Reuters news stories, tagged with topics) (s.13)
- Gutenberg Project: very large collection of literary and other texts, machine-readable once out of copyright (s.13)
- Linguistic Data Consortium (LDC) (s.13)
- UC Irvine ML Repository: 468 datasets (screenshot says 488) incl. Amazon reviews, email spam, sentiment-labelled sentences (s.14)
- Kaggle (data-mining competition) data sets (s.14)
Corpus sizes in tokens/types: see Concept - Tokens and Types.
Related
Concept - Text Preprocessing | Concept - Tokens and Types