This dataset contains the ID, categories, and raw text from each file in NLTK's Reuters corpus.

Content

Each file (row) is a news document. Each document can be classified as one or more categories.  Files are also classified as test or train files, which can be found in the ID column and used for textual model building/validation. There are 10,788 total news documents in the file.

Acknowledgements

Thank you to NLTK and its contributors for everything you do.

Code

This dataset was generated using the following python code.

import nltk
import pandas as pd
nltk.download('reuters')
nltk.download('punkt')

# Extract fileids from the reuters corpus
fileids = reuters.fileids()

# Initialize empty lists to store categories and raw text
categories = []
text = []

# Loop through each file id and collect each files categories and raw text
for file in fileids:
    categories.append(reuters.categories(file))
    text.append(reuters.raw(file))

# Combine lists into pandas dataframe. reutersDf is the final dataframe. 
reutersDf = pd.Dataframe({'ids':fileids, 'categories':categories, 'text':text})