Context The BillSum dataset is the first corpus for automatic summarization of US Legislation. The corpus contains the text of bills and human-written summaries from the US Congress and California Legislature. It was published as a paper at the EMNLP 2019 New Frontiers in Summarization workshop. We publish it on Kaggle to encourage collaboration and iteration on this problem. The BillSum dataset is interesting to explore for two reasons: First, legislation has not previously been considered for this task, despite US Congress and state governments releasing tens of thousands of bills every year. Second, approximately half of the states do not provide human-written summaries of their legislation, automatic summarization can fill that gap and allow citizens find relevant bills faster. Our current code can be found at: https://github.com/FiscalNote/BillSum Content The BillSum dataset consists of three parts: US training bills, US test bills and California test bills. The US bills were collected from the **Govinfo** service provided by the [United States Government Publishing Office (GPO)](https://github.com/unitedstates/congress). Our corpus consists of bills from the 103rd-115th (1993-2016) sessions of Congress. The data was randomly split into 28,408 train bills and 5014 test bills. For California, bills from the 2015-2016 session were scraped directly from the [legislature's website](http://leginfo.legislature.ca.gov); the summaries were written by their Legislative Counsel. For a quick visual example: https://www.kaggle.com/akornilo/data-introduction The data is organized as follows: - **official_dataset** folder contains the files with the original dataset. (See README.md for field descriptions) - **cleaned_dataset** folder contains the original text, as well as the clean version of all the fields. (The cleaning was done with [this script](https://github.com/FiscalNote/BillSum/blob/master/billsum/data_prep/clean_text.py)) **Data Structure** - text: bill text - summary: (human-written) bill summary - title: bill title (can be used for generating a summary) - bill_id: An identified for the bill - in US data it is SESSION_BILL-ID, for CA BILL-ID In the "clean" versions of the file - the cleaned texts are under the "clean_{text/summary/title}" fields. Evaluation If you develop your own algorithm for summaries, you can aggregate your Rouge Scores with [this script](https://github.com/FiscalNote/BillSum/blob/master/billsum/utils/compute_rouge_from_texts.py) We look forward to hearing what you discover :)