Charlottesville, Virgina ======================== Charlottesville is home to a statue of Robert E. Lee which is slated to be removed. (For those unfamiliar with American history, Robert E. Lee was a US Army general who defected to the Confederacy during the American Civil War and was considered to be one of their best military leaders.) While many Americans support the move, believing the main purpose of the Confederacy was to defend the institution of slavery, many others do not share this view. Furthermore, believing Confederate symbols to be merely an expression of Southern pride, many have not taken its planned removal lightly. As a result, many people--including white nationalists and neo-Nazis--have descended to Charlottesville to protest its removal. This in turn attracted many counter-protestors. Tragically, one of the counter-protestors--Heather Heyer--was killed and many others injured after a man intentionally rammed his car into them. In response, President Trump blamed "both sides" for the chaos in Charlottesville, leading many Americans to denounce him for what they see as a soft-handed approach to what some have called an act of "domestic terrorism." This dataset below captures the discussion--and copious amounts of anger--revolving around this past week's events. The Data ======== Description ----------- This data set consists of a random sample of 50,000 tweets per day (in accordance with the Twitter Developer Agreement) of tweets mentioning Charlottesville or containing "#charlottesville" extracted via the Twitter Streaming API, starting on August 15. The files were copied from a large Postgres database containing--currently--over 2 million tweets. Finally, a table of tweet counts per timestamp was created using the whole database (not just the Kaggle sample). The data description PDF provides a full summary of the attributes found in the CSV files. Note: While the tweet timestamps are in UTC, the cutoffs were based on Eastern Standard Time, so the August 16 file will have timestamps ranging from `2017-08-16 4:00:00` UTC to `2017-08-17 4:00:00` UTC. Format ------ The dataset is available as either separate CSV files or a single SQLite database. License ======= I'm releasing the dataset under the CC BY-SA 4.0 license. Furthermore, because this data was extracted via the Twitter Streaming API, its use must abide by the [Twitter Developer Agreement][1]. Most notably, the display of individual tweets should satisfy [these requirements][2]. More information can be found in the data description file, or on Twitter's website. Acknowledgements ============ Obviously, I would like to thank Twitter for providing a fast and reliable streaming service. I'd also like to thank the developers of the Python programming language, psycopg2, and Postgres for creating amazing software with which this data set would not exist. Image Credit ------------------ The banner above is a personal modification of these images: - Evan Nesterak: [Image Source][3] [Image License][4] - Wikipedia user Cville Dog [Image Source][5] - The Associated Press [Image Source][6] Inspiration ========== I almost removed the header "inspiration" from this section, because this is a rather sad and dark data set. However, this is preciously why this is an important data set to analyze. Good history books have never shied away from unpleasant events, and never should we. This data set provides a rich opportunity for many types of research, including: - Natural language processing - Sentiment analysis - Data visualization Furthermore, given the political nature of this dataset, there are a lot of social science questions that can potentially be answered, or at least piqued, by this data. [1]: https://dev.twitter.com/overview/terms/ [2]: https://dev.twitter.com/overview/terms/display-requirements [3]: https://www.flickr.com/photos/153804281@N02/36421659232/ "Image Source" [4]: https://en.wikipedia.org/wiki/Unite_the_Right_rally#/media/File:White_supremacists_clash_with_police_(36421659232).jpg "Image License" [5]: https://en.wikipedia.org/wiki/Unite_the_Right_rally#/media/File:Lee_Park,_Charlottesville,_VA.jpg "Image Source" [6]: https://www.voanews.com/a/rallies-in-aftermath-of-charlottesville-violence/3983710.html "Image Source"