# Background WSDM (pronounced "wisdom") is one of the premier conferences on web-inspired research involving search and data mining. [The 12th ACM International WSDM Conference][1] will take place in Melbourne, Australia during Feb. 11-15, 2019. This task is organized by ByteDance, the Platinum Level Sponsor of the conference. ByteDance is a global Internet technology company started from China. Our goal is to build a global content platform that enable people to enjoy various content in various forms. We inform, entertain, and inspire people across language, culture and geography. One of the challenges which we are facing is to combat different types of fake news. Fake news here refers to all forms of false, inaccurate or misleading information, which now poses a big threat to human civilization. At Bytedance, we have created a large-scale database to store existing fake news articles. Any new article must go through a test on the truthfulness of content before being published. We conduct matching between the new article and the articles in the database. Articles identified as containing fake news will be withdrawn after human verification. The accuracy and efficiency of the process, therefore, becomes crucial for us to make the platform safe, reliable, and healthy. # About This Dataset This dataset is released as the competition dataset of [Task: Fake News Classification][1] with the following task: Given the title of a fake news article A and the title of a coming news article B, participants are asked to classify B into one of the three categories. - agreed: B talks about the same fake news as A - disagreed: B refutes the fake news in A - unrelated: B is unrelated to A ## File - **train.csv** - training data contains 320,767 news pairs in both Chinese and English. This file provides the only data you can use to finish the task. Using external data is not allowed. - **test.csv** - testing data contains 80,126 news pairs in both Chinese and English. The approximately 25% of the testing data is set to be public and is used to calculate your accuracy shown on the leading board. The remaining 75% private data is used to calculate your final result of the competition. - **sample_submission.csv** - sample answer to the testing data. ## Data fields - **id** - the id of each news pair. - **tid1** - the id of fake news title 1. - **tid2** - the id of news title 2. - **title1_zh** - the fake news title 1 in Chinese. - **title2_zh** - the news title 2 in Chinese. - **title1_en** - the fake news title 1 in English. - **title2_en** - the news title 2 in English. - **label** - indicates the relation between the news pair: agreed/disagreed/unrelated. The English titles are machine translated from the related Chinese titles. This may help participants from all background to get better understanding of the datasets. Participants are highly recommended to use the Chinese version titles to finish the task. # Evaluation Metrics We use **Weighted Categorization Accuracy** to evaluate your performance. Weighted categorization accuracy can be generally defined as: $$ WeightedAccuracy(y, \hat{y}, \omega) = rac{1}{n} \displaystyle{ um_{i=1}^{n}} rac{\omega_i(y_i=\hat{y}_i)}{ um \omega_i} $$ where \\(y\\) are ground truths, \\(\hat{y}\\) are the predicted results, and \\(\omega_i\\) is the weight associated with the \\(i\\)th item in the dataset. In our test set, we assign each testing item a weight according to its category. The weights of the three categories, agreed, disagreed and unrelated are \\( rac{1}{15}\\), \\( rac{1}{5}\\), \\( rac{1}{16}\\), respectively. We set the weights in consideration of the imbalance of the data distribution to minimize the bias to your performance caused by the majority class (unrelated pairs accounts for approximately 70% of the dataset). [1]: https://www.kaggle.com/c/fake-news-pair-classification-challenge