Goal This is a small project lead by Yury Kashnitsky within OpenDataScience and Amsterdam Data Science communities. We plan to explore transfer & semi-supervised learning techniques for NLP tasks, mainly for classification. The idea is to develop best practices for using such models as BERT & ULMFiT (maybe smth else as well) for production-grade usage. Possible outcomes of this collaboration: - primarily, shared experience within this group, and advance in our own projects - articles sharing our experience (ex. Medium) - shared models, ex. trained LM for ULMFiT in Dutch - small library, ex. to productionize ULMFiT models (if they turn out to work best) Anybody is welcome to join and **share findings** via Kernels and Discussions. Datasets We are gathering several datasets in English, Russian and Dutch. Each of them addresses the **general task** - to utilize loads of unlabeled texts to improve classification of (scarce) labeled texts. So for each task we have the following files: - train.csv (small) - validation.csv (small) - unlabeled.csv (large) - test.csv (optionally, within competitions) Current datasets are: - Amazon pet product reviews classification (English, 6 classes, 52k train, 17k valid, 17k test, 100k unlabeled), [competition](https://www.kaggle.com/c/amazon-pet-product-reviews-classification/) , see [Kernels](https://www.kaggle.com/c/amazon-pet-product-reviews-classification/kernels) for baselines: logit-tfidf, ULMFiT & BERT - Amazon healthcare reviews (English) (6 classes, 7k train, 3k valid, 200k unlabeled ) - Clickbait news detection (English, 3 classes, 25k train, 5.5k valid, 3.5k test, 80k unlabeled), [competition](https://www.kaggle.com/c/clickbait-news-detection), see [Kernels](https://www.kaggle.com/c/clickbait-news-detection/kernels) for baselines: logit-tfidf, ULMFiT & BERT. - Dutch book reviews (Dutch, 2 classes, 14k train, 6k valid, 90k validation). Acknowledgements Thanks to Vladislav Lyalin for the clickbait news data (original [competition](https://www.kaggle.com/c/dlinnlp-spring-2019-clf) by ipavlov) and to [Benjamin van der Burgh](https://github.com/benjaminvdb) for Dutch reviews data (source [repository](https://github.com/benjaminvdb/110kDBRD)). Background image credit: Jeremy Howard, [fast.ai Lesson 4](https://course.fast.ai/videos/?lesson=4)