Context This is a pretrained BERT model, dubbed BERT-TINY. It is not my work, and is merely reproduced for use on kaggle from https://github.com/google-research/bert. It is much smaller than BERT base, so may be useful for quick training iterations and research on general performance insights to BERT type models. A description of the model and it's utility presumably is contained in: @article{turc2019, title={Well-Read Students Learn Better: On the Importance of Pre-training Compact Models}, author={Turc, Iulia and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina}, journal={arXiv preprint arXiv:1908.08962v2 }, year={2019} } though I haven't yet made eye contact with that paper. Source This pretrained model is publicly available at https://github.com/google-research/bert