License These models are all released under the same license as the source code (Apache 2.0). Context From the README of the GitHub repository (https://github.com/google-research/bert): > BERT, or Bidirectional Encoder Representations from Transformers, is a new method of pre-training language representations which obtains state-of-the-art results on a wide array of Natural Language Processing (NLP) tasks. >Our academic paper which describes BERT in detail and provides full results on a number of tasks can be found here: https://arxiv.org/abs/1810.04805 . Content This dataset contains the latest pre-trained models, along with the code from Google Research's BERT GitHub repository, as of the data of last retrieval (as noted in the file descriptions and the last commit date). >We are releasing the BERT-Base and BERT-Large models from the paper. Uncased means that the text has been lowercased before WordPiece tokenization, e.g., John Smith becomes john smith. The Uncased model also strips out any accent markers. Cased means that the true case and accent markers are preserved. Typically, the Uncased model is better unless you know that case information is important for your task (e.g., Named Entity Recognition or Part-of-Speech tagging). > >These models are all released under the same license as the source code (Apache 2.0). > >For information about the Multilingual and Chinese model, see the Multilingual README. > >When using a cased model, make sure to pass --do_lower=False to the training scripts. (Or pass do_lower_case=False directly to FullTokenizer if you're using your own script.) Models included in this dataset: - BERT-Base, Uncased: 12-layer, 768-hidden, 12-heads, 110M parameters - BERT-Large, Uncased: 24-layer, 1024-hidden, 16-heads, 340M parameters - BERT-Base, Cased: 12-layer, 768-hidden, 12-heads , 110M parameters - BERT-Large, Cased: 24-layer, 1024-hidden, 16-heads, 340M parameters - BERT-Base, Multilingual Cased (New, recommended): 104 languages, 12-layer, 768-hidden, 12-heads, 110M parameters - BERT-Base, Chinese: Chinese Simplified and Traditional, 12-layer, 768-hidden, 12-heads, 110M parameters **NOT** included: - BERT-Base, Multilingual Uncased (Orig, not recommended) (Not recommended, use Multilingual Cased instead): 102 languages, 12-layer, 768-hidden, 12-heads, 110M parameters Each .zip file contains three items: - A TensorFlow checkpoint (bert_model.ckpt) containing the pre-trained weights (which is actually 3 files). - A vocab file (vocab.txt) to map WordPiece to word id. - A config file (bert_config.json) which specifies the hyperparameters of the model. Acknowledgements This is not my work; it is the work of Google Research. Please read their paper and the README in the repository for more information about how BERT works, why it's useful and important, and how to use it.