Context Using my [Safebooru's image metadata][1] set, I create the list of used tags. Content There are two versions of list: one of having full set of tags and other with the type. The first one contains 369,289 tags with its used frequency. The second one contains 15,283 tags with its used frquency and Safebooru defined type. Typed tag file is quite small, since there are two many insignificant tags (of which usage is less than 128) and it takes too much time to parse every 369K data. Note that some tags were discarded while processing, as they have non-ASCII character. It was because I used trie data structure to store tags, I can't take few unicode tag such as ⑨. I modified some types mannualy. Originally few tag (less than 50) were marked as ambiguous, and these had mutiple tag types. As I'm not interested in ambiguous, I ignored them. While parsing webpages these kind of tags has comma at the last (ex: copyright, ), so I deleted them. Acknowledgements First I created the first file using [this script][2]. Then by using [another one][3], I asked [Safebooru's server][4] for every tags whether they are general or something else. It took about more than 12 hours, maybe. Since Safebooru's Tag page has some serious bugs, some tags failed to be parsed. These buggy tags are marked as 'unknown' type, uses at your own risks. (ex: multiple_girls) Inspiration Yet more classification is needed. General is too much wide section, but I don't have any idea to classify them right now. Very interesting statistics I found was approximation of amount of tags. Let x be the truncation parameter, and y is the number of tags which were used more than x times. I plot it and very surprised! With my rough calculation, y = total_number_of_tags * x^-0.70819 approximately. I don't know why, but it was very interesting. ![Both axis were applied the logarithmic scale][5] [1]: https://www.kaggle.com/phryxia/safebooru-2018 [2]: https://github.com/Phryxia/madproject/blob/master/01_extract_used_tags.py [3]: https://github.com/Phryxia/madproject/blob/master/02_classify_tag_type.py [4]: https://safebooru.org/index.php?page=tags&s=list [5]: https://blogfiles.pstatic.net/MjAxODA1MjhfMjEy/MDAxNTI3NDcyMDU0NTE0.07BxhiaGVHflsdCL1SwAL3nBwZ_w1ROmtzK3Id41FMog.BsEzTzWOBmYWUB_-wFBJS0ZVqsfOh5BpAQBzd2PR9tog.PNG.xaya_epica/009.png