Select Language

AI社区

公开数据集

Word2vec在维基百科上训练数据(单字母+双字母),以捕捉unigram和bigram

Word2vec在维基百科上训练数据(单字母+双字母),以捕捉unigram和bigram

8.62G
326 浏览
0 喜欢
0 次下载
0 条讨论
NLP,Computer Science,Software,Programming,Neural Networks Classification

这是一个单词嵌入模型,创建于维基百科+各种来源的评论。与从基于短语的方法(不考虑相邻词的短语/双词上下文)创建双词不同,这......

数据结构 ? 8.62G

    Data Structure ?

    * 以上分析是由系统提取分析形成的结果,具体实际数据为准。

    README.md

    这是一个单词嵌入模型,创建于维基百科+各种来源的评论。与从基于短语的方法(不考虑相邻词的短语/双词上下文)创建双词不同,这是通过考虑中心词周围的所有单词和双词上下文来训练的。

    Content

    The source code for the dataset is available here, installation can done by "python setup.py install"

    Training script can be found here

    https://github.com/s4sarath/gensim_ngram/blob/master/train_ngram.py

    Training model can be found here

    https://github.com/s4sarath/gensim_ngram/blob/master/README.md

    Sample similar words from the model

    model.wv.most_similar

    a.) amazing product

    [('amazing product', 1.0), ('awesome product', 0.9272927), ('amazing product,', 0.888031), ('incredible product', 0.8867724), ('amazing product!', 0.88521475), ('amazing product.', 0.8845437), ('awesome product!', 0.8644207), ('amazing product!!', 0.8612526), ('amazing product!!!', 0.85835207), ('awesome product.', 0.8530247), ('awesome product!!', 0.8516336), ('awesome product,', 0.8495761), ('awesome item', 0.8434567), ('product. amazing', 0.84247625), ('incredible product.', 0.8421844), ('awesome product!!!', 0.84074044), ('wonderful product', 0.8406575), ('awesome device', 0.836467), ('incredible product!', 0.8337494), ('fantastic product', 0.8330554)]

    b.) brad pitt

    [('brad pitt', 1.0), ('julia roberts', 0.84390914), ('angelina jolie', 0.84303164), ('ben affleck', 0.8231394), ('matt damon', 0.81166387), ('affleck', 0.8074477), ('george clooney', 0.80540144), ('costner', 0.80255926), ('tom hanks', 0.8017744), ('dustin hoffman', 0.79872185), ('natalie portman', 0.798303), ('ryan gosling', 0.79511935), ('dicaprio', 0.79246503), ('kevin spacey', 0.7921234), ('alec baldwin', 0.7907918), ('actor brad', 0.7901952), ('russell crowe', 0.78980654), ('kevin costner', 0.7894964), ('christopher walken', 0.7882538), ('jennifer aniston', 0.7878684)]

    c.) mohanlal

    [('mohanlal', 1.0), ('mammootty', 0.9794469), ('kamal haasan', 0.9596181), ('haasan', 0.9563364), ('rajkumar', 0.95312166), ('gopi', 0.9529321), ('sivaji', 0.95167804), ('madhavan', 0.9510826), ('dileep', 0.95085794), ('chiranjeevi', 0.95059955), ('jayaram', 0.9503455), ('nagesh', 0.9484335), ('sathyaraj', 0.9479996), ('rajinikanth', 0.94777143), ('suresh gopi', 0.9466225), ('sivaji ganesan', 0.94393903), ('prakash raj', 0.9437847), ('sathyan', 0.9431832), ('prabhu', 0.942392), ('bharath', 0.9391954)]

    d.) machine learning

    [('machine learning', 1.0000001), ('learning algorithms', 0.8841063), ('data mining', 0.8291545), ('machine translation', 0.814913), ('support vector', 0.80520463), ('algorithms', 0.8029659), ('learning theory', 0.8026564), ('algorithms and', 0.80255526), ('information retrieval', 0.7991563), ('neural networks', 0.7982512), ('vector machines', 0.79787594), ('machine intelligence', 0.79575825), ('learning algorithm', 0.7918976), ('reinforcement learning', 0.7897328), ('language processing', 0.78945714), ('and computational', 0.7862742), ('vector machine', 0.78508246), ('knowledge representation', 0.7850384), ('algorithmic', 0.7817018), ('distributed systems', 0.7809721)]

    e.) mortal kombat

    [('mortal kombat', 0.99999994), ('kombat', 0.92918265), ('tekken', 0.855644), ('kombat ii', 0.8423183), ('virtua fighter', 0.82694477), ('soulcalibur', 0.8240025), ('ninja gaiden', 0.8233547), ('darkstalkers', 0.8189633), ('kombat vs', 0.8051237), ('kombat armageddon', 0.80245066), ('kombat series', 0.80217266), ('samurai shodown', 0.8003039), ('resident evil', 0.8001634), ('game mortal', 0.7937777), ('in capcom', 0.7936872), ('kombat mortal', 0.7936853), ('mortal', 0.79330146), ('kombat deception', 0.7923815), ('onimusha', 0.7913557), ('virtua', 0.79038495)]

    f.) nissan

    [('nissan', 1.0000002), ('mazda', 0.9355751), ('toyota', 0.89277387), ('lexus', 0.89011514), ('subaru', 0.8749101), ('toyota corolla', 0.86015534), ('nissan skyline', 0.85717183), ('mazda rx', 0.8544719), ('volkswagen', 0.8482176), ('bmw', 0.84316957), ('mitsubishi', 0.8426397), ('honda', 0.8378298), ('infiniti', 0.8358605), ('celica', 0.83509576), ('chevrolet corvette', 0.8315984), ('isuzu', 0.8309591), ('nissan gt', 0.8307908), ('datsun', 0.8291819), ('chevrolet', 0.8271923), ('opel', 0.8265841)]

    Inspiration

    Conventional phrase based word2vec ( including gensim Phrase approach ) is not considering phrase based context/window or neighbor words.

    ×

    帕依提提提温馨提示

    该数据集正在整理中,为您准备了其他渠道,请您使用

    注:部分数据正在处理中,未能直接提供下载,还请大家理解和支持。
    暂无相关内容。
    暂无相关内容。
    • 分享你的想法
    去分享你的想法~~

    全部内容

      欢迎交流分享
      开始分享您的观点和意见,和大家一起交流分享.
    所需积分:35 去赚积分?
    • 326浏览
    • 0下载
    • 0点赞
    • 收藏
    • 分享