Transfer Learning word2vec

[ctrado18]

Hey there,

I am confused about the comment from here: https://datascience.stackexchange.com/questions/10695/how-to-initialize-a-new-word2vec-model-with-pre-trained-model-weights

Is it possible to use pretrained vectors like from fasttext to train new on custom data?

From the accepted answer it seems so?

[menshikh-iv]

hello @ctrado18, pre-trained vectors isn't enough to continue training (training requires 2 matricies, pre-trained vectors is just one matrix). For continuation, you need full model (not only word-vectors, like glove mentioned in comments).

load_word2vec_format returns KeyedVectors, not Word2Vec model, see also https://radimrehurek.com/gensim/models/keyedvectors.html. Comments from your link contains some "hacky" variant that makes it works somehow (but this not the same if you have an full model and continue training).

About fasttext: facebook share full model in .bin format (both matricies + model parameters), for this reason, you can continue training with it without any problems (we fixed all related issues in #2313 and will release it at the end of month, wait for 3.7.0 a bit).

Next time, please use mailing list for questions, GitHub for bug reports and feature requests.

[ctrado18]

Hey @menshikh-iv Thank you? I want to use exactly this fasttext. Do you know if it is already possible with spacy word vectors too?

Next tie I use mailing list.

[menshikh-iv]

Do you know if it is already possible with spacy word vectors too?

sorry, idk about spacy