Loading word2vec model cannot be done with a reasonable memory capacity

[SMMousaviSP]

Hi,
I wanted to do augmentation based on word2vec similarity so I downloaded the word2vec model as said in the README file:

from nlpaug.util.file.download import DownloadUtil

DownloadUtil.download_word2vec(dest_dir='.') # Download word2vec model

A zip file was downloaded and I extracted it, then when I tried to load it with the code below it took too long and crashed because of not enough memory. I also tried to do this on Google Colab which gives me 12 GB of memory, but didn't work for the same reason.

import nlpaug.augmenter.word as naw

text = "Sample text to test augmentation"
aug = naw.WordEmbsAug(
    model_type='word2vec', model_path='GoogleNews-vectors-negative300.bin',
    action="substitute")
augmented_text = aug.augment(text)
print("Original:")
print(text)
print("Augmented Text:")
print(augmented_text)

The .bin file is 3.5 GB, why it's not working even with 12 GB of memory?

[makcedward]

Using gensim package to load files. Impoved loading speed and memory consumption. You may retry by getting the latest dev version (pip install gensim git+https://github.com/makcedward/nlpaug.git)

[makcedward]

Enhanced in 1.1.8 version