Similarity using Sense2Vec along with Spacy

[armsp]

I am not able to leverage the similarity examples from the blog or issue #24 to my case. I just wanted to get the "sense" of "bear" i.e Homonyms in an example like "The bear growled. She could not bear the pain."

import spacy
nlp = spacy.load("en_core_web_md")
from sense2vec import Sense2VecComponent
text = "The bear growled. She could not bear the pain."
s2v = Sense2VecComponent('./reddit_vectors-1.1.0')
nlp.add_pipe(s2v)
doc = nlp(text)

After this i can't get any official reference for the similarity API that took two vectors. I have tried with the following but i get errors-

s2v._.similarity(doc[1]._.s2v_vec,doc[6]._.s2v_vec)
nlp._.similarity(doc[1]._.s2v_vec,doc[6]._.s2v_vec)

[ines]

The sense2vec pipeline component only adds the in_s2v, s2v_freq, s2v_vec and s2v_most_similar extension attributes to the Token and Span objects. That's why _.similarity won't work. The ._. properties are reserved for custom attributes added by the user or by plugins (you can read more about this here).

Internally, spaCy's .similarity method compares two vectors by calculating the cosine similarity (see here for the implementation).

It's not 100% clear from your example what you're trying to do, but if you want to compare the two vectors you've extracted, you could just calculate their consine similarity (see here for a simple example using numpy). Ideally, you should find that the similarity is low, because the sense2vec vectors also take the part-of-speech tag into account.

Just for completeness, you can also customise spaCy's built-in similarity methods. Maybe the sense2vec component could include an option for this, or expose its own ._.similarity method that compares two vectors.

[armsp]

@ines i was following this article https://explosion.ai/blog/sense2vec-with-spacy
I wanted to find the similarity between bear|NOUN and bear|VERB just for testing purposes. I wanted to see if sense2vec differentiates them.
Spacy gives the exact same vector to both since it actually works on the word's Lemma.
What i am trying to figure out is if i can find words/sentences that are ambiguous like He went to the bank. where bank can have multiple meanings. I am not sure how sense2vec would help me but i wanted to know.

[ines]

Yes, the regular word vectors only consider the lexeme, the context-insensitive entry in the vocabulary.

The vectors in the sense2vec model also take the part-of-speech tag into account. So what you did was almost correct – you got the vector for bear|NOUN and the vector for bear|VERB. Now you just need to compare them and calculate their similarity.

If you already know the words and their senses, you can also just use the sense2vec library directly and not via spaCy. Just look up the terms and you'll get a vector.

[armsp]

With reference to your last comment "If you already know the words and their senses, you can also just use the sense2vec library directly and not via spaCy. Just look up the terms and you'll get a vector."

Are there any ways to find the sense of the word if its not given already? Also is it possible to find those words in the first place?

[ines]

Are there any ways to find the sense of the word if its not given already? Also is it possible to find those words in the first place?

Well, in that case, you can use spaCy's part-of-speech tags, which should be fairly accurate. Using the sense2vec spaCy extension also makes sense then, because it lets you get the sense2vec vector straight from the token.

Also is it possible to find those words in the first place?

Do you mean, if you have a bunch of text and want to find homonyms in it? You can always load in existing lists, use the Matcher to find them and then compare the POS tags etc. You could also iterate over the tokens in your text, find tokens that have the same token.text and then compare them (and store their Doc or sentence Span if you've found the same token text with different POS tags).

(Just keep in mind that models are statistical and the tagger can always make mistakes, too – especially for words that are ambiguous like this.)

[armsp]

Thanks @ines That is a good starting point. It can handle cases when we have a bunch of text.
However what about the case when we have a single sentence like "I went to the bank." Is there no other way than comparing the Wikipedia list with the tokens to tell that bank is ambiguous?

[ines]

However what about the case when we have a single sentence like "I went to the bank." Is there no other way than comparing the Wikipedia list with the tokens to tell that bank is ambiguous?

In our sense2vec demo, we've added an "auto" option that tries to guess the best-matching sense. We do this by simply iterating over the available options and getting the one with the highest frequency. You can find the relevant bit of code here.

You could do something similar for your nouns and verbs, and check whether there are other entries for different part-of-speech tags (for example, check for VERB if your target word in a NOUN). You can then check the frequency and if it's reasonably high, it's a possible homonym. (You definitely want to use a high enough frequency threshold, though – otherwise, it might also turn up entries that got assigned incorrect POS tags. For example, "cat" might have been mistakenly tagged as a verb once or twice.)

[ines]

Closing this, since #77 will address a bunch of interoperability stuff with spaCy. Also, various things have changed under the hood in the meantime.