Is Sentence-Simularity Missing?

[TGlock]

Hi -
Great library! Until today I was successfully using await pipeline('sentence-similarity', 'sentence-transformers/all-MiniLM-L6-v2') to create embeddings in the browser.

However, I am now seeing error when "awaiting pipeline('sentence-similarity', 'sentence-transformers/all-MiniLM-L6-v2')"

unsupported pipeline: sentence-similarity. Must be one of [text-classification,token-classification,question-answering,fill-mask,summarization,translation,text2text-generation,text-generation,zero-shot-classification,automatic-speech-recognition,image-to-text,image-classification,image-segmentation,zero-shot-image-classification,object-detection,feature-extraction]

Note that https://huggingface.co/docs/transformers.js/index indicates 'sentence-similarity' is allowed.

How to reproduce
(In browser)

import { pipeline } from 'https://cdn.jsdelivr.net/npm/@xenova/transformers';
const embedder = await pipeline('sentence-similarity', 'sentence-transformers/all-MiniLM-L6-v2')

throws error

Error: Unsupported pipeline: sentence-similarity. Must be one of [text-classification,token-classification,question-answering,fill-mask,summarization,translation,text2text-generation,text-generation,zero-shot-classification,automatic-speech-recognition,image-to-text,image-classification,image-segmentation,zero-shot-image-classification,object-detection,feature-extraction]
    at Semantic.M [as _pipeline] (transformers:70:13556)

Any help illustrating how to load and use all-MiniLM-L6-v2 in the browser is greatly appreciated!

Perhaps it is now found in a new pipeline group? I cannot locate 'sentence-similarity' in the codebase... It looks like there has been some refactoring and hoping this was not purposefully lost?

[xenova]

Hi there. Yes you're right, I will add the "sentence-similarity" alias back. I think I did that because the python library doesn't call it sentence similarity, it calls it "feature extraction" instead.

In the meantime, you can use "feature-extraction" as the task instead (which does the same thing).

<script type="module">
    import { pipeline } from 'https://cdn.jsdelivr.net/npm/@xenova/transformers';
    
    let extractor = await pipeline('feature-extraction', 'Xenova/all-MiniLM-L6-v2');
    
    // Provide sentences
    let sentences = [
        'This framework generates embeddings for each input sentence',
        'Sentences are passed as a list of string.',
        'The quick brown fox jumps over the lazy dog.'
    ]

    // compute features
    let output = await extractor(sentences);

    // Convert Tensor to JS list
    output = output.tolist();

    // Compute pairwise cosine similarity
    // for (let i = 0; i < sentences.length; ++i) {
    //     for (let j = i + 1; j < sentences.length; ++j) {
    //         console.log(`(${i},${j}):`, extractor.cos_sim(output[i], output[j]))
    //     }
    // }
    
    // Or the one-liner:
    let pairwiseScores = [[output[0], output[1]], [output[0], output[2]], [output[1], output[2]]].map(x => extractor.cos_sim(...x))

    console.log(pairwiseScores)
    // [0.5022028990512902, 0.11238511099290324, 0.09594821521426729]
</script>

JSFiddle: https://jsfiddle.net/erw1zx9j/2/

For now, you'll have to do the similarity check yourself (see above), but it would probably be nice to include that in the sentence similarity pipeline (to differentiate it from feature extraction). In fact, not even the python library has that pipeline, so I could maybe bring it up with them to add it:

>>> from transformers import pipeline
>>> pipeline('sentence-similarity')
KeyError: "Unknown task sentence-similarity, available tasks are ['audio-classification', 'automatic-speech-recognition', 'conversational', 'depth-estimation', 'document-question-answering',        
'feature-extraction', 'fill-mask', 'image-classification', 'image-segmentation', 'image-to-text', 'mask-generation', 'ner', 'object-detection', 'question-answering', 'sentiment-analysis',
'summarization', 'table-question-answering', 'text-classification', 'text-generation', 'text2text-generation', 'token-classification', 'translation', 'video-classification',
'visual-question-answering', 'vqa', 'zero-shot-audio-classification', 'zero-shot-classification', 'zero-shot-image-classification', 'zero-shot-object-detection', 'translation_XX_to_YY']"

[TGlock]

Thanks for the quick reply! I suspect that will work as I had tried ‘feature extraction’ but had neglected to change the path from ‘sentence transformers’ to Xenova and I could see a 404 error as the path to the onnx file was incorrect.

[xenova]

Happy to help :) Until ONNX weight distribution becomes more popular, I (or anyone) can create copies of the model repos with a folder called onnx (containing the weights). Hopefully one day every model on the hub will come with ONNX weights, so you can pass the original model id in :)

If you would like to convert and upload your own, tag it with "transformers.js", so they can be searched on the hub: https://huggingface.co/models?other=transformers.js

[xenova]

Related: huggingface/transformers#22923

[chibchombiano26]

If anyone is interested, I have updated the example for this. This is a Vercel test: https://paragraph-similarity-calculator-6bsqzkc2n-chibchombiano26.vercel.app/. Here is the source code for the example: https://paragraph-similarity-calculator-6bsqzkc2n-chibchombiano26.vercel.app/.