Skip to content

Commit

Permalink
Add model 2023-09-14-distilbert_base_uncased_aisera_texts_v3_en
Browse files Browse the repository at this point in the history
  • Loading branch information
ahmedlone127 committed Sep 15, 2023
1 parent d4f7b32 commit 83a4a45
Showing 1 changed file with 93 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
layout: model
title: English distilbert_base_uncased_aisera_texts_v3 DistilBertEmbeddings from Theimisa
author: John Snow Labs
name: distilbert_base_uncased_aisera_texts_v3
date: 2023-09-14
tags: [distilbert, en, open_source, fill_mask, onnx]
task: Embeddings
language: en
edition: Spark NLP 5.1.2
spark_version: 3.0
supported: true
engine: onnx
annotator: DistilBertEmbeddings
article_header:
type: cover
use_language_switcher: "Python-Scala-Java"
---

## Description

Pretrained DistilBertEmbeddings model, adapted from Hugging Face and curated to provide scalability and production-readiness using Spark NLP.`distilbert_base_uncased_aisera_texts_v3` is a English model originally trained by Theimisa.

{:.btn-box}
<button class="button button-orange" disabled>Live Demo</button>
<button class="button button-orange" disabled>Open in Colab</button>
[Download](https://s3.amazonaws.com/auxdata.johnsnowlabs.com/public/models/distilbert_base_uncased_aisera_texts_v3_en_5.1.2_3.0_1694735601782.zip){:.button.button-orange.button-orange-trans.arr.button-icon}
[Copy S3 URI](s3://auxdata.johnsnowlabs.com/public/models/distilbert_base_uncased_aisera_texts_v3_en_5.1.2_3.0_1694735601782.zip){:.button.button-orange.button-orange-trans.button-icon.button-copy-s3}

## How to use



<div class="tabs-box" markdown="1">
{% include programmingLanguageSelectScalaPythonNLU.html %}
```python


document_assembler = DocumentAssembler() \
.setInputCol("text") \
.setOutputCol("documents")


embeddings =DistilBertEmbeddings.pretrained("distilbert_base_uncased_aisera_texts_v3","en") \
.setInputCols(["documents","token"]) \
.setOutputCol("embeddings")

pipeline = Pipeline().setStages([document_assembler, embeddings])

pipelineModel = pipeline.fit(data)

pipelineDF = pipelineModel.transform(data)

```
```scala


val document_assembler = new DocumentAssembler()
.setInputCol("text")
.setOutputCol("embeddings")

val embeddings = DistilBertEmbeddings
.pretrained("distilbert_base_uncased_aisera_texts_v3", "en")
.setInputCols(Array("documents","token"))
.setOutputCol("embeddings")

val pipeline = new Pipeline().setStages(Array(document_assembler, embeddings))

val pipelineModel = pipeline.fit(data)

val pipelineDF = pipelineModel.transform(data)


```
</div>

{:.model-param}
## Model Information

{:.table-model}
|---|---|
|Model Name:|distilbert_base_uncased_aisera_texts_v3|
|Compatibility:|Spark NLP 5.1.2+|
|License:|Open Source|
|Edition:|Official|
|Input Labels:|[documents, token]|
|Output Labels:|[embeddings]|
|Language:|en|
|Size:|247.2 MB|

## References

https://huggingface.co/Theimisa/distilbert-base-uncased-aisera_texts-v3

0 comments on commit 83a4a45

Please sign in to comment.