[v6] Implement ML-Kit Natural Language

[Salakar]

[ml-kit-natural-language]

firebase.ml().nlp

• JS
• iOS
• Android
  • Language ID
  • Translate Language
  • Smart Replies
• Tests (95-100% coverage)
• 100% Firebase API coverage
• Flow Types
• Typescript Definitions
• Update package README

firebase.json supported flags

Added support for the following flags to support optional models (so they're not all bundled together into your app unless you decide you need them):

{
  "react-native": {
    "ml_natural_language_id_model" : true,
    "ml_natural_language_translate_model" : true,
    "ml_natural_language_smart_reply_model" : true
  }
}

[RWOverdijk]

yesyesyesyesyesyesyesyesyesyesyesyesyesyesYESYEEEEES

Cool.

[Salakar]

Hey @RWOverdijk, @sejas, @zhigang1992, @gianpaj, @fungilation and others interested in ML Kit.

If you can, please could you provide your feedback/thoughts on the API for the ML Kit Natural Language integration, as I'm currently unsure if this API is clear enough or if anything needs changing?

// -- LANGUAGE_LANGUAGE_ID
// --------------------------
firebase.ml().language().identifyLanguage(text): Promise<string>;

// -- LANGUAGE_SMART_REPLIES
// --------------------------
firebase.ml().language().newSmartReplyConversation(): SmartReplyConversation;

//    -> SmartReplyConversation
//      -> destroy(): void;
//      -> clear(): void;
//      -> getSuggestedReplies(): Promise<SuggestedReply[]>;
//      -> addLocalUserMessage(message, timestamp = Date.now()): void;
//      -> addRemoteUserMessage(message, timestamp = Date.now(), remoteUserId): void;

// -- LANGUAGE_TRANSLATE
// --------------------------
firebase.ml().language().translateText(text, options): Promise<string>;

firebase.ml().language().translateGetAvailableModels(): Promise<Object[]>;

firebase.ml().language().translateDeleteDownloadedModel(languageId: Number): Promise<void>;

firebase.ml().language().translateDownloadRemoteModel(languageId: Number, downloadConditions: Object): Promise<void>;

[RWOverdijk]

@Salakar This reply is me being in the middle of something and the quality might be poor so... Sorry if that's the case 😅

My first question would be, how close is it to the firebase APIs themselves? I'm not working with react-native-camera and its MLKit implementation for detecting faces (ick) and there's almost no resemblance making it hard to debug stuff.

Secondly I wonder why every subset is a function. It feels like a lot is being constructed that I wouldn't use (but I might be off).

Third, I don't think I dig the translate prefix. Are you translating a delete of a downloaded modal, or are you deleting a downloaded model in the namespace translate? To follow the pattern you'd have to use .translate() and then .deleteDownloadedModel for example.

• .translate().text()
• .translate().deleteDownloadedModel()
• .translate().availableModels()

etc.

I hope this helps.

[gianpaj]

I don't know anything about these natural language methods in ML kit.
But when the Vision API will be developed I would probably look this official Flutter Firebase repo:
https://github.com/flutter/plugins/tree/master/packages/firebase_ml_vision
and this example app
https://github.com/firebase/mlkit-custom-image-classifier

Would be great to make an example app like that.
I'll be happy to help out with that one we have a draft API working

[Salakar]

Preview vid of smart replies working, if you want to sneak peak: https://twitter.com/mikediarmid/status/1128837402481635331

---

Will come back to the API discussion, getting a basic JS API in place and completing Android / iOS code first - can re-work JS API if needed once native stable.

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@9083803). Click here to learn what that means.
The diff coverage is 97.09%.

@@            Coverage Diff            @@
##             master    #2117   +/-   ##
=========================================
  Coverage          ?   90.37%           
=========================================
  Files             ?       42           
  Lines             ?      934           
  Branches          ?        0           
=========================================
  Hits              ?      844           
  Misses            ?       90           
  Partials          ?        0

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (master@9083803). Click here to learn what that means.
The diff coverage is 97.09%.

@@            Coverage Diff            @@
##             master    #2117   +/-   ##
=========================================
  Coverage          ?   90.37%           
=========================================
  Files             ?       42           
  Lines             ?      934           
  Branches          ?        0           
=========================================
  Hits              ?      844           
  Misses            ?       90           
  Partials          ?        0

[RuairiOliver]

Hi @Salakar,

Can translation be enabled? I see you have a lot of code in there for it but it's all commented out.

Thanks