Add empty recognizer support for CrosstrainedRecognizer (composer scenario)

[luhan2017]

Fixes #

Description

We plan to set default recognizer in composer as Luis+QnA CrosstrainedRecognizer.
And it will fit into different recognizer types by the .lu/.qna content.

Recognizer: "dialogname.lu.qna"

dialogname.lu.qna,dialog will be generated as the following rule:

.lu and .qna both not empty =>

{ 
    "$kind": "Microsoft.CrossTrainedRecognizerSet" 
    recongizers:[ 
        "dialogname.lu", 
        "dialogname.qna” 
   ] 
} 

.lu and .qna both empty =>

{ 
    "$kind": "Microsoft.CrossTrainedRecognizerSet" 
    recongizers:[ 
   ] 
} 

.lu empty =>

{ 
    "$kind": "Microsoft.CrossTrainedRecognizerSet" 
    recongizers:[ 
        "dialogname.qna” 
   ] 
} 

.qna empty =>

{ 
    "$kind": "Microsoft.CrossTrainedRecognizerSet" 
    recongizers:[ 
        "dialogname.lu"
   ] 
} 

This change is to support the empty recognizer scenario in CrossTrainedRecognizer.

Specific Changes

Testing

[luhan2017]

@vishwacsena , I've synced with @feich-ms , here are two things need to confirm with you.

• The empty check should be supported in the cli side. currently the empty check happened in composer side, and composer will send a list of .lu file which is not empty to lubuild. But to support the default luis+qna recognizer as the above, we need to support empty check in the cli said, which means composer will send all the file list to cli, cli should check the file and decide what to generate for the crosstrainrecognizer.
• as Fei said, if we have a.dialog, b.dialog, c.dialog in one bot, for the qna, it will be merged in to single one: "a.qna", it should the recognizer of b and c looks like, should they always include a.qna as the recognizer? what if b or c's qna file is empty?

[vishwacsena]

@luhan2017 @feich-ms

1. Yes. It makes sense for composer to just post all .lu and .qna files and for luis:build and qnamaker:build to be smart about what it does when it detects empty content. This means that if there is no .lu or .qna content, the library should not write out an empty luis and qnamaker recognizer configuration but the higher up structure can be true I guess but needs to be validated.
2. For QnA, since all dialogs share a single KB, I believe it is ok to continue to merge content from a.qna, b.qna, c.qna into a single KB. Now if only a.qna and c.qna have content in them, we should have the recognizer for those dialogs configured to also include the qnamaker recognizer. I thought we already handle this in qnamaker:build.. @feich-ms ?

It might also be helpful to expose and use a hasContent API for LU/ QnA in bf-lu in case composer needs it.

[tomlm]

.Count() == 0 [](start = 27, length = 13)

.Any() == false is more efficient.

[tomlm]

[feich-ms]

2. For QnA, since all dialogs share a single KB, I believe it is ok to continue to merge content from a.qna, b.qna, c.qna into a single KB. Now if only a.qna and c.qna have content in them, we should have the recognizer for those dialogs configured to also include the qnamaker recognizer. I thought we already handle this in qnamaker:build.. @feich-ms ?

@vishwacsena yes, qnamaker:build already did like this.

I think the requirement from @luhan2017 is that, for empty lu and qna file, bf-cli should generate empty CrossTrainedRecognizerSet file like below:

{ 
    "$kind": "Microsoft.CrossTrainedRecognizerSet" 
    recongizers:[ 
   ] 
} 

Currently, for empty lu or qna file, bf-cli will throw exceptions when pasing empty content here https://github.com/microsoft/botframework-cli/blob/master/packages/lu/src/parser/lu/luMerger.js#L514.

So maybe we can adjust the logic to not throw exceptions and generate emtpy cross-trained recognizer but of course need validation from runtime.

[feich-ms]

@luhan2017 @vishwacsena BF-CLI related changes are here microsoft/botframework-cli#870. Now for empty lu and qna files, luis:build and qnamaker:build will write out empty crosstrained recognizer instead of throwing exceptions. Please note that only empty crosstrained recognizer will be generated, single recognizer and multiLanguage recogizer will not write out.