Unexpected behavior while adding chitchat source to .qna file and cross training.

[DishaAgarwal]

If multi-turn prompts and chitchat source are included on the same file, chitchat prompts were somehow getting linked to unrelated follow up prompts from the same .qna file (if id is not assigned to the follow up prompts).

PFA screenshot of KB for reference

"Are you old" was a question from the chitchat qna and store hours had two prompts: seattle stores & portland store defined.

PFA project folder:

08.todo-bot-luis-qnamaker.zip

[vishwacsena]

@feich-ms I can repro the issue and do not repro it with a simple qnamaker:convert but can repro with qnamaker:build. Can you take a look at what's causing it and prep a fix? You can use the zip file @DishaAgarwal provided and follow the instructions in readme.md to setup/ repro the bot. The issue is that somehow the prompts are linked to incorrect IDs.

[feich-ms]

sure, will take a look.

[feich-ms]

@vishwacsena I have figured out the root cause. As you know, we are merging qna contents from multiple files into one and then publish the merged one. In above generated qna files, the chitchat.qna has id for each QA pair, but the QA pair of store hours in RootDialog.qna has no explicit ids. We will resolve the multi turn prompts and generate ids for them when parsing RootDialog.qna. We will have ids for Seattle store and Portland store, but unfortunately the ids for prompts Seattle store and Portland store may be same as the ids pre-defined in chitchat.qna, And if the same ids in chitchat.qna are placed before the ids in RootDialog.qna when after merging these two files, the multi turn prompt will run into above issues.

• In short term: 1. remove all ids in chitchat.qna or 2. make sure chitchat.qna is imported at last which means you can use -qnaConfig to specify the orders when loading multiple qna files. Below is the qna config file to specify qna file orders. We can change the cmd to bf qnamaker:build --qnaConfig ..\generated\qnaConfig.json --out ..\generated --botName TodoBotWithLuisAndQnA --log --subscriptionKey <Your QnA subscription key>

qnaConfig.json:

{
    "models": [
        "AddToDoDialog.qna",
	"DeleteToDoDialog.qna",
	"GetUserProfileDialog.qna",
	"RootDialog.qna",
	"ViewToDoDialog.qna",
	"ChitChat.qna"
    ]
}

After applied qna config, the multi turn works now.

@DishaAgarwal the short solution can unblock you quickly if you want to have a try.

• In long term: adjust the logic in qna builder to auto detect such id overlapping scenarios and throw errors to let users resolve the ids.

How do you think? @vishwacsena

[DishaAgarwal]

@feich-ms Thanks for the analysis! It seems right as this issue doesn't occur if i explicitly specify ID to the QnA pair for 'store hours'. While I am not blocked in the short term, I believe it would be good to work on getting this resolved.
@vishwacsena How do you think we should go ahead with the resolution? If the user doesn't specify the ID and a overlap occurs should we throw an error or auto-increment ID?

[vishwacsena]

@feich-ms I had logic in the core parser to handle this in that we would do this ..

1. Resolve follow up questions and add IDs to the prompts since they need to be linked via ID
2. ID all content in the document
3. If collating, then re-ID all qna pairs to start linearly but ensure the follow up prompt IDs match the right QnA from the right document

Have you already look at this code? I suppose we could add logic there to ensure that when collating multiple qna documents, we handle qna pairs with prompts before we merge other QnA pairs?

[feich-ms]

@vishwacsena thanks for the suggestions. I know the logic there but I only resolved the multi turn IDs in each file and then directly merge them, that means in each file the IDs are guarteened right but it may run into conflicts after merging with other files that also have same ids. Based on your suggestion, I think we can do another round re-ID after merging them to make sure the multi turn ids are always matched. I will take a fix soon. Thanks.

[feich-ms]

Fixing in this PR #902