"Repeat this dialog" action does not properly repeat current dialog context when consuming Skill

[johnkm516]

Describe the bug

"Repeat this dialog"'s expected behavior is re-evaulate the user's utterance from the Root bot's recognizer (or at least the dialog in which this action is called), not the skill's recognizer. Even though the Repeat this dialog action is in the Root bot's dialog the dialog's context is still in the Skill, not the root bot.

Version

OS

• macOS
• Windows
• Ubuntu

To Reproduce

Steps to reproduce the behavior:

1. Download Enterprise Assistant template
2. Change recognizer from Orchestrator to Default (LUIS)
3. Call a skill via utterance such as "Who are John's peers"
4. If "Who are John's peers" results in multiple results, and user inputs a different utterance without selecting from the multi-prompt, the recognizer's intent of the user's new input results in "None", and the bot doesn't properly handle the user's new utterance (the bot only handles utterances in the People skill, so it is properly interrupted if I input an utterance related to the People skill, but not the root bot).

Expected behavior

From the Enterprise Assistant template's comments, the flow of dialog should be like this :

user : Who are John's peers?
Bot : Here is a list of John's peers : 
user : What's my schedule for tomorrow? (User inputs new utterance without selecting from previous multi-prompt)
Bot : Your schedule for tomorrow is :

Current behavior

user : Who are John's peers?
Bot : Here is a list of John's peers : 
user : What's my schedule for tomorrow? (User inputs new utterance without selecting from previous multi-prompt)
Bot : *LUIS intent recognizer points to None, bot outputs no response. 

Screenshots

User triggers getPeers intent, bot responds with list of peers multi-prompt.
User inputs "What are you" (expected behavior is to route this utterance from the root bot's recognizer so it points to QnAMaker's chitchat KB)
Recognizer's intent routes to None instead or other intents with very low probability via the skill's recognizer, NOT root bot's recognizer.

The expected behavior as per Enterprise Assistant's comment in ConnectToSkill :

Additional context

Possibly related to this issue over on botbuilder-dotnet : https://github.com/microsoft/botbuilder-dotnet/issues/3930

Possible related issue and resource for minimum reproducible code : https://stackoverflow.com/questions/67541109/bot-framework-how-do-i-stop-a-waterfall-dialog-which-is-waiting-for-a-prompt-re

This might be a feature request if not a bug. I would like an action that cancels all dialogs and routes the user input starting from the beginning as if the user is starting a new conversation. "Manually" triggering a user input if you will. I can get the user's message from turn.activity.text, all I want is the ability to call an action that stops all dialogs and re-call's LUIS all the way from the beginning of the stack at the root bot without the user having to manually do it (in order for the user to "manually" do it, they have to repeat the same utterance 2-3 times to exit out of the skill context)

If anyone has any suggestions / workarounds on how to get the behavior I want it would be greatly appreciated.

[lauren-mills]

Hi @johnkm516, I think the issue with this scenario not working for you as the Enterprise Assistant is designed today is that ChitChat is the UnknownIntent fallback, rather than a peer intent to the People intent. Interruptions never fire for the None intent of parent dialogs to prevent interrupting ongoing flows whenever there is a query the parent bot doesn't understand.

Have you tested this interruption flow with any other intents besides the one that routes to ChitChat? If it works for another intent, then you may need to change how chitchat is added to your Enterprise Assistant or you may even want to add an additional knowledge base that specifically handles your custom QnA pairs to your root LUIS intent recognition.

Please let us know if you are able to interrupt the People skill with any other intents and we can go from there with guidance on how you may want to change the root bot QnA setup to work like you want.

[johnkm516]

@lauren-mills
Editing out the entire last post since I was approaching the problem the wrong way and this post was only going to confuse people.

[johnkm516]

@lauren-mills
After looking at the ListDialog UnknownIntent, which in the Enterprise Assistant Template says,

If the dialog has been launched with $doSelection set to false, the list card will be displayed and the dialog will not automatically end to allow the user to say \"next\" or \"previous\" to navigate between the pages. If in that mode, the user says an unknown utterance that no dialog in the stack can handle, the message will land here and should bubble the unknownIntent event back to the parent.

This part was not working correctly for me. ListDialog's unknownintent emitting an unknownintent would trigger itself in an infinite loop instead of actually bubbling up to parent (despite bubble up = yes being set). So I changed unknown intent into None intent, which solved the issue. Emitting the unknown intent correctly bubbles up to the People dialog, which triggers unknown intent and ends dialog with property set to False. But then the bot just ends there, without going back to parent dialog in Root Bot (ConnectToSkill) and returning the $skillHandled value (in fact I put a trace just after Connect To Skill command and that trace won't even print).

So in an effort to fix this, instead of emitting unknown intent in ListDialog, I put End this Dialog with property set False starting from the top of the stack in ListDialog to People dialog's OnIntent GetPeers. In a series of,
if (not(dialogHandled)) { End this Dialog (return false) }.
This allowed me to actually return a False value back to RootBot, so I can check for $skillHandled properly. However, as per the original post, even when $skillHandled = False and AfterSkillComplete trace is triggered, "Repeat this Dialog" doesn't fire properly due to the bug I am describing. Again, this is specifically in the case when you try to Repeat this Dialog after consuming a skill; the root bot is stuck in the skill's turn context and there is no way of getting out of it.

Here are the screenshots to describe what I'm talking about :

As seen in the screenshot :

1. The ListDialog adaptive card response. This response is showing the paginated list of results for GetPeers.
2. A user utterance completely unrelated to People Skill. The desired behavior is for the bot to remove dialogs from the top of the stack back to the Root Bot, and then have the Root bot recognize the utterance in the Root Bot's LUIS recognizer.
3. AfterSkillComplete is triggered from this if block in ConnectToSkill in Root Bot.
   
4. Following 3), I have tried "Repeat this Dialog" as the template originally had. Nothing happens. So just as a test to see what happens I put Emit Event : UnknownIntent, just to see if that triggers, which it did. I don't know what Repeat this dialog won't trigger.

Please let me know if you can replicate this behavior.

[lauren-mills]

Hi John, thanks for these detailed steps. I will take a look this week and see if I can figure out what's going wrong here. There may be an issue in the underlying SDK that's affecting this behavior.

[johnkm516]

@lauren-mills
I do think there is something wrong with the turn context when the root bot consumes a skill.
I investigated a bit further, and as seen in the screenshot in my previous post, the "Skill State" trace always comes before the "Bot state" trace. The expected behavior is that the bot exits the skill and print the "Skill State" after returning from the Connect To Skill call, but instead of returning from the skill and unstacking back to the Root Bot's dialog state, the Root Bot's turn context is stuck in the skill and there is nothing I can do at this point of the dialog to exit out of the skill, other than Cancel All Dialogs.

Just as a note, the root bot does interrupt correctly if I specifically invoke an intent. For example, if I say "cancel" to invoke the cancel intent, the root bot routes correctly. The problem is when I say something that the Root Bot doesn't expect; it doesn't route to the unknown intent. This part is expected behavior since the Root Bot doesn't keep track of ListDialog's intents. If I can just get Repeat This Dialog to work now that I've worked backwards from the stack, it would achieve the behavior I want. My plan is to query a backend API for my company whenever the root bot's dialog state reaches unknownintent, and then route to QnAMaker afterwards if my backend API has no result.

Edit :
I am noticing that the Skill State is showing endOfConversation.
Perhaps the Skill State's endOfConversation is propagating to the Root Bot, so Repeat this Dialog does nothing when called.
Another clue is that when I tried emitting unknown intent in the root bot after the Connect To Skill action, despite the unknown intent event firing, turn.activity.text is null.

[lauren-mills]

I'm still looking into this. I do see the same behavior. The first thing I noticed was the people skill doesn't have score thresholds set up on the top level intents which will hurt the ability for the chitchat to ever fire even if everything else works (which it doesn't). On my end, I'm not seeing that anything after the Connect To Skill actions is firing at all, including the RepeatDialog action so I'm still looking into that. Thanks for your patience.

[johnkm516]

@lauren-mills
Thank you for looking into it.

I'm not seeing that anything after the Connect To Skill actions is firing at all, including the RepeatDialog action so I'm still looking into that.

I mention this issue above. The way I solved it is to simply do a chain of End this Dialog returning False, and then adding
if (not(dialogHandled)) { End this Dialog (return false) }.
at each dialog of the dialog call stack, so we work backwards from the stack. In the current way the Enterprise Template is implemented, triggering unknown intent in ListDialog (even if it did work correctly and routed to People dialog), I think would add more onto the top of the stack, which I don't think is desired behavior. Here are screenshots to show what I did :

All "End this Dialog" in the screenshots return False.
I had to update all dialogs in a similar fashion, such as Get Collaborators, Get Direct Reports, etc (you can try doing this with just GetPeers to test my solution).
In the case of GetUserDialog, I first check if userID exists, then check if userID = False before moving onto the rest of the dialog.

Doing this allows me to actually return from the Connect To Skill command returning False, so AfterSkillComplete does print. But again, the problem is Repeat This Dialog doesn't work even after I unstack from all the dialogs in People skill and return to the Root Bot.

I think it's safe to say this is a problem with the SDK...