Compare new luis model vs current production (batch testing)

[bartbilliet]

Describe the solution you'd like

Compare a new Luis model in Composer vs Luis model in production

Luis has a batch testing feature which allows to check if your newly added utterances will not negatively affect existing intent recognition.
It would be great to have the ability to run batch testing from Composer to see upfront how changes made to User Input will influence the intent recognition results, compared to the current Luis model used in production. When implementing this type of feature, it should be possible to run batch testing in Composer before publishing updates to the bot in Azure.

[hibrenda]

BF CLI tool (bf luis :test) has already support the luis testing through .lu file format. We may need more design in UX part. @sangwoohaan @mareekuh .

[mareekuh]

Agreed - we need lu testing, better integrated with the product. I have some designs for this but they're older.

[bartbilliet]

@mareekuh if you need feedback, let me know 😉

[tdurnford]

@mareekuh Do you have any updates to report?

[mareekuh]

I do not - it's on the Design backlog.
@bartbilliet - I am very familiar with the feature/functionality that is currently available in LUIS, but it would help to get more specifics around the scenarios in which batch-testing and/or evaluation of language model/understanding would enable/improve your/your customers' bots. Or, without these capabilities, the hurdles you experience or need to find work-arounds for? I can incorporate these when defining and prioritizing solutions.

groetjes,
Marieke