Skip to content
This repository was archived by the owner on Jan 15, 2025. It is now read-only.

Comments

Optimize LUFileParser.g4 to accelerate parsing of simple intent section with a large number of entities#677

Merged
feich-ms merged 5 commits intomasterfrom
feich/luParserRefactor
Mar 20, 2020
Merged

Optimize LUFileParser.g4 to accelerate parsing of simple intent section with a large number of entities#677
feich-ms merged 5 commits intomasterfrom
feich/luParserRefactor

Conversation

@feich-ms
Copy link
Contributor

fix issue #676. Before the fix, the lu parser takes unreasonable time to parse lu content with a large number of entity definitions under an intent. e.g. 100 entities will cost 4000+ milliseconds. After the fix, it takes 200-300 milliseconds.

@feich-ms feich-ms added the lu non luis api commands label Mar 19, 2020
@feich-ms feich-ms requested a review from vishwacsena March 20, 2020 00:04
@boydc2014
Copy link
Contributor

Please attach with root cause analysis and the explanation of the fix

@feich-ms
Copy link
Contributor Author

Root cause: entitySection(and newEntitySection) are child rules of both paragraph and simpleIntentSection, and simpleIntentSection is child rule of paragraph, that means each entitySection following an intent will first be identified as entitySection in paragraph and then look back to search bigger scope to match simpleIntentSection. When there are a large number of entities, this knid of back search will be very expensive.

Why this fix works: To fix this, we can remove entitySection(and newEntitySection) from either paragraph or simpleIntentSection to avoid this kind of back search. We choose removing it from paragraph. But to keep our logic that entitySection can be standalone, we optimize the simpleIntentSection rule to make it able to match standalone entitySection. Below simpleIntentSection rule will match standalone intent, standalone entity, intent with entities. And then we adjusted the code in luParser.js to consume the new standalone entity context. This fix solve our issue and reduce the paring time signally.

simpleIntentSection
    : (intentDefinition? (entitySection | newEntitySection)+) | intentDefinition
    ;

@feich-ms
Copy link
Contributor Author

@vishwacsena @munozemilio I merged this minor fix of parser rules, please let me know if you have concerns.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

lu non luis api commands

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants