Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add a flag to disable plotting confusion matrix in test #3549

Closed
Ghostvv opened this issue May 22, 2019 · 3 comments · Fixed by #4951
Closed

add a flag to disable plotting confusion matrix in test #3549

Ghostvv opened this issue May 22, 2019 · 3 comments · Fixed by #4951
Assignees
Labels
area:rasa-oss 🎡 Anything related to the open source Rasa framework type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR

Comments

@Ghostvv
Copy link
Contributor

Ghostvv commented May 22, 2019

Rasa version: master

Python version: 3.7

Operating system (windows, osx, ...): macos

Issue:
Plotting confusion matrix can be time consuming and sometimes we don't care for it, therefore it makes sense to add an option to ignore the plot. Also, for nlu, I think it is better to create pdf directly instead of creating matplotlib windows first (it doesn't work on gcloud from terminal)

in case of --debug option, still turn off matplotlib debug

Another thing right now confusion matrix is unreadable (at least for core), color boxes and numbers are not aligned, also numbers overlay:
story_confmat

@tmbo tmbo added the type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR label May 22, 2019
@wochinge wochinge added the area:rasa-oss 🎡 Anything related to the open source Rasa framework label Nov 28, 2019
@degiz
Copy link
Contributor

degiz commented Dec 11, 2019

I've tested current (master) behaviour in our docker image (that would pick up Matplotlib TkAgg backend), here are results:

  1. "rasa test nlu" renders confmat.png and hist.png, no issues with pup-up window
  2. "rasa test coe" renders story_confmat.pdf, no issues pop-up window

On Mac OS I also don't have any pop-ups for both of the commands.
So if there was an issue with creating pop-up window on headless environments, it's not there anymore.

So I'd suggest that this ticket's goal is to introduce "--no-plot" flag with default value set to "false" that would completely skip rendering of confusion matrix and histogram.

@Ghostvv if there's an issue with readability of confusion matrix I think we should create a separate "enhancement" issue and track it separately.

@Ghostvv
Copy link
Contributor Author

Ghostvv commented Dec 11, 2019

sure, we should tackle them separately

@wochinge
Copy link
Contributor

@degiz moved this to review-required

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:rasa-oss 🎡 Anything related to the open source Rasa framework type:enhancement ✨ Additions of new features or changes to existing ones, should be doable in a single PR
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants