Graph generated upon comparing multiple NLU pipelines is not rendered correctly

[ganeshviyer]

Rasa version: 2.3.4

Rasa SDK version (if used & relevant): (not relevant; using an NLU-only pipeline)

Rasa X version (if used & relevant): (not relevant for this issue)

Python version: 3.7.9

Operating system (windows, osx, ...): mac OS Big Sur 11.1

Issue:
See this forum topic for more info - https://forum.rasa.com/t/is-there-a-doc-on-how-to-interpret-pipeline-comparison-results/41165/2

I was looking to find a resource that would help me interpret the results of the pipeline comparison, but it appears that there are multiple graphs plotted over each other (as noted by @samsucik).

In addition, he notes that the graph also has lost information about the scale of its axes. I'm not sure what the expected result should be.

Also tagging @dakshvar22.

Error (including full traceback):
See attached file in the forum - https://forum.rasa.com/uploads/short-url/koZaCVIBB4ozHLZdbBMbnJUt9IK.pdf

Command or request that led to error:

rasa test nlu --nlu data/nlu.yml --config available_config/spacy.yml available_config/mitie.yml

Content of configuration file (config.yml) (if relevant):
spacy.yml

language: "en"  # your two-letter language code

pipeline:
  - name: SpacyNLP
    model: "en_core_web_md"
    case_sensitive: False
  - name: SpacyTokenizer
  - name: SpacyFeaturizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper

mitie.yml

language: "en"  # your two-letter language code

pipeline:
  - name: MitieNLP
    model: "data/total_word_feature_extractor.dat"
  - name: MitieTokenizer
  - name: MitieFeaturizer
  - name: RegexFeaturizer
  - name: LexicalSyntacticFeaturizer
  - name: CountVectorsFeaturizer
  - name: CountVectorsFeaturizer
    analyzer: "char_wb"
    min_ngram: 1
    max_ngram: 4
  - name: DIETClassifier
    epochs: 100
  - name: EntitySynonymMapper

Content of domain file (domain.yml) (if relevant): (not relevant here)

Definition of done

• Cause of this bug determined
• Fix, if straightforward. Otherwise, create an issue detailing the cause and potential solutions

[ganeshviyer]

In addition, could you provide me with directions on where to find the documentation for how to interpret pipeline comparison results? Many thanks in advance! :-)

[sara-tagger]

Thanks for the issue, @koernerfelicia will get back to you about it soon!

You may find help in the docs and the forum, too 🤗

[koernerfelicia]

@ganeshviyer looks like this has already been routed to the right Inbox. Thanks for submitting the issue! :)

[joejuzl]

@koernerfelicia I think this should probably be handled by research as it's "owned" by them according to the code boundaries. (rasa.nlu.test). WDYT?

[koernerfelicia]

@joejuzl yeah, that seems fair!

[tttthomasssss]

@jupyterjazz assigned as reviewer.