Enable union-attr mypy check and fix issues

[ancalita]

Proposed changes:

• Closes Enable mypy union-attr error code #9096

Status (please check what you already did):

• added some tests for the functionality
• updated the documentation
• updated the changelog (please check changelog for instructions)
• reformat files using black (please check Readme for instructions)

[ancalita]

There are 6 errors left unfixed:

rasa/shared/core/domain.py:1499: error: Item "bool" of "Union[bool, List[Any]]" has no attribute "__iter__" (not iterable)  [union-attr]
rasa/shared/core/training_data/visualization.py:345: error: Item "UserUttered" of "Union[ActionExecuted, UserUttered]" has no attribute "action_name"  [union-attr]
rasa/engine/validation.py:454: error: Item "None" of "Optional[ParameterInfo]" has no attribute "type_annotation"  [union-attr]
rasa/nlu/classifiers/sklearn_intent_classifier.py:243: error: Item "None" of "Optional[Any]" has no attribute "predict_proba"  [union-attr]
rasa/core/agent.py:453: error: Item "None" of "Optional[MessageProcessor]" has no attribute "log_message"  [union-attr]
rasa/core/test.py:874: error: Item "None" of "Optional[MessageProcessor]" has no attribute "parse_message"  [union-attr]

The first one appears in a domain method that will be removed once the domain loading tech debt PR gets merged.
The other 5 I found difficult to fix - so I think it's best to ignore them with the no-qa comment? Unless there are other suggestions?

[m-vdb]

🐞

[m-vdb]

for the other errors you mentioned:

rasa/shared/core/domain.py:1499: error: Item "bool" of "Union[bool, List[Any]]" has no attribute "iter" (not iterable) [union-attr]

Is there a way to fix the type definition of self.intent_properties? It's not accurate. Maybe with a TypedDict?

rasa/shared/core/training_data/visualization.py:345: error: Item "UserUttered" of "Union[ActionExecuted, UserUttered]" has no attribute "action_name" [union-attr]

I think it's because mypy cannot pickup the type of o_cleaned[i]. Have you tried defining a variable, e.g. o = o_cleaned[i] to see if mypy picks it up properly?

rasa/engine/validation.py:454: error: Item "None" of "Optional[ParameterInfo]" has no attribute "type_annotation" [union-attr]

this seems like a bug actually, because not needs_passed_to_kwargs is in fact not has_kwargs or required_type is not None, so it means that required_type can actually be None 🤔

rasa/nlu/classifiers/sklearn_intent_classifier.py:243: error: Item "None" of "Optional[Any]" has no attribute "predict_proba" [union-attr]

self.clf type is not accurate, but that's not what's causing the issue. That's weird. Does reveal_type() reveal anything?

rasa/core/agent.py:453: error: Item "None" of "Optional[MessageProcessor]" has no attribute "log_message" [union-attr]

what happens if you do a if self.processor check before?

rasa/core/test.py:874: error: Item "None" of "Optional[MessageProcessor]" has no attribute "parse_message" [union-attr]

should we raise an internal error line 826 if processor is not defined?

[ancalita]

for the other errors you mentioned:

@m-vdb I've managed to implement feedback for a few of them, but have got 2 left that cannot fix:

rasa/nlu/classifiers/sklearn_intent_classifier.py:243: error: Item "None" of "Optional[Any]" has no attribute "predict_proba" [union-attr]
self.clf type is not accurate, but that's not what's causing the issue. That's weird. Does reveal_type() reveal anything?

I used reveal_type and mypy didn't show anything 😕

rasa/core/agent.py:453: error: Item "None" of "Optional[MessageProcessor]" has no attribute "log_message" [union-attr]
what happens if you do a if self.processor check before?

I thought of this, but didn't know what to return, since it expects a DialogueStateTracker instance. Instantiate an empty one with the sender id from the user message?

[m-vdb]

for the last two:

• the first one is very weird. Maybe it's because self.clf can be None, but the type is not accurate (check the init method). It should be clf: Optional["sklearn.model_selection.GridSearchCV"] = None,.
• this one seems to be the same "agent_must_be_ready" issue. I think you can ignore it

[m-vdb]

Nice 🎉 I've left a couple of more minor comments, but it's ready to merge!

[github-actions]

🚀 A preview of the docs have been deployed at the following URL: https://10942--rasahq-docs-rasa-v2.netlify.app/docs/rasa