Graph Executor does not track mutable input dependencies

[samskalicky]

Description

In the Symbol/Module flow, the graph executor does not track mutable input dependencies when setting up the use_vars and mutate_vars arrays here:
https://github.com/apache/incubator-mxnet/blob/master/src/executor/graph_executor.cc#L1222-L1234

Notice that inputs are only added to use_vars on line 1224, and mutate_vars only contains outputs.

Compare this to the CachedOp flow in Gluon where we get the mutable inputs here:
https://github.com/apache/incubator-mxnet/blob/master/src/imperative/imperative_utils.h#L245

and then add all mutable inputs to the write_vars array on line 294 (here read_vars is the same as use_vars, and write_vars is the same as mutate_vars in the graph executor above).

This means that its possible that write dependencies would not be tracked correctly in the symbol/module flows for operators that use the FMutateInputs attribute. Currently it looks like only BatchNorm uses this.

Test case

This can be tested using BatchNorm, by creating a symbol graph that only contains batch norm and run with the same running mean and variance many times

[mxnet-label-bot]

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Feature, Bug