Dont overwrite hook handles in flop profiler

[Sanger2000]

This is a solution to #2046.

As explained there:

I ran into the same issue. The problem here is that huggingface instantiates activation function modules like NewGELUActivation at the python global scope. So, when deepspeed recursively registers hooks to model, several forward hooks are added for the same NewGELUActivation object. But, they all overwrite the same removable handle attributes saved to the object. Then, when deepspeed tries to remove these hooks, it can only remove 1, leaving several stale hooks on the module.

[ghost]

All CLA requirements met.

[cli99]

Thanks for the PR.