Add functional normalizations

[kaiidams]

• torch.nn.functional.batch_norm
• torch.nn.functional.group_norm
• torch.nn.functional.instance_norm
• torch.nn.functional.layer_norm
• torch.nn.functional.local_response_norm

[kaiidams]

2788171

Unittest TestGriffinLim() failed, but probably not related to the commit.

https://dev.azure.com/dotnet/TorchSharp/_build/results?buildId=76783&view=logs&j=9c097a9c-392c-59c2-1a86-9d92fd9d1b40&t=145477f8-80b5-526e-c5ab-a59ef63ac75a&l=169

[xUnit.net 00:00:13.36]     TorchSharp.TestTorchAudio.TestGriffinLim [FAIL]
  Failed TorchSharp.TestTorchAudio.TestGriffinLim [611 ms]
  Error Message:
   System.AccessViolationException : Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
  Stack Trace:
     at TorchSharp.torch.Tensor.THSTensor_stft(IntPtr x, Int64 n_fft, Int64 hop_length, Int64 win_length, IntPtr window, Boolean normalized, Int64 onesided, Boolean return_complex)
   at TorchSharp.torch.Tensor.stft(Int64 n_fft, Int64 hop_length, Int64 win_length, Tensor window, Boolean center, PaddingModes pad_mode, Boolean normalized, Nullable`1 onesided, Nullable`1 return_complex) in D:\a\1\s\src\TorchSharp\Tensor\Tensor.cs:line 7119
   at TorchSharp.torchaudio.functional.griffinlim(Tensor specgram, Tensor window, Int64 n_fft, Int64 hop_length, Int64 win_length, Double power, Int32 n_iter, Double momentum, Nullable`1 length, Boolean rand_init) in D:\a\1\s\src\TorchSharp\TorchAudio\Functional.cs:line 210
   at TorchSharp.TestTorchAudio.TestGriffinLim() in D:\a\1\s\test\TorchSharpTest\TestTorchAudio.cs:line 150

[NiklasGustafsson]

@kaiidams -- thank you for your prolific contributions!

[MiWeiss]

@kaiidams

I was looking for torch.nn.functional.normalize but it appears not to be part of TorchSharp. Considering looking into opening a PR for this functionality I stumbled over this PR. Do you remember if there was any reason why you did not include torch.nn.functional.normalize?

[NiklasGustafsson]

No, I do not. Open a new issue to track, before you do a PR.

[kaiidams]

The PR added functional versions of normalization layers like torch.nn.LayerNorm and BatchNorm. torch.nn.functional.normalize is for normalization of vectors. They are both named "normalize" but are different. I think you can implement them and make a PR if you need them.

[MiWeiss]

Thanks @NiklasGustafsson and @kaiidams for your answers!