Add LayerNorm support for Vivado

[rianbrooksflynn]

Description

This PR adds support for Layer Normalization using either Keras or PyTorch with the Vivado backend in io_parallel mode.

This implementation uses a lookup table for inverse square root; the inputs to the lookup table follow a logarithmic distribution for better accuracy.

Tests have been added for both Keras and Pytorch parsing.

Credit is due to @Ethan0Jiang and @LostEcho365 (Zhixing Jiang and Dennis Yin) for their Vivado implementation and Keras parsing support; my contributions were making a change to the inverse square root lookup table implementation, implementing PyTorch support, and adding unit tests. (Here's a link to their pre-print.) The original code authors have given permission for their code to be merged into hls4ml.

Linked issue: #1109

Type of change

• New feature (non-breaking change which adds functionality)
• A new research paper code implementation

Tests

Two unit tests added: test/pytest/test_layernorm.py and test/pytest/test_layernorm_pytorch.py

Checklist

• I have read the guidelines for contributing.
• I have commented my code, particularly in hard-to-understand areas.
• I have made corresponding changes to the documentation.
• My changes generate no new warnings.
• I have installed and run pre-commit on the files I edited or added.
• I have added tests that prove my fix is effective or that my feature works.

[jmitrevs]

For the "broken" diffs, we should look to see what the whitespace error is and fix it before merging.

[JanFSchulte]

I did a bit of digging and it's not a whitespace problem but rather the file endings are improperly encoded. Likely the person we got the code from was using a Windows machine. @rianbrooksflynn, you can install the dos2unix package and just do dos2unix file.name to fix this.

[jmitrevs]

We should squash the commits when we merge this.

[vloncar]

Looks good, could use some minor cosmetics.

[vloncar]

These attributes should be set as default in _register_layer_attributes() not here.

Also, is 4096 necessary for this implementation to work? All other tables are 1024.

[rianbrooksflynn]

Unfortunately according to my tests, 4096 is necessary to achieve absolute differences of < 0.05 in accuracy.

[rianbrooksflynn]

Thanks @vloncar for your review and apologies that it took me until now to get around to implementing your feedback! I would appreciate a second look from you at this pull request.

[rianbrooksflynn]

pre-commit.ci autofix

[JanFSchulte]

Fixed the latest merge conflicts. @vloncar, can you check if Rian's fixes are enough to have this merged?