Add support for MultiHeadAttention layers in tensorflow.js

[batman4444]

I am working on creating and and mode in the python then converting the model over to tfjs to run inference. It does not appear that this is currently not supported.

System information

• TensorFlow.js version (you are using): tensorflow_node 4.22
• Are you willing to contribute it (Yes/No): don't have the skillset

Describe the feature and the current behavior/state.
Models created in python with a MultiHeadAttention layer will successfully be able to transfer to tfjs. Currently MultiHeadAttention layers are not supported.

Will this change the current api? How?
add support for MultiHeadAttention Layers

Who will benefit with this feature?
Anyone who is trying to use modern transformer architectures

Any Other info.

[shmishra99]

Hi @batman4444 ,

We appreciate you highlighting this. I am escalating this as a feature request to support converting models with MultiHeadAttention layer into TFJS models. Thank You!!