English-To-German Neural Machine Translation Using Transformer

Neural Machine Translation (NMT) is a family model or an approach to solving machine translation problems through an artificial neural network, typically deep learning. In other words, the model is dispatched to translate a sequence of words from the source language to the target language. In this case, the source language would be English and the target would be German. To fabricate the model, the Transformer layers are leveraged. The NMT model is trained on the Multi30K dataset. The model is then assessed on a subset of the dataset, which is the Flickr 2016 test dataset.

Experiment

Follow this link to play along and explore the NMT model.

Result

Quantitative Result

The performance of the model in terms of cross-entropy loss and translation edit rate (TER) on the test dataset.

Metrics	Score
Loss	1.951
TER	0.811

Loss Curve

Loss of training and validation versus the epochs.

Qualitative Result

Here, the NMT model's qualitative performance is associated with the Transformer's attention maps.

The attention maps from each of the Transformer's heads. Almost every corresponding word pair (English-German) at each head pays attention mutually.

Credit

• Language Translation With TorchText
• 6 - Attention is All You Need Notebook Tutorial
• Multi30K Dataset
• Attention Is All You Need
• Self-Attention and Positional Encoding
• PyTorch Lightning