[Question]: How to train an end-to-end Entity Linking model?

[anna-shopova]

Question

Hello,

I am interested in implementing an end-to-end Entity Linking model using a custom dataset. I would greatly appreciate any guidance on how to proceed, particularly concerning the required input data format, the training procedure, and the inference process.

Thank you in advance for your time and assistance.

[stefan-it]

Hi,

I would also be interested in that!

Unfortunately, I no longer can find the example training code that @alanakbik sent me some time ago 😅

[anna-shopova]

Hi @stefan-it, you could have a look at this Tutorial which I found in PR 3369.

[anna-shopova]

As I see you already found it 😁

[anna-shopova]

Thanks a lot for the tutorial @helpmefindaname!

I trained a Multitask model with aligned training data. However, when I use the model for predictions, I can only get the "ner" predictions, but not the "nel" or both together.

When using two corpuses, there is this trick where we set the label type to be the same ("ner"). Is there a similar trick or how should I train/use the multitask model with a single corpus for both tasks?

[helpmefindaname]

Hi @anna-shopova
I think that feature only existed in my head so far 😅
should be as simple as this
I still need to test it myself tho

[shigapov]

I can confirm what @anna-shopova wrote about training NER/NEL model via a multitask model. It predicts only NER-tags. But it might be a problem with saving a multitask model, see #3446 and #3446 (comment).

[alanakbik]

Closed by #3653