Add (M)Luke model training for Token Classification in the examples

[jplu]

What does this PR do?

This PR adds the possibility to train the (M)Luke model for a Token Classification task with the accelerate package. It also adds a tiny update to give the possibility to train over multiple datasets configuration, for example being able to concatenate multiple languages of the XTREME PAN-X dataset and train the model over it.

One can easily test with the command:

accelerate launch run_ner_no_trainer.py --model_name_or_path studio-ousia/mluke-base --dataset_name xtreme --dataset_config_name PAN-X.fr,PAN-X.en --output_dir /data/mluke/ --task_name ner --return_entity_level_metrics

/cc @sgugger @LysandreJik

[sgugger]

Hi there, thanks a lot for you PR! It will be great to be able to fully use LUKE and mLUKE for token classification!

Now the issue is that we try to keep each example pretty simple so that users can easily tweak and customize them. Adding this in the run_ner_no_trainer example adds a lot of complexity so how about we make this into its own example instead? It could go into a research project of its own.
Same for the data collator. It's very specific to LUKE so it should be added in a module file in the same folder as the example instead of adding this to the main lib directly, I think.

[jplu]

Sure! I will rethink how to properly reorder the things and will let you know once pushed!

[NielsRogge]

Awesome, thanks a lot for adding this! Also cc'ing the original authors, @ikuyamada @Ryou0634.

I completely agree with Sylvain here, adding it to the existing run_ner.py script would make it confusing for people that want to leverage BERT/RoBERTa/etc.

[jplu]

@sgugger I moved everything into a dedicated folder that is only focus on Luke.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[NielsRogge]

Hi, let's finish this PR and merge it!

@jplu are you able to rebase with master?

[LysandreJik]

Hey @jplu, thanks for your PR!

I'd just move it to examples/research_projects to have a specific LUKE example, vs having a luke folder in the examples folder :)

[jplu]

Sorry for the late reply. I will update accordingly to what you asked ASAP.

[jplu]

Ok, done on my side!

[NielsRogge]

Suggested change

	# You should update this to your particular problem to have better documentation of `model_type`
	MODEL_CONFIG_CLASSES = list(MODEL_MAPPING.keys())
	MODEL_TYPES = tuple(conf.model_type for conf in MODEL_CONFIG_CLASSES)
	# You should update this to your particular problem to have better documentation of `model_type`
	MODEL_CONFIG_CLASSES = list(MODEL_MAPPING.keys())
	MODEL_TYPES = tuple(conf.model_type for conf in MODEL_CONFIG_CLASSES)

To be removed (see comments below).

[NielsRogge]

Can you also add a README that briefly explains what the script is about and how you can run a basic version of it?

[sgugger]

Thanks a lot for reworking your PR! I've left a couple of suggestions :-)

[jplu]

Done! Let me know if something is missing.

[sgugger]

Thanks a lot for amending the PR in this direction! :-)