Docker and ease of debugging

[aleSuglia]

Hello,

Thanks again for releasing this codebase. I just wanted to ask your opinion about Docker. Docker has definitely its benefits when running code on production but I believe it's not really well-suited for research prototyping. In particular, I believe that a research codebase should be really easy to use, debug and modify. Docker prevents you from freely debugging in your favourite IDE (PyCharm). I think it would be easy to do so if the codebase was easily installable using Anaconda just like other framework such as AllenNLP.

Looking at the codebase, seems that the major problem when it comes to dependencies, is Horovod. Is there any way you can somehow refactor your codebase following Horovod-GPU-project? I think this will allow everybody to easily install Horovod without the burden of having to learn Docker internals.

Another important point was the fact that currently seems that the codebase supports only GPU execution. However, most of the time, it would be useful to test/debug your model on CPU as well. In this case, the major issue that I see is that you assume that apex is installed. Huggingface decided to deprecate Apex because is now been outperformed by the internal Torch implementation (see huggingface/transformers#9377 for details).

I think these two points will incredibly improve the user experience of your codebase for people that: 1) don't have GPU; 2) don't have experience/the current environment to use Docker (some companies might not like it).

I would love to hear your thoughts on this. I'm planning to modify the codebase myself tackle such problems so please let me know if you would be interested in a PR.

Thanks,
Alessandro

[linjieli222]

Hi Alessandro,

Thank you for your valuable suggestions. We really appreciate your inputs and your detailed comments about our codebase.

Let me try to address each of your concerns below.

We believe Docker is a better option as the environment set up has been completed and it is easier for reproducing results in our paper. Unfortunately, we currently do not have the bandwidth to refactor our codebase to Horovod-GPU-project. I am not an expert with PyCharm, but I have found some tutorials to config PyCharm to work with Docker, e.g.: https://www.jetbrains.com/help/pycharm/using-docker-as-a-remote-interpreter.html#summary . However, your concern is also valid, for people who are not familiar with Docker or cannot use Docker, it could be a bad user experience.

About Apex, we do agree that internal Torch implementation may be a better option.

It would be great if you have the capacity to tackle the above problems, we absolutely welcome any PR that could potentially improve our codebase.

Thanks,
Linjie

[aleSuglia]

I'm working on a complete refactoring using PyTorch-Lightning. I'll keep you posted :)