Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[examples] unit test for run_bart_sum #3544

Merged
merged 25 commits into from
Apr 15, 2020

Conversation

sshleifer
Copy link
Contributor

@sshleifer sshleifer commented Mar 31, 2020

  • add lightning to examples/requirements.txt
  • first lightning unittest!

@@ -102,8 +102,8 @@ def optimizer_step(self, epoch, batch_idx, optimizer, optimizer_idx, second_orde
self.lr_scheduler.step()

def get_tqdm_dict(self):
tqdm_dict = {"loss": "{:.3f}".format(self.trainer.avg_loss), "lr": self.lr_scheduler.get_last_lr()[-1]}

avg_loss = getattr(self.trainer, "avg_loss", 0.0)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.trainer.avg_loss was raising AttributeError

@sshleifer sshleifer marked this pull request as ready for review March 31, 2020 05:28
@codecov-io
Copy link

codecov-io commented Mar 31, 2020

Codecov Report

Merging #3544 into master will increase coverage by 0.97%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #3544      +/-   ##
==========================================
+ Coverage   76.84%   77.81%   +0.97%     
==========================================
  Files         100      100              
  Lines       17064    17064              
==========================================
+ Hits        13112    13279     +167     
+ Misses       3952     3785     -167     
Impacted Files Coverage Δ
src/transformers/modeling_tf_utils.py 88.32% <0.00%> (+0.17%) ⬆️
src/transformers/modeling_openai.py 81.54% <0.00%> (+1.34%) ⬆️
src/transformers/modeling_xlnet.py 75.77% <0.00%> (+2.29%) ⬆️
src/transformers/modeling_ctrl.py 98.24% <0.00%> (+2.63%) ⬆️
src/transformers/modeling_roberta.py 95.71% <0.00%> (+10.00%) ⬆️
src/transformers/modeling_tf_pytorch_utils.py 89.93% <0.00%> (+81.20%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e5c393d...af9dd75. Read the comment docs.

@sshleifer sshleifer changed the title [WIP] unittest for run_bart_sum Unittest for run_bart_sum Apr 3, 2020
@sshleifer sshleifer requested review from srush and thomwolf April 3, 2020 05:32
from utils import SummarizationDataset


try:
Copy link
Contributor Author

@sshleifer sshleifer Apr 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to support being called from from pwd or examples/

@sshleifer sshleifer changed the title Unittest for run_bart_sum [examples] unit test for run_bart_sum Apr 3, 2020
@@ -0,0 +1,33 @@
# Script for verifying that run_bart_sum can be invoked from its directory
Copy link
Contributor Author

@sshleifer sshleifer Apr 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Runs same logic as run_train.sh on a tiny dataset

examples/requirements.txt Outdated Show resolved Hide resolved
@julien-c
Copy link
Member

julien-c commented Apr 7, 2020

LGTM from a superficial glance

try:
from .utils import SummarizationDataset
except ImportError:
from utils import SummarizationDataset
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I asked @aaugustin if there's an easy way to not resort to this kind of hack

@sshleifer
Copy link
Contributor Author

Planning on merging this April 15 at 7pm EST barring objections.

@sshleifer sshleifer requested a review from LysandreJik April 15, 2020 15:36
@sshleifer sshleifer merged commit c59b1e6 into huggingface:master Apr 15, 2020
@sshleifer sshleifer deleted the testability-run-bart-sum branch April 15, 2020 22:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants