Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Checkpoints not saved #26

Closed
ylhsieh opened this issue Nov 16, 2018 · 6 comments
Closed

Checkpoints not saved #26

ylhsieh opened this issue Nov 16, 2018 · 6 comments

Comments

@ylhsieh
Copy link

ylhsieh commented Nov 16, 2018

There is an option save_checkpoints_steps that seems to control checkpointing. However, there is no actual saving operation in the run_* scripts. So, should we add that functionality or remove this argument?

@valsworthen
Copy link

valsworthen commented Nov 17, 2018

In the run_squad.pyscript, I added the following lines after the training loop:

logger.info(***** Saving fine-tuned model *****)
output_model_file = os.path.join(args.output_dir, "pytorch_model.bin")
if n_gpu > 1:
    torch.save(model.module.bert.state_dict(), output_model_file)
else:
    torch.save(model.bert.state_dict(), output_model_file)

The code runs and I was able to load the model to test on the Adversarial SQuAD datasets.

I do not use the other run_* scripts but this may be applicable as well.

Edit: the files have been modified in the latest commits so I think it's now necessary to check the loading of fine-tuned models in the script.

@thomwolf
Copy link
Member

You are right this argument was not used. I removed it, thanks. These examples are provided as starting point to write training scripts for the package module. I don't plan to update them any further (except fixing bugs).

@Jasperty
Copy link

Jasperty commented Mar 3, 2019

In the run_squad.pyscript, I added the following lines after the training loop:

logger.info(***** Saving fine-tuned model *****)
output_model_file = os.path.join(args.output_dir, "pytorch_model.bin")
if n_gpu > 1:
    torch.save(model.module.bert.state_dict(), output_model_file)
else:
    torch.save(model.bert.state_dict(), output_model_file)

The code runs and I was able to load the model to test on the Adversarial SQuAD datasets.

I do not use the other run_* scripts but this may be applicable as well.

Edit: the files have been modified in the latest commits so I think it's now necessary to check the loading of fine-tuned models in the script.

what is your result on adversarial-squad?

@valsworthen
Copy link

At that time I got:
AddSent
BERT base 58.7 EM / 66.2 F1
BERT large 65.5 EM / 71.9 F1

AddOneSent
BERT base 67.0 EM / 74.7 F1
BERT large 72.7 EM / 79.1 F1

@Jasperty
Copy link

Jasperty commented Mar 4, 2019

At that time I got:
AddSent
BERT base 58.7 EM / 66.2 F1
BERT large 65.5 EM / 71.9 F1

AddOneSent
BERT base 67.0 EM / 74.7 F1
BERT large 72.7 EM / 79.1 F1

Thanks a lot! Do you release your paper? i want to cite your result and paper in my paper.

@valsworthen
Copy link

Unfortunately it was not part of a paper, just preliminary results.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants