Checkpoints not saved

[ylhsieh]

There is an option save_checkpoints_steps that seems to control checkpointing. However, there is no actual saving operation in the run_* scripts. So, should we add that functionality or remove this argument?

[valsworthen]

In the run_squad.pyscript, I added the following lines after the training loop:

logger.info(***** Saving fine-tuned model *****)
output_model_file = os.path.join(args.output_dir, "pytorch_model.bin")
if n_gpu > 1:
    torch.save(model.module.bert.state_dict(), output_model_file)
else:
    torch.save(model.bert.state_dict(), output_model_file)

The code runs and I was able to load the model to test on the Adversarial SQuAD datasets.

I do not use the other run_* scripts but this may be applicable as well.

Edit: the files have been modified in the latest commits so I think it's now necessary to check the loading of fine-tuned models in the script.

[thomwolf]

You are right this argument was not used. I removed it, thanks. These examples are provided as starting point to write training scripts for the package module. I don't plan to update them any further (except fixing bugs).

[Jasperty]

In the run_squad.pyscript, I added the following lines after the training loop:

logger.info(***** Saving fine-tuned model *****)
output_model_file = os.path.join(args.output_dir, "pytorch_model.bin")
if n_gpu > 1:
    torch.save(model.module.bert.state_dict(), output_model_file)
else:
    torch.save(model.bert.state_dict(), output_model_file)

The code runs and I was able to load the model to test on the Adversarial SQuAD datasets.

I do not use the other run_* scripts but this may be applicable as well.

Edit: the files have been modified in the latest commits so I think it's now necessary to check the loading of fine-tuned models in the script.

what is your result on adversarial-squad?

[valsworthen]

At that time I got:
AddSent
BERT base 58.7 EM / 66.2 F1
BERT large 65.5 EM / 71.9 F1

AddOneSent
BERT base 67.0 EM / 74.7 F1
BERT large 72.7 EM / 79.1 F1

[Jasperty]

At that time I got:
AddSent
BERT base 58.7 EM / 66.2 F1
BERT large 65.5 EM / 71.9 F1

AddOneSent
BERT base 67.0 EM / 74.7 F1
BERT large 72.7 EM / 79.1 F1

Thanks a lot! Do you release your paper? i want to cite your result and paper in my paper.

[valsworthen]

Unfortunately it was not part of a paper, just preliminary results.