Skip to content
This repository was archived by the owner on Sep 24, 2025. It is now read-only.
This repository was archived by the owner on Sep 24, 2025. It is now read-only.

Pretrain experiment missing restrict_attention_bucket_size command line flag definition #184

@maxploter

Description

@maxploter

An AttributeError for the restrict_attention_bucket_size flag is raised when running the tapas_pretraining_experiment script. Explicitly providing the command line flag doesn't solve the the problem (Unknown command line flag 'restrict_attention_bucket_size').
Probably, the flag was not defined in the script file during changes related to MATE?

@eisenjulian I wonder if the command line flag was overlooked in the following commit?
733aca5

Command:

python tapas/experiments/tapas_pretraining_experiment.py \
  --eval_batch_size=32 \
  --train_batch_size=512 \
  --num_eval_steps=100 \
  --save_checkpoints_steps=5000 \
  --num_train_examples=512000000 \
  --max_seq_length=128 \
  --input_file_train="uncased_L-2_H-128_A-2/results_sample/train.tfrecord" \
  --input_file_eval="uncased_L-2_H-128_A-2/results_sample/test.tfrecord" \
  --init_checkpoint="uncased_L-2_H-128_A-2/bert_model.ckpt.data-00000-of-00001" \
  --bert_config_file="uncased_L-2_H-128_A-2/bert_config.json" \
  --model_dir="uncased_L-2_H-128_A-2" \
  --do_train

Stacktrace:

Traceback (most recent call last):
  File "tapas/experiments/tapas_pretraining_experiment.py", line 193, in <module>
    app.run(main)
  File "/gpfs/space/home/ploter/.conda/envs/tapas_clean_env/lib/python3.7/site-packages/absl/app.py", line 308, in run
    _run_main(main, args)
  File "/gpfs/space/home/ploter/.conda/envs/tapas_clean_env/lib/python3.7/site-packages/absl/app.py", line 254, in _run_main
    sys.exit(main(argv))
  File "tapas/experiments/tapas_pretraining_experiment.py", line 125, in main
    restrict_attention_bucket_size=FLAGS.restrict_attention_bucket_size,
  File "/gpfs/space/home/ploter/.conda/envs/tapas_clean_env/lib/python3.7/site-packages/absl/flags/_flagvalues.py", line 474, in __getattr__
    raise AttributeError(name)
AttributeError: restrict_attention_bucket_size

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions