Skip to content

Runtime Error when running Demo #67

Open
@rumourscape

Description

I am trying to run Demo

 python -m stopes.pipelines.bitext.global_mining_pipeline src_lang=fuv tgt_lang=zul +preset=demo embed_text=laser3

But I am receiving the following error:

2024-03-11 11:48 INFO 1036353:stopes.moses - Preprocess fuv (en), 1 files
2024-03-11 11:48 INFO 1036353:stopes.moses - moses command: $/bin/bash -o pipefail -c 'zcat /speech/advait/align/stopes/demo/data/fuv.gz | perl /speech/advait/align/stopes/stopes/modules/preprocess/moses/scripts/tokenizer/remove-non-printing-char.perl | perl /speech/advait/align/stopes/stopes/modules/preprocess/moses/scripts/tokenizer/normalize-punctuation.perl -l en | perl /speech/advait/align/stopes/stopes/modules/preprocess/moses/scripts/tokenizer/deescape-special-chars.perl | perl /speech/advait/align/stopes/stopes/modules/preprocess/moses/scripts/tokenizer/lowercase.perl'
submitit ERROR (2024-03-11 11:48:15,483) - Submitted job triggered an exception
2024-03-11 11:48 ERROR 1036353:submitit - Submitted job triggered an exception
Traceback (most recent call last):
  File "/speech/advait/miniconda3/envs/stopes/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/speech/advait/miniconda3/envs/stopes/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/_submit.py", line 11, in <module>
    submitit_main()
  File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/submission.py", line 76, in submitit_main
    process_job(args.folder)
  File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/submission.py", line 69, in process_job
    raise error
  File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/submission.py", line 55, in process_job
    result = delayed.result()
  File "/speech/advait/.local/lib/python3.10/site-packages/submitit/core/utils.py", line 133, in result
    self._result = self.function(*self.args, **self.kwargs)
  File "/speech/advait/align/stopes/stopes/core/stopes_module.py", line 97, in __call__
    res = self.run(iteration_value=iteration_value, iteration_index=iteration_index)
  File "/speech/advait/align/stopes/stopes/modules/preprocess/preprocess_encode_module.py", line 91, in run
    return encode_module.run(  # type:ignore[no-any-return]
  File "/speech/advait/align/stopes/stopes/modules/preprocess/line_processor.py", line 139, in run
    proc.process_lines(lines_with_numbers)
  File "/speech/advait/align/stopes/stopes/modules/preprocess/encode_to_npy.py", line 63, in process_lines
    array = self.encode_to_np(lines_with_number)
  File "/speech/advait/align/stopes/stopes/modules/preprocess/laser_sentence_encoder.py", line 400, in encode_to_np
    return self.encoder.encode_sentences(tokenized_sents)
  File "/speech/advait/align/stopes/stopes/modules/preprocess/laser_sentence_encoder.py", line 157, in encode_sentences
    results.append(self._process_batch(batch))
  File "/speech/advait/align/stopes/stopes/modules/preprocess/laser_sentence_encoder.py", line 95, in _process_batch
    sentemb = self.encoder(tokens, lengths)["sentemb"]
  File "/speech/advait/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/speech/advait/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/speech/advait/align/stopes/stopes/modules/preprocess/laser_sentence_encoder.py", line 201, in forward
    encoder_out = super().forward(src_tokens, src_lengths)
  File "/speech/advait/.local/lib/python3.10/site-packages/fairseq/models/transformer/transformer_encoder.py", line 165, in forward
    return self.forward_scriptable(
  File "/speech/advait/.local/lib/python3.10/site-packages/fairseq/models/transformer/transformer_encoder.py", line 294, in forward_scriptable
    lr = layer(x, encoder_padding_mask=encoder_padding_mask_out)
  File "/speech/advait/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/speech/advait/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
    return forward_call(*args, **kwargs)
  File "/speech/advait/.local/lib/python3.10/site-packages/fairseq/modules/transformer_layer.py", line 319, in forward
    output = torch._transformer_encoder_layer_fwd(
RuntimeError: Mask Type should be defined

This is taken from the error logs.

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions