Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Source/Target length distinction #1

Merged
merged 2 commits into from
Feb 23, 2017
Merged

Source/Target length distinction #1

merged 2 commits into from
Feb 23, 2017

Conversation

pltrdy
Copy link
Contributor

@pltrdy pltrdy commented Feb 23, 2017

Preprocess parameters

Removed parameter -seq_length
New parameters -src_seq_length and -tgt_seq_length


Training speed token/s

In both LUA/PyTorch OpenNMT, the training process prints a speed, in token/sec, but:

  • LUA OpenNMT is printing source token/sec
  • PyOpenNMT is printing target token/sec

This can lead to important differences, especially when src/tgt sequence length are different (e.g. summarization), and therefore lead to false conclusion about performances.

See also: pytoch/example/issue#75

Removed parameter -seq_length
New parameters -src_seq_length and -tgt_seq_length
@pltrdy pltrdy changed the title Splitting preprocess.py parameters -seq_length Source/Target length distinction Feb 23, 2017
@srush srush merged commit 7ed8bcd into OpenNMT:master Feb 23, 2017
bmccann added a commit that referenced this pull request Apr 19, 2017
srush pushed a commit that referenced this pull request Aug 22, 2017
srush pushed a commit that referenced this pull request Sep 8, 2017
srush pushed a commit that referenced this pull request Nov 5, 2017
Fix unintended float type for num_block in SRU.py
pltrdy referenced this pull request in pltrdy/OpenNMT-py May 14, 2018
Fix 'whitespace-only translation handling'
srush pushed a commit that referenced this pull request Jul 6, 2018
Gldkslfmsd pushed a commit to Gldkslfmsd/OpenNMT-py that referenced this pull request Sep 6, 2018
saikat107 pushed a commit to saikat107/OpenNMT-py that referenced this pull request Mar 12, 2019
Fix 'whitespace-only translation handling'
saikat107 pushed a commit to saikat107/OpenNMT-py that referenced this pull request Mar 12, 2019
bhacquin pushed a commit to bhacquin/OpenNMT-py that referenced this pull request Jun 11, 2019
bhacquin pushed a commit to bhacquin/OpenNMT-py that referenced this pull request Jun 14, 2019
ldeseynes pushed a commit to ldeseynes/OpenNMT-py that referenced this pull request Jun 4, 2020
minor changes to make code simpler/more explicit
@kevinGit777 kevinGit777 mentioned this pull request Jul 14, 2021
avaucher referenced this pull request in rxn4chemistry/OpenNMT-py Jun 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants