Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multilevel text field #1216

Merged
merged 18 commits into from
Jan 25, 2019
Merged

Multilevel text field #1216

Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
8af58cb
Move batching and field logic from inputter to dsets.
flauted Jan 22, 2019
f395638
Remove _feature_tokenize from inputter.
flauted Jan 22, 2019
2bd30a2
Merge branch 'master' into fields
flauted Jan 22, 2019
3177176
Remove src_lengths from audio.
flauted Jan 22, 2019
71c1271
Make lengths a torch.int instead of self.dtype in AudioSeqField.
flauted Jan 22, 2019
3957afb
Don't output src_lengths in AudioDataset.make_examples since they're …
flauted Jan 22, 2019
2dbb48a
Add temp fix for checking if data is text in make_features.
flauted Jan 22, 2019
31cae5a
First pass at a multi-level field design.
flauted Jan 23, 2019
ea19d87
Remove some unused code.
flauted Jan 23, 2019
4551293
Merge branch 'master' into multilevel-text-field
flauted Jan 23, 2019
47fa339
Remove batch.src_is_text attr.
flauted Jan 23, 2019
184c426
Clean up by adding __iter__ to MultiField.
flauted Jan 24, 2019
1d95347
Fix extract embeddings.
flauted Jan 24, 2019
d952335
Clean build_vocab (including bad indentation level), incorporate #119…
flauted Jan 24, 2019
d13c6b1
Remove make_features.
flauted Jan 24, 2019
d76ec5b
Merge branch 'master' into multilevel-text-field
flauted Jan 24, 2019
dfc6036
Update semantics in direct calls to batch.src, batch.tgt.
flauted Jan 25, 2019
31246c9
Test for old-style text fields while checking for old-style vocab.
flauted Jan 25, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Don't output src_lengths in AudioDataset.make_examples since they're … 
…no longer necessary
  • Loading branch information
@flauted
flauted committed Jan 22, 2019
commit 3957afb611898f0e998f683aa25f9903ffe5e827
3 changes: 1 addition & 2 deletions onmt/inputters/audio_dataset.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,8 +116,7 @@ def make_examples(
window_stride, window, normalize_audio
)

yield {side: spect, side + '_path': line.strip(),
side + '_lengths': spect.size(1), 'indices': i}
yield {side: spect, side + '_path': line.strip(), 'indices': i}


class AudioSeqField(Field):
Expand Down