Skip to content
This repository has been archived by the owner on Dec 11, 2023. It is now read-only.

Translated strings still contain BPE delimiter and EOS token. #49

Open
voidmagic opened this issue Aug 2, 2017 · 0 comments
Open

Translated strings still contain BPE delimiter and EOS token. #49

voidmagic opened this issue Aug 2, 2017 · 0 comments

Comments

@voidmagic
Copy link
Contributor

I am using python 3.6.1 with tensorflow version of 1.2.1.
It seems like the function get_translation didn't work well.
The problem may caused by the reverse_target_vocab_table which convert target sample id into target words.
The result of look up operation on reverse_target_vocab_table is not pure str type. Instead it is something like byte string, which is

[b'<s>', b'</s>']

And this causes mismatch between '</s>' and b'</s>'.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant