Skip to content

Commit

Permalink
Fix encoding error in dataset builder testing checksum for python2.
Browse files Browse the repository at this point in the history
PiperOrigin-RevId: 245839197
  • Loading branch information
adarob authored and copybara-github committed Apr 29, 2019
1 parent c9e4a2f commit 0915af0
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions tensorflow_datasets/testing/dataset_builder_testing.py
Original file line number Diff line number Diff line change
Expand Up @@ -311,9 +311,9 @@ def _bytes_flatten(element):
ret += _bytes_flatten(v)
elif isinstance(element, str):
if hasattr(element, "decode"):
# Python2 considers bytes to be str, but are almost always utf-8
# Python2 considers bytes to be str, but are almost always latin-1
# encoded bytes here. Extra step needed to avoid DecodeError.
element = element.decode("utf-8")
element = element.decode("latin-1")
element = element.encode("utf-8")
ret += element
elif isinstance(element, np.ndarray):
Expand Down

0 comments on commit 0915af0

Please sign in to comment.