surrogateescape handler broken when encountering UnicodeEncodeError

[wence-]

Consider:

$ python --version
2.7.9

from future.utils.surrogateescape import register_surrogateescape
register_surrogateescape()
print u'\u03b1'.encode("iso-8859-1", "surrogateescape")
=> future/utils/surrogateescape.pyc in replace_surrogate_encode(mystring)
     84         if not 0xD800 <= code <= 0xDCFF:
     85             # Not a surrogate. Fail with the original exception.
---> 86             raise exc
     87         # mybytes = [0xe0 | (code >> 12),
     88         #            0x80 | ((code >> 6) & 0x3f),

NameError: global name 'exc' is not defined

This is because replace_surrogate_encode tries to refer to the exception of its calling context, which obviously fails. A simple fix might be to pass that exception in as an argument.

[pombredanne]

@edschofield do you want a patch for this? This is an easy fix I guess?

[edschofield]

@pombredanne: Yes please! That would be very helpful.

[anbai106]

Hi,
I have this issue recently on my institute cluster when using anaconda2, this is my python version

Python 2.7.13 :: Anaconda 4.4.0 (64-bit)

I know the problem comes from this function and I followed your instruction, make it pass, but it gave me new errors, but this works fine on my Mac, so I dont know what is the problem... is it because of the platform? the python script I have forced the encoding to be utf-8 at the header of the script.

Any suggestion would be appreciated

Hao

[pombredanne]

@anbai106 you can feed unicode paths alright all the way on Windows and macOS on Python 2.
On Linux, this is where surrogate escaping comes into play hence the fact you hit the bug there quite likely

[graingert]

@jmadler @wence- looks like this was fixed here:
da5e6d2