Fix #1982 #1983
Merged
Fix #1982 #1983
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
September 27, 2012 17:51
Since the codepoints resulting from a decode depend on the encoding, this test can only be an example of error handling, but the current code doesn't treat it as as such. so IMHO this is wrong, it breaks another fix which I believe does the right thing in this respect, for another issue. and so It is removed by this patch. Unicode<-> bytes conversion and coercing should be done at I/O entry points, not internal code.
internal representations should be unicode objects, not strings, conversion should be at I/O contact-points only. Note: this still does not address the problem of pprinting indices that have a tuple of unicode strings, as a value. This issue came up before, monkey-patched in f4e3ab6, but second time around - a more general solution is needed.
|
I'm definitely aware of the Unicode whack-a-mole. It's used up a huge amount of development time. Having to deal with different console encodings is also a big pain. I'd be willing to accept a big PR cleaning this up after 0.9 is out; I guess we have a test suite and CI to protect us on Python 2 and Python 3. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fixes #1982
Had to hemstring another test so the fix will not break the tests, see
first patch in series I think it's the right thing to do.
There was another fix for a similar unicode problem in 703bbba,
there seems to be some Unicode whack-a-mole going on...
I'm willing to try my hand at fixing the unicode issues more seriously,
but that probably involves many changes throughout the codebase,
and some pain until things settle down.
would you consider such a big PR?