Add a six.moves for collections and collections.abc in preparation fo… #241

WildCard65 · 2018-05-11T00:45:28Z

…r Python 3.7+ (Closes #155)

webknjaz · 2018-05-28T18:24:11Z

@benjaminp could you please take a look at this PR?

rth · 2018-06-09T11:37:48Z

This would be very helpful to use six for improving compatibility with the upcoming Python 3.7

serhiy-storchaka · 2018-06-12T16:58:56Z

six.py

@@ -300,6 +300,7 @@ class _MovedItems(_LazyModule):
    MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"),
    MovedModule("xmlrpc_client", "xmlrpclib", "xmlrpc.client"),
    MovedModule("xmlrpc_server", "SimpleXMLRPCServer", "xmlrpc.server"),
+    MovedModule("collections_abc", "collections", "collections.abc", sys.version_info[0:2] >= (3, 3)),


Why not just sys.version_info >= (3, 3) instead of sys.version_info[0:2] >= (3, 3)?

Consistency with the PYX variables at file's beginning

jnothman · 2018-06-14T03:30:08Z

six.py

@@ -102,9 +102,9 @@ def __get__(self, obj, tp):

 class MovedModule(_LazyDescr):

-    def __init__(self, name, old, new=None):
+    def __init__(self, name, old, new=None, use_new=True):


How does this help?

Sorry, I see now. Perhaps you should pass use_new in by name to make the code below clear.

Would it be better to parametrise by "version_changed" and do the comparison here?
Or leave it all on the call side, but use

MovedModule("collections_abc", "collections", "collections.abc" if sys.version_info[0:2] >= (3, 3) else "collections")

Let's wait to see what @benjaminp says before we change anything, I only use that parameter because any Python 3 version below 3.3 doesn't include the "abc" submodule.

Alternatively to a boolean use_new one could add an argument that specifies from which version on the new behavior should be used, e.g. since='3.3' or new_since='3.3'.

jaraco · 2018-07-03T19:37:38Z

Since six deals with the Python 2/3 boundary, this request seems misplaced, unless the project seeks to expand its scope for general cross-version compatibility.

WildCard65 · 2018-07-03T19:47:31Z

Python 3.8 is slated to remove support for ABCs in the collections module.

…

On Tue, Jul 3, 2018, 3:37 PM Jason R. Coombs, ***@***.***> wrote: Since six deals with the Python 2/3 boundary, this request seems misplaced, unless the project seeks to expand its scope for general cross-version compatibility. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#241 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AExYE9xgP2PYQBSdKkAZnc6ujjiGm5_Wks5uC8gDgaJpZM4T6v3a> .

jnothman · 2018-07-03T22:25:19Z

@jaraco: if not making this change forces libraries to either use hacks or drop Python 2 support (when otherwise they support Py>=3.4), then surely it's within six's scope.

…1431) Closes #11121 This PR removes the deprecation warning about ABC being moved from `collections` to `collections.abc` when importing scikit-learn in Python 3.7. In the end, I put `collections.abc.{Sequence, Iterable, Mapping, Sized}` in the namespace of `sklearn.utils.fixes`. This was the simplest way I could find, and while it has the drawback of obfuscating the real module name, other approached appeared more problematic and a similar approach is currently used e.g. for `utils.fixes.signature` which is an alias for `inspect.signature`. We can't just patch six with benjaminp/six#241, because sklearn uses six from 5 years ago, which would need updating and I'm not sure if it could have side effects (e.g. for pickling backward compatibility etc). **Edit**: This adds a test checking that generally no warnings are raised when importing scikit-learn top-level modules.

…ikit-learn#11431) Closes scikit-learn#11121 This PR removes the deprecation warning about ABC being moved from `collections` to `collections.abc` when importing scikit-learn in Python 3.7. In the end, I put `collections.abc.{Sequence, Iterable, Mapping, Sized}` in the namespace of `sklearn.utils.fixes`. This was the simplest way I could find, and while it has the drawback of obfuscating the real module name, other approached appeared more problematic and a similar approach is currently used e.g. for `utils.fixes.signature` which is an alias for `inspect.signature`. We can't just patch six with benjaminp/six#241, because sklearn uses six from 5 years ago, which would need updating and I'm not sure if it could have side effects (e.g. for pickling backward compatibility etc). **Edit**: This adds a test checking that generally no warnings are raised when importing scikit-learn top-level modules.

…r Python 3.7+ (Closes #155)

WildCard65 · 2018-11-14T14:38:24Z

@timhoffm I decided to use your suggestion of adding a "since" version variable.

timhoffm · 2018-11-14T20:17:49Z

six.py

        super(MovedModule, self).__init__(name)
-        if PY3:
+        if sys.version_info[0:len(since)] >= since:


The 0 can be left out.

As well as slicing at all.

You can suggest changes via GH now :)

Suggested change

if sys.version_info[0:len(since)] >= since:

if sys.version_info >= since:

timhoffm · 2018-11-14T20:18:38Z

six.py

@@ -300,6 +300,7 @@ class _MovedItems(_LazyModule):
    MovedModule("urllib_robotparser", "robotparser", "urllib.robotparser"),
    MovedModule("xmlrpc_client", "xmlrpclib", "xmlrpc.client"),
    MovedModule("xmlrpc_server", "SimpleXMLRPCServer", "xmlrpc.server"),
+    MovedModule("collections_abc", "collections", "collections.abc", (3, 3)),


I would use since as kwarg here to make it more readable.

Suggested change

MovedModule("collections_abc", "collections", "collections.abc", (3, 3)),

MovedModule("collections_abc", "collections", "collections.abc", since=(3, 3)),

Made MovedAttribute class have similar behaviour as MovedModule (future proofing)

WildCard65 · 2018-11-14T20:59:22Z

Suggested changes have been made, I also decided to future proof MovedAttribute incase it ever becomes required.

taleinat · 2018-12-16T09:33:07Z

This looks ready to go, and I would appreciate avoiding the deprecation warning with Python 3.7.

tirkarthi · 2018-12-23T06:20:08Z

Related CPython PR : python/cpython#10596

NeilGirdhar · 2019-03-04T19:59:23Z

Can we push this through? This would help with other projects.

stmcginnis · 2019-04-03T14:19:48Z

Any way to help move this along?

benjaminp · 2019-04-25T05:16:46Z

My initial reaction is similar to jaraco's. The planned release date for Python 3.8 is two months before Python 2's EoL.

Note the CPython PR to actually remove collections.abc, python/cpython#10596, is stuck due to the small part of the external Python universe that is vendorized into CPython.

NeilGirdhar · 2019-04-25T13:04:12Z

@benjaminp Is there a compatibility library for changes between Python versions 2 and 3.8? There needs to be one for the same reason that six was needed. Why not have six do this?

(Edited: I meant from Python 2 to Python 3.8.)

serhiy-storchaka · 2019-04-25T13:48:19Z

For compatibility between Python 3 versions you have to just import from collections.abc. six.moves is not needed for this.

NeilGirdhar · 2019-04-25T14:05:33Z

@serhiy-storchaka You're right. Unfortunately, many projects need to support Python 2, which has no collections.abc. Therefore, a compatibility library is needed for this. See the linked issues in this thread.

jaraco · 2019-04-25T15:27:40Z

Is there a compatibility library for changes between Python versions 3.x and 3.y?

There isn't, probably because many of the compatibility issues are selective (only between particular versions of Python), such as packages in the backports namespace or projects like configparser or importlib_resources that provide backports but following another pattern. It doesn't necessarily make sense to have one library to collect all compatibility modules and in fact such an approach has the unfortunate downside of having to drag along lots of historically compatibility that a given project may not need (and to eventually deprecate that in the project as versions become unsupported). Instead, I think it's better for each compatibility layer to have its own library. It might make sense to have a suite of compatibility libraries that cover specific compatibility concerns, such as this one which is a moved import between Python 3.3 and 3.4, might go in a "python_33_compat" package (or similar), and relied upon by projects requiring Python 3.3 or older.

jaraco · 2019-04-25T18:53:39Z

Back in the Python 2 days, I used to maintain a project called jaraco.compat that would provide forward-compatibility (backports) of various language features. Perhaps that project could be revived to support small compatibilities like this one.

jaraco · 2019-04-25T21:46:54Z

I'm rolling out jaraco.compat 2.2 with py33compat.collections.abc.

ndevenish · 2019-05-27T12:22:28Z

Is there anything blocking this? Was surprised to find that there was no option in six to avoid:

if six.PY3:
  from collections.abc import Iterable, Sequence
else:
  from collections import Iterable, Sequence

jaraco · 2019-05-27T17:16:27Z

If you want a one-liner, use from py33compat.collections.abc import Iterable, Sequence:

~ $ python2.7 -m pip-run -q jaraco.compat>=3.1 -- -c "from py33compat.collections.abc import Iterable, Sequence" && echo done                                                  
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
done
~ $ python3.7 -m pip-run -q jaraco.compat>=3.1 -- -c "from py33compat.collections.abc import Iterable, Sequence" && echo done                                                  
done

NeilGirdhar · 2019-05-27T17:32:59Z

@jaraco The problem is that a lot of projects don't want to add another dependency. I don't see why this isn't in six.

webknjaz mentioned this pull request May 13, 2018

Add is_item_sequence util function ansible/ansible#39924

Merged

jnothman mentioned this pull request May 23, 2018

DeprecationWarning: Using or importing the ABCs from 'collections' in Python 3.7 scikit-learn/scikit-learn#11121

Closed

serhiy-storchaka reviewed Jun 12, 2018

View reviewed changes

jnothman reviewed Jun 14, 2018

View reviewed changes

jnothman mentioned this pull request Jul 1, 2018

pip install fails (compile error) with Python 3.7rc1 scikit-learn/scikit-learn#11320

Closed

rth mentioned this pull request Jul 4, 2018

[MRG] Fix DeprecationWarning due to collections.abc in Python 3.7 scikit-learn/scikit-learn#11431

Merged

This was referenced Aug 1, 2018

Use collections.abc to avoid DeprecationWarning in Python 3.7 cupy/cupy#1515

Merged

Use collections.abc to avoid DeprecationWarning in Python 3.7 chainer/chainer#5172

Merged

khpeek mentioned this pull request Oct 1, 2018

Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated networkx/networkx#3170

Closed

Add a six.moves for collections and collections.abc in preparation fo…

cc7b892

…r Python 3.7+ (Closes #155)

WildCard65 mentioned this pull request Nov 14, 2018

Add environ and environb #255

Open

timhoffm reviewed Nov 14, 2018

View reviewed changes

Implemented suggested changes.

51e2569

Made MovedAttribute class have similar behaviour as MovedModule (future proofing)

timhoffm approved these changes Nov 14, 2018

View reviewed changes

webknjaz approved these changes Nov 14, 2018

View reviewed changes

tirkarthi mentioned this pull request Dec 23, 2018

Python 3.7 Collections Deprecation Message grappa-py/grappa#47

Closed

NeilGirdhar mentioned this pull request Feb 27, 2019

Fixed deprecated access of collections members. tensorflow/tensorflow#26173

Merged

dhirschfeld mentioned this pull request Apr 3, 2019

Silence DeprecationWarning numba/numba#3935

Merged

hazmat345 mentioned this pull request May 14, 2019

Deprecation warning loganasherjones/yapconf#96

Closed

plamut mentioned this pull request May 30, 2019

Expose Publish Retry Settings anguillanneuf/google-cloud-python#1

Open

NeilGirdhar mentioned this pull request May 31, 2019

Add six.moves.collections.abc #155

Closed

jdevera mentioned this pull request Jun 1, 2019

importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated corydolphin/flask-cors#239

Closed

KelSolaar mentioned this pull request Sep 15, 2019

PR: Fix "from collections import Mapping, MutableMapping" with Python 3.8. colour-science/colour#489

Closed

bhrutledge mentioned this pull request Oct 31, 2019

collections.abc isn't handled by install_aliases PythonCharmers/python-future#525

Open

benjaminp closed this in 0dc54ea Nov 5, 2019

jessemapel mentioned this pull request Nov 22, 2019

Future: Python 3.8 compatibility AndrewAnnex/SpiceyPy#325

Closed

jveitchmichaelis mentioned this pull request Dec 4, 2019

Use collections.abc for some imports if running under Python 3 pyudev/pyudev#386

Merged

tkrassowski mentioned this pull request Dec 16, 2019

fix(bigquery): import Mapping from collections.abc not from collections googleapis/google-cloud-python#9826

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a six.moves for collections and collections.abc in preparation fo… #241

Add a six.moves for collections and collections.abc in preparation fo… #241

WildCard65 commented May 11, 2018

webknjaz commented May 28, 2018

rth commented Jun 9, 2018 •

edited

Loading

serhiy-storchaka Jun 12, 2018

WildCard65 Jun 14, 2018

jnothman Jun 14, 2018

jnothman Jun 14, 2018

jnothman Jun 14, 2018

WildCard65 Jun 14, 2018

timhoffm Nov 1, 2018 •

edited

Loading

jaraco commented Jul 3, 2018

WildCard65 commented Jul 3, 2018 via email

jnothman commented Jul 3, 2018 •

edited

Loading

WildCard65 commented Nov 14, 2018

timhoffm Nov 14, 2018

serhiy-storchaka Nov 14, 2018

webknjaz Nov 14, 2018

timhoffm Nov 14, 2018

webknjaz Nov 14, 2018

WildCard65 commented Nov 14, 2018

taleinat commented Dec 16, 2018

tirkarthi commented Dec 23, 2018

NeilGirdhar commented Mar 4, 2019

stmcginnis commented Apr 3, 2019

benjaminp commented Apr 25, 2019

NeilGirdhar commented Apr 25, 2019 •

edited

Loading

serhiy-storchaka commented Apr 25, 2019

NeilGirdhar commented Apr 25, 2019 •

edited

Loading

jaraco commented Apr 25, 2019

jaraco commented Apr 25, 2019

jaraco commented Apr 25, 2019

ndevenish commented May 27, 2019

jaraco commented May 27, 2019

NeilGirdhar commented May 27, 2019

	if sys.version_info[0:len(since)] >= since:
	if sys.version_info >= since:

	MovedModule("collections_abc", "collections", "collections.abc", (3, 3)),
	MovedModule("collections_abc", "collections", "collections.abc", since=(3, 3)),

Add a six.moves for collections and collections.abc in preparation fo… #241

Add a six.moves for collections and collections.abc in preparation fo… #241

Conversation

WildCard65 commented May 11, 2018

webknjaz commented May 28, 2018

rth commented Jun 9, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

timhoffm Nov 1, 2018 • edited Loading

Choose a reason for hiding this comment

jaraco commented Jul 3, 2018

WildCard65 commented Jul 3, 2018 via email

jnothman commented Jul 3, 2018 • edited Loading

WildCard65 commented Nov 14, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WildCard65 commented Nov 14, 2018

taleinat commented Dec 16, 2018

tirkarthi commented Dec 23, 2018

NeilGirdhar commented Mar 4, 2019

stmcginnis commented Apr 3, 2019

benjaminp commented Apr 25, 2019

NeilGirdhar commented Apr 25, 2019 • edited Loading

serhiy-storchaka commented Apr 25, 2019

NeilGirdhar commented Apr 25, 2019 • edited Loading

jaraco commented Apr 25, 2019

jaraco commented Apr 25, 2019

jaraco commented Apr 25, 2019

ndevenish commented May 27, 2019

jaraco commented May 27, 2019

NeilGirdhar commented May 27, 2019

rth commented Jun 9, 2018 •

edited

Loading

timhoffm Nov 1, 2018 •

edited

Loading

jnothman commented Jul 3, 2018 •

edited

Loading

NeilGirdhar commented Apr 25, 2019 •

edited

Loading

NeilGirdhar commented Apr 25, 2019 •

edited

Loading