Direct sub-classing of pathlib.Path

@pfmoore

BPO	24132
Nosy	@pfmoore, @pitrou, @keithy, @qb-cea, @miss-islington, @FFY00, @barneygale, @Xtrem532, @nyuszika7h, @kfollstad
PRs	bpo-24132: refactor pathlib to allow easy subclassing #6248 bpo-40107: stop using os.open() to implement pathlib.Path.open() #25240 bpo-42998: add 'user' parameter to pathlib.Path.home() #25271 bpo-43012: remove `pathlib._Accessor` #25701 bpo-44136: remove `pathlib._Flavour` #26141 bpo-24132: Direct sub-classing of pathlib.Path #26438 bpo-44412: add `os.path.fileuri()` function. #26708 bpo-24132: Add direct subclassing of PurePath/Path in pathlib #26906 bpo-24132: Add `pathlib._AbstractPath` #31085 gh-68320, gh-88302 - Allow for `pathlib.Path` subclassing #31691

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2015-05-06.05:26:53.017>
labels = ['type-feature', 'library', '3.11']
title = 'Direct sub-classing of pathlib.Path'
updated_at = <Date 2022-03-09.23:54:09.299>
user = 'https://bugs.python.org/projetmbc'

bugs.python.org fields:

activity = <Date 2022-03-09.23:54:09.299>
actor = 'barneygale'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2015-05-06.05:26:53.017>
creator = 'projetmbc'
dependencies = []
files = []
hgrepos = []
issue_num = 24132
keywords = ['patch']
message_count = 28.0
messages = ['242643', '242651', '242656', '242657', '242689', '242696', '242699', '242700', '242701', '242702', '242703', '242705', '246007', '305799', '305811', '305827', '305830', '305918', '310634', '314561', '314582', '314625', '365277', '381321', '392695', '401946', '412354', '414821']
nosy_count = 15.0
nosy_names = ['paul.moore', 'pitrou', 'bronger', 'elguavas', 'piotr.dobrogost', 'Kevin.Norris', 'projetmbc', 'keithy', 'qb-cea', 'miss-islington', 'FFY00', 'barneygale', 'Xtrem532', 'nyuszika7h', 'kfollstad']
pr_nums = ['6248', '25240', '25271', '25701', '26141', '26438', '26708', '26906', '31085', '31691']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue24132'
versions = ['Python 3.11']

Hello.

I have noticed a problem with the following code.

from pathlib import Path

class PPath(Path):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

test = PPath("dir", "test.txt")

This gives the following error message.

 
Traceback (most recent call last):
  File "/Users/projetmbc/test.py", line 14, in <module>
    test = PPath("dir", "test.txt")
  File "/anaconda/lib/python3.4/pathlib.py", line 907, in __new__
    self = cls._from_parts(args, init=False)
  File "/anaconda/lib/python3.4/pathlib.py", line 589, in _from_parts
    drv, root, parts = self._parse_args(args)
  File "/anaconda/lib/python3.4/pathlib.py", line 582, in _parse_args
    return cls._flavour.parse_parts(parts)
AttributeError: type object 'PPath' has no attribute '_flavour'

This breaks the sub-classing from Python point of view.

There is an ugly hack to sub-class Path but it's a bit unpythonic.

One issue with your code - what would you expect str(test) to produce? "dir/test.txt" or "dir\test.txt"? That's the point of the "flavour" - is it a Windows path or a Unix path?

Agreed that an easier method of creating Path subclasses that handle this type of thing would be useful, but any solution needs to make sure that developers don't overlook the Windows vs Unix implications.

Can you give an actual use case (as opposed to the toy example)?

The Path classes were not designed to be subclassable by the user.
I'm not against making subclassing easier, but someone will have to propose a viable approach for that.

Hello.

I will give a real example in 5 hours after my job. I will try tomorrow a
solution to ease the subclassing using another dedicazted class PathPlus,
sorry for the name. The idea would be to use this new class for
customization, and also to define WindowsPath and PosixPath sub-classing
this new class. By default PathPlus would be an empty class. I do not know
if this works well. Maybe my idea is a bad one.

*Christophe BAL*
*Enseignant de mathématiques en Lycée **et développeur Python amateur*
*---*
*French math teacher in a "Lycée" **and **Python **amateur developer*

2015-05-06 13:05 GMT+02:00 Antoine Pitrou <report@bugs.python.org>:

Antoine Pitrou added the comment:

The Path classes were not designed to be subclassable by the user.
I'm not against making subclassing easier, but someone will have to
propose a viable approach for that.

----------
versions: +Python 3.5 -Python 3.4

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue24132\>

Here are for example two extra methods that I have implemented.

def __sub__(cls, path):
    """
This magic method allows to use ``onepath - anotherpath`` instead of the
long
version ``onepath.relative_to(anotherpath)`` given by ``pathlib.Path``.
    """
    return cls.relative_to(path)

def _ppath_common_with(cls, paths):
    """
This method returns the path of the smaller common "folder" of the current
path
and at least one paths.

python::
from mistool import os_use

    path   = os_use.PPath("/Users/projects/source/doc")
    path_1 = os_use.PPath("/Users/projects/README")
    path_2 = os_use.PPath("/Users/projects/source/misTool/os_use.py")

    print(path.common_with((path_1, path_2)))
    """
    if not isinstance(paths, (list, tuple)):
        paths = [paths]

    commonparts = list(cls.parts)

    for onepath in paths:
        i = 0

        for common, actual in zip(commonparts, onepath.parts):
            if common == actual:
                i += 1
            else:
                break

        commonparts = commonparts[:i]

        if not commonparts:
            break

    commonpath = pathlib.Path("")

    for part in commonparts:
        commonpath /= part

    return commonpath

*Christophe BAL*
*Enseignant de mathématiques en Lycée **et développeur Python amateur*
*---*
*French math teacher in a "Lycée" **and **Python **amateur developer*

2015-05-06 14:13 GMT+02:00 Christophe BAL <report@bugs.python.org>:

Christophe BAL added the comment:

Hello.

I will give a real example in 5 hours after my job. I will try tomorrow a
solution to ease the subclassing using another dedicazted class PathPlus,
sorry for the name. The idea would be to use this new class for
customization, and also to define WindowsPath and PosixPath sub-classing
this new class. By default PathPlus would be an empty class. I do not know
if this works well. Maybe my idea is a bad one.

*Christophe BAL*
*Enseignant de mathématiques en Lycée **et développeur Python amateur*
*---*
*French math teacher in a "Lycée" **and **Python **amateur developer*

2015-05-06 13:05 GMT+02:00 Antoine Pitrou <report@bugs.python.org>:

>
> Antoine Pitrou added the comment:
>
> The Path classes were not designed to be subclassable by the user.
> I'm not against making subclassing easier, but someone will have to
> propose a viable approach for that.
>
> ----------
> versions: +Python 3.5 -Python 3.4
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue24132\>
> _______________________________________
>

----------

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue24132\>

For that type of function, I'd suggest you use a standalone function rather than subclassing and methods or operator overloading. You don't gain enough to be worth the complexity of having to subclass path objects. And duck typing means that your function works for any subclass of (Pure)Path without change.

I don't agree with you. I prefer to add new functionalities to the paths I
use. This is the power of OOP. It is easier and cleaner to use
*mypath.common_with(otherpath)* than *common_with(*mypath, **other path)
.

Python is highly OOP, so you can't say *"don't use subclassing in your
case"*. As a user, I should have the possibility to use the method I want.

Another example is the use of *onepath - anotherpath* instead of
*onepath.relative_to(*another path) . That's the power of the magic
method to add this kind of feature.

*Christophe BAL*
*Enseignant de mathématiques en Lycée **et développeur Python amateur*
*---*
*French math teacher in a "Lycée" **and **Python **amateur developer*

2015-05-06 20:21 GMT+02:00 Paul Moore <report@bugs.python.org>:

Paul Moore added the comment:

For that type of function, I'd suggest you use a standalone function
rather than subclassing and methods or operator overloading. You don't gain
enough to be worth the complexity of having to subclass path objects. And
duck typing means that your function works for any subclass of (Pure)Path
without change.

----------

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue24132\>

I have no problem with that - it's a style choice certainly.

As I said, I'd like to see simpler subclassing of pathlib objects. I just think it'll be quite hard to do (given the complexities of classes for Windows/Unix as well as pure and concrete paths). So if it's just about examples like this, I personally would take the easier route and just go with standalone functions. If someone else felt strongly enough to design and implement a subclassing solution, that's fine though.

Are you the author of path lib ?

*Christophe BAL*
*Enseignant de mathématiques en Lycée **et développeur Python amateur*
*---*
*French math teacher in a "Lycée" **and **Python **amateur developer*

2015-05-06 21:01 GMT+02:00 Paul Moore <report@bugs.python.org>:

Paul Moore added the comment:

I have no problem with that - it's a style choice certainly.

As I said, I'd like to see simpler subclassing of pathlib objects. I just
think it'll be quite hard to do (given the complexities of classes for
Windows/Unix as well as pure and concrete paths). So if it's just about
examples like this, I personally would take the easier route and just go
with standalone functions. If someone else felt strongly enough to design
and implement a subclassing solution, that's fine though.

----------

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue24132\>

Are you the author of path lib ?

Nope, that's Antoine.

OK.
I will try to find a way to achieve an easier and cleaner way to sub class
pathlib.Path and co.

What is the good way to propose a patch ?

*Christophe BAL*
*Enseignant de mathématiques en Lycée **et développeur Python amateur*
*---*
*French math teacher in a "Lycée" **and **Python **amateur developer*

2015-05-06 21:09 GMT+02:00 Paul Moore <report@bugs.python.org>:

Paul Moore added the comment:

> Are you the author of path lib ?

Nope, that's Antoine.

----------

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue24132\>

What is the good way to propose a patch ?

If you have a patch, attach it here, and it will get reviewed.

If I were designing pathlib from scratch, I would not have a separate Path class. I would instead do something like this:

In pathlib.py:

    if os.name == 'nt':
        Path = WindowsPath
    else:
        Path = PosixPath

Alternatively, Path() could be a factory function that picks one of those classes at runtime.

Of course, that still leaves the issue of where to put the method implementations which currently live in Path. We could change the name of Path to _Path and use the code above to continue providing a Path alias, but that might be too confusing. Another possibility is to pull those methods out into top-level functions and then alias them into methods in WindowsPath and PosixPath (perhaps using a decorator-like-thing to pass the flavor, instead of attaching it to the class).

The main thing, though, is that Path should not depend on its subclasses. That really strikes me as poor design, since it produces issues like this one.

Progress report:

I've been working on tidying up the pathlib internals over the 3.9 and 3.10 releases. We're now in a position where:

pathlib._Flavour is entirely pure, and doesn't make any os calls
pathlib._Accessor handles all os access.

The internal abstractions are now much tighter, which allows us to begin refactoring them with confidence!

The next step is to remove accessors, in bpo-43012.

After that I'll finally be in a position to start working on this bug!

I agree this would be nice. For now, I'm doing this as a hack:

class Path(type(pathlib.Path())):
    ...

New changeset 08f8301 by Barney Gale in branch 'main':
bpo-43012: remove pathlib._Accessor (GH-25701)
08f8301

If/when python/issues-test-cpython#31691 lands, I think this bug can be resolved: the original repro case will no longer raise AttributeError, and subclasses will be able to customize behaviour without needing to define further "flavour" or "accessor" subclasses.

gh-68320, gh-88302 - Allow for private `pathlib.Path` subclassing (GH…

Merged main into regmachine branch (#50)

Sorry if I'm "late to the party," but ALL of the discussion from non-mannequins seems (to me) to miss the point entirely.

The pathlib module fails (grossly, IMO) to respect two key "Pythonic" principles:
#. If the implementation is hard to explain, it's a bad idea.

Navigation Menu

Repository navigation

Direct sub-classing of pathlib.Path #68320

13 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Direct sub-classing of pathlib.Path #68320

Description

Activity

projetmbc commented on May 6, 2015

pfmoore commented on May 6, 2015

pitrou commented on May 6, 2015

projetmbc commented on May 6, 2015

projetmbc commented on May 6, 2015

pfmoore commented on May 6, 2015

projetmbc commented on May 6, 2015

pfmoore commented on May 6, 2015

projetmbc commented on May 6, 2015

pfmoore commented on May 6, 2015

projetmbc commented on May 6, 2015

pfmoore commented on May 6, 2015

KevinNorris commented on Jul 1, 2015

13 remaining items

barneygale commented on May 2, 2021

nyuszika7h commented on Sep 16, 2021

miss-islington commented on Feb 2, 2022

barneygale commented on Mar 9, 2022

mzipay commented on Apr 6, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions