18
\$\begingroup\$

This arises from the SO Question subclassing pathlib.Path.

The pathlib sources show that Path, when invoked, selects one of its subclasses, WindowsPath or PosixPath, to invoke in place of itself, within it's __new__ method. The following diagram shows a rough idea, with black lines indicating inheritance, red the constructor and green the instance returned.

Inheritance and Instantiations in pathlib

Kevin points out that while this works for Path itself it prevents any subclass from having the same functionality. Now I'm not entirely clear as to why this is and Kevin does not offer an explanation. Presumably the __new__ method is called before the MRO is setup and one can not rely on super to do the right thing just yet, namely bumping up to the super classes __new__ to select the appropriate subclass.

The answers provided for the question itself include the following alternatives and I'm curious as to which is the best method/better practice.

  1. Since everyday should be Christmas let's Wrap the class (a.k.a. Encapsulation/Clojures). I find quite often this is actually the go to answer for a lot of inheritance questions which I feel kinda defeats the point of subclassing as one then has to manually expose the wrapped classes methods.

    from pathlib import Path as _Path_
    
    class Path():
     def __init__(*args, **kvps):
      super().__init__(*args, **kvps)
      self._path_ = _Path_(*args, **kvps)
    
  2. Instantiate the type of an instantiated class. This is a bit of trickery as you're invoking the class you plan to subclass. It selects the final class for you and you then determine the type it selected and subclass that.

    from pathlib import Path
    
    class Path(type(pathlib.Path())):
     pass
    
  3. Tack on the required features to the selected class. This is roughly what projetmbc did.

    from pathlib import Path as _Path_, PosixPath, WindowsPath
    
    class Path(_Path_):
     def __new__(cls, *args):
      if cls is Path:
        cls = WindowsPath if os.name == 'nt' else PosixPath
    setattr(cls, "extramethod", _extramethod)
    return cls._from_parts(args)
    

    The following is probably a simpler means of achieving the same goal.

    Path.extramethod = _extramethod
    # Or possibly 
    WindowsPath.extramethod = _extramethod
    PosixPath.extramethod = _extramethod
    

    In the same vane we could simply copy the __new__ method from Path the pathlib sources but that just screams bad practice.

  4. There is one solution missing from the suite of answers there that Kevin hints towards which is to specify the missing attribute reported in the original error message.

    from pathlib import Path as _Path_, _windows_flavour, _posix_flavour
    import os
    
    class Path(_Path_):
     _flavour = _windows_flavour if os.name == 'nt' else _posix_flavour
    
  5. BKSpurgeon prompted the following strategy. This sweeps the underlying API details away which is quite nice.

    from pathlib import Path as _Path_, PosixPath as _PosixPath_, WindowsPath  as _WindowsPath_
    import os
    
    class Path(_Path_) :
     def __new__(cls, *args, **kvps):
      print("Path*")
      return super().__new__(WindowsPath if os.name == 'nt' else PosixPath, *args, **kvps)
    
    class WindowsPath(_WindowsPath_, Path) :
     pass
    
    class PosixPath(_PosixPath_, Path) :
     pass
    

    The figure below illustrates how the interaction plays out in this case. Basically the constructor in Path* selects one of it's sub-classes, WindowsPath* or PosixPath*, whose MRO map back to Path* and Path and the respective intermediate subclass, WindowsPath or PosixPath. Modified inheritance and instantiation flow

I feel (1) is kind of cop out, (2) is a bit of a hack, (3) and (4) seem the better approaches but require peaking under the hood (5) is probably the best as it is quite clean.

If it's possible could one elaborate upon which method 1-5, or if applicable their sub-methods, should one really employ here. The real test is probably checking the MRO in each case and that the final instance includes all the subclasses methods and attributes. I'm quite convinced that pathlib should really be using Metaclasses under the hood and would be keen to hear any commentary upon this aswell.

\$\endgroup\$
8
  • 1
    \$\begingroup\$ The tags pathlib and metaclass do not exist if you're a moderator perhaps we could include them ? \$\endgroup\$
    – Carel
    May 3, 2017 at 19:11
  • 1
    \$\begingroup\$ in python are you not able to override the base class constructor? - if so then simply override it with your own implementation that would stop the base class / super class from instantiating WindowsPath or PosixPath. \$\endgroup\$
    – BenKoshy
    May 4, 2017 at 13:34
  • \$\begingroup\$ @BKSpurgeon The third strategy tackles that but it kind of defeats the point I should think. You want the subclass to invoke the base classes constructor. The base classes constructor is hot swapping the base class itself, Path, for either a preferred subclass, PosixPath or WindowsPath, based upon the operating system, the _flavour attribute. You'd like the subclass to do the same. Though your comment has given me a better option to consider. I'll post it now. \$\endgroup\$
    – Carel
    May 4, 2017 at 17:20
  • \$\begingroup\$ @BKSpurgeon I was going to post the fifth option as the solution but decided to lob it into the question as an alternative. Until some one posts a reason not to I'm going to go with the fifth variant for now. I think the real problem is the pathlib implementation employs a sneaky trick and this carries on to the user. \$\endgroup\$
    – Carel
    May 4, 2017 at 19:28
  • \$\begingroup\$ i'm glad you have some sort of solution worked out. I don't understand what is going on here. it looks like a very interesting problem though. if you have more details about the background: i.e. why you are creating another derived class that would be helpful. i'm hoping to come back to this - but that will be well after you've found a solution. \$\endgroup\$
    – BenKoshy
    May 4, 2017 at 23:47

1 Answer 1

2
\$\begingroup\$

@Danninno asked us to close off the question, to be honest I forgot about the request for a bit until recently (I've been using the wrapper quite a lot).

Answer

I see that within my own code I went with Option 4. Mostly for the reasons stated in the question. It turned out to be the better fit of the 5 possible options. Option 5 was/is a close contender as it's more OOP'y but it does generate a crazy class structure to simply hide an implementation detail.

Motivation/Discussion

When I originally asked the question I was hoping for bit of a fight/haggle/explanatory answer (Even if I received an opinionated one i.r.o. the forum rules) or atleast a good argument as to which method was the better/worse one. Mostly so that I could improve my decision making process. I see now that some classes in Python are done by it's developers to showcase certain features (After all they like to play/push the bounds somewhat). In this instance Path is constructed in such a way as to avoid the use of Metaclasses (I might be a bit off here, as it's been a while since I looked at the source for pathlib). Another example of this "playing" is seen in the NamedTuple implementation which essentially evals a string to implement it's class, which eliminates any implied inheritance.

\$\endgroup\$
1
  • \$\begingroup\$ Great discussion! For simplicity I'll stay with (1) encapsulation for now, since other approaches increase solution complexity quite a lot and It doesn't worth yet \$\endgroup\$
    – artu-hnrq
    Aug 28, 2022 at 4:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.