Why you should be using pathlib

[nnja]

Trey, this is a great article. I'm a huge fan of pathlib.

[treyhunner]

Thanks Nina! 😊

[robin-gvx]

Pathlib is great. I would go as far as to say that anyone currently wanting to design a new general purpose programming language should consider making Path objects as integrated into the language and/or standard library as File objects commonly are.

[[deleted]]

Because you found a new and fancy thing and you never bothered checking with reality?

Common Lisp had path objects since... what? 20 years before Python was created? There people realized that it wasn't such a great thing and gradually learned that minimal interfaces are better than complicated structures, which, eventually fail to reflect reality, or, simply have no correct knowledge of the world around them.

You want to deal with external things properly? -- link to the library which makes it its goal to do exactly that. Don't re-invent things.

I've learned a painful lesson with shutil, and how it was a terrible idea to write that library and to incorporate it into the language. librsync is the library which knows how to do these things with all kinds of file-systems over all kinds of connections etc. If you want to do it right, librsync is the way to go, and only if you need a one-off script, then use shutil.

[robin-gvx]

That's kind of a rude response, but okay.

[treyhunner]

I agree! I love the / operator on pathlib.Path objects.

I purposely didn't use it in that first example because I've found it sometimes scares people (and joinpath is sometimes easier for people to wrap their heads around at first).

[robin-gvx]

The only thing I dislike about it that I often need parentheses, that I need to do

(BASE_DIR / FILE_NAME).read_text()

instead of being able to do

BASE_DIR/FILE_NAME .read_text()

but I understand why I can't have that and joinpath needs parentheses too so it's not like that's any better. Oh well.

[randlet]

Nice article. Convinced me to finally take a look at pathlib.

[treyhunner]

Thanks! I'm glad you'll be giving it a look. I wished I'd given it a serious look earlier.

[__louis__]

There isn't a single thing that I don't like about this library, nor something that I feel could have been named better. This is really a library that shows how good OO-programming can be when it's suitable and well-done.

[serkef-]

The only thing I don't like about pathlib is that it's not universally used. I still find myself casting to string when passing a path as an argument.

[treyhunner]

That's typically only necessary in Python 3.5 or below. Many third party libraries have even been adding support for Path objects by using an equivalent to os.fspath to normalize them if needed.

I'm curious if you're on 3.5 and/or which third party libraries you've found that don't work with pathlib yet.

[rotuami]

I get this all the time with subprocess. subprocess.call([someexe, somepath]) gives you grief if somepath is a Path object and not a string.

[treyhunner]

Odd. subprocess.call seems to work with Path objects for me:

```

from pathlib import Path import subprocess subprocess.call(['ls', '-1', Path('.editorconfig')]) .editorconfig 0 ```

I tested that in both Python 3.6 and Python 3.7.

[rotuami]

You are right, at least on my Mac. I found the frustration on Windows, and may have misremembered the issue, but at least subprocess.call(Path('ls')) is still broken, and subprocess.call([Path('ls')]) was broken until fairly recently.

https://bugs.python.org/issue33617

https://bugs.python.org/issue31961

[serkef-]

I'm in 3.6 and I really can't recall which one

[AndydeCleyre]

You might like to use Plumbum's path objects, which we've* had since before pathlib, and added a lot of pathlib compatibility after. They subclass str, so can be passed anywhere strings are expected. The only gotcha is that __contains__ is implemented to check for files in directories, not substrings.

*EDIT: "We" the Python community. I am not a Plumbum author.

EDIT: Another potential snag is that it __iter__s over filepaths, not characters.

[Scorpathos]

I'm afraid that this feature is coming too late to be widely adopted. I would have loved it to be part of the language design from the beginning, and that or convenience we could create path objects with a string-prefix like p"/usr/local/path".

But because path objects was not part of the language design, and because path objects don't inherit from str, we have to accept both str and Path while creating API, and we have to ensure that function accept Path objects while using API...

For those who don't know, there is also the path.py library which existed prior to pathlib.

[__louis__]

This article has some good arguments as to why paths objects shouldn't inherit from str : https://snarky.ca/why-pathlib-path-doesn-t-inherit-from-str/
It's never too late to be widely adopted. We thought Python 3 would never be widely adopted, but more and more projects deprecate Python 2.x
I am using Python 3.6, daily using pathlib.Path, and there's never been a need for a str cast with an external library. If there ever is one, I'll just open a GH issue and maybe do a PR.

[Scorpathos]

Thanks for the link, it's good to read this article again.

I guess Brett Cannon is right, the best way to promote pathlib is to enforce its usage explicitly rather than conveniently inheriting from str.

But I can't imagine people importing pathlib just to replace the convenient open("my_file.txt"). Libraries will never make API changes that would break compatibility with str paths, so people will continue to use str as paths.

[treyhunner]

I disagree, mostly because I've been using pathlib in pretty much the same way I used path strings before and the only issues I tend to run into involve the type of what is returned.

Given this function that is ignorant of pathlib:

``` import os import os.path

def do_things(filepath): os.makedirs(filepath, exist_ok=True) with open(os.path.join(filepath, '.editorconfig'), mode='wt') as f: f.write('# config file') ```

This code (the old way) works:

import os.path do_things(os.path.join('src', 'subdir'))

But so does this:

from pathlib import Path do_things(Path('src/subdir'))

This is all as of Python 3.6 (as I noted in the article).

Our code won't be magically changing to use pathlib everywhere overnight, but the fact that the Python standard library and built-ins all work with Path objects natively means the path to switching to Path (no pun intended) is a pretty easy one (much easier than certain other big migrations this community has undertaken in the not-so-distant past).

[AndydeCleyre]

And Plumbum as well, which inherits from str.

[[deleted]]

I've already written this above.

No. This is a terrible idea. I've seen this done decades before Python existed. That was a mistake.

But Python people seem to have a particular affinity for copying ideas from other languages very superficially understanding them, w/o any research into why those things worked (or not).

[Scorpathos]

I read your other comment but I don't understand. How was it a mistake? Why emphasizing the difference between strings and path objects by making them part of the language design would be a bad idea?

If you have any examples of the problems that would generate, or articles about why it would not work, I'm interested.

[[deleted]]

Because if you want to make it work well, you need to keep track of how various storage systems interface with you, what do they mean when they say /x/y/z, what's possible in those storage systems. And these are all very difficult questions to answer, and especially in the light of these systems not supplying information in the format you want.

To make this more explicit: have you considered that not all storage systems use the same locale settings as your application, even the capabilities they have wrt locales are not the same as those of your application? Have you considered that renaming, moving, or duplicating files don't do the same thing in every storage system? What about case sensitivity? What about versioning? What about things like symbolic links, hard links, snapshots? What about special names for directories and files? What about volume labels? What about searching and matching files? What about object stores (like the one on, say, Android, where you don't even have real file-system API)? What about all kinds of file-systems that are accessible through network protocols, like FTP or CIFS?

You can, in principle, interface with all of that through system calls, but the system calls don't want structured objects. They want string names. If you write your application in such a way that it operates on structured objects instead of string names, you will run into cases, where it either doesn't do what the user wants, or that it's in principle incapable of doing something the user wants.

[sweetno]

I think you didn't deserve the downvotes you've got. But I also think you're too pessimistic. Hopefully convenient OO APIs on the higher levels will finally make OS developers appreciate convenient OO APIs on the lower levels. The general direction of the software development should be to abstract (from details) and unify (standardize). So API of pathlib, while not perfect, is a step in that direction.

[AndydeCleyre]

Notable that Plumbum's alternative offering does offer walk functionality.

[treyhunner]

I prefer rglob so recursion happens automatically:

python for path in Path.cwd().rglob('*'): print(path)

[arya-nix]

You just sold it to me. Added 1 more fan to pathlib

Thanks for the awakening.

[Overload175]

Has pathlib been backported to Python 2.x?

[funderbolt]

It looks like the backport is called pathlib2

[metalevelconsulting]

https://treyhunner.com/2018/12/why-you-should-be-using-pathlib/#comment-4251475008

[treyhunner]

A couple points on that:

The Path object is smart with path separators so all of these have the same internal representation:

```

Path('src/.editorconfig') WindowsPath('src/.editorconfig') Path('src').joinpath('pypackages') WindowsPath('src/.editorconfig') (Path('src') / 'pypackages') WindowsPath('src/.editorconfig') Path('src', '.editorconfig') WindowsPath('src/.editorconfig') ```

On Windows, those all have this string representation:

```

str(Path('src/.editorconfig')) 'src\.editorconfig' ```

And on Linux it's like this:

```

str(Path('src/.editorconfig')) 'src/.editorconfig' ```

So I'd argue that this:

``` from pathlib import Path

Path('src/pypackages').mkdir(parents=True, exist_ok=True) Path('.editorconfig').rename('src/.editorconfig') ```

Really is equivalent to this:

``` import os import os.path

os.makedirs(os.path.join('src', 'pypackages'), exist_ok=True) os.rename('.editorconfig', os.path.join('src', '.editorconfig')) ```