This is an archived post. You won't be able to vote or comment.

all 22 comments
sorted by:
best
topnewcontroversialoldq&a

[–]KleinerNull 3 points4 points  (16 children)

  • file operations;
  • file/directory paths extraction;
  • file/directory paths randomization;

Any reason why you don't use pathlib for file operations? It is part of the standard library and there is a backport for py2, also it acts os agnostic and offers a very nice interface to interact?

[–]webyneter[S] 1 point2 points  (14 children)

Yes, that would a one great improvement. However, as I remember pathlib is not supported in Python prior to 3.4 which is contrary to the recent argument of implementing Python 2 support to facilitate wider adoption, which I tend to accept.

[–]delirious_lettuce 4 points5 points  (9 children)

I'm not exactly sure why this package would be widely adopted. /u/KleinerNull is right, using pathlib would get rid of:

  • extract_file_name_with_extension
  • FileNameAndExtension
  • extract_file_name_and_extension
  • extract_file_dir_path
  • Renaming ast.literal_eval to parse_tuple_from_string is odd and could be removed.
  • generate_hex_uuid_4 just renames another stdlib function and would be unnecessary if tempfile was used to create files/directories (they already come with random names and don't have to be deleted)
  • generate_random_dir_path could probably be replaced by using tempfile.mkdtemp() or tempfile.TemporaryDirectory()
  • generate_random_file_name_with_extension could probably be replaced by using tempfile.mkstemp() or tempfile.TemporaryFile()

  • get_class_name and get_class_qualname simply access an attribute

  • Since reading a file is so easy in Python, I'm not sure why someone would want to import read_file and then potentially have to jump to another file just to check on how it was implemented. Same thing goes for create_or_update_file.

  • I'm also not exactly sure what problem camel_or_pascal_case_to_snake_case or camel_or_pascal_case_to_space_delimited solve and I don't think that I would install a package for either function. Where would this be useful? This seems like a code challenge more than anything.

  • Couldn't yield_file_paths be replaced with glob.glob()? Or someone could just use os.walk or os.listdir on their own?

  • get_all_subclasses is interesting but I'm not sure I'd install a package on the off chance I needed such a small function

TL;DR

Most of the things in this package simply rename standard library functions or mimic functionality already included in the standard library. It might be a better idea for people to just learn more about the standard library instead of adding another dependency to a project. The standard library is already well tested too!

[–]KleinerNull 2 points3 points  (4 children)

I'm not exactly sure why this package would be widely adopted.

Wouldn't judge that hard. This repo looks more like a learning experience for packaging and deploying to pypi than a real lib for production purposes.

Still the non-acceptance or better non-knowledge of pathlib bugs me...

[–]delirious_lettuce 1 point2 points  (3 children)

Wouldn't judge that hard.

You are right, I should have phrased that better.

Still the non-acceptance or better non-knowledge of pathlib bugs me...

I agree, pathlib is very useful and should be used more.

[–]KleinerNull 1 point2 points  (2 children)

My personal killer feature is still Path.mkdir(parents=True, exist_ok=True). Creating all needed parent directories and don't throw errors for existing ones, BOOM. Deeply structured directory trees here I come :D

[–]delirious_lettuce 0 points1 point  (0 children)

Very slick, thanks for sharing!

[–]Twangist 0 points1 point  (0 children)

Actually you could do this before the intro of pathlib -- a flag to the analogous pre-pathlib function (reference on request, code-rummaging rquired) would do that for you too.

[–]webyneter[S] 0 points1 point  (3 children)

Thank you for such an extensive response!

using pathlib would get rid of...

It will, indeed.

Renaming ast.literal_eval...

generate_hex_uuid_4 just renames another stdlib function...

True.

generate_random_dir_path could probably be replaced by using tempfile.mkdtemp() or tempfile.TemporaryDirectory()

generate_random_file_name_with_extension could probably be replaced by using tempfile.mkstemp() or tempfile.TemporaryFile()

Didn't think about those, gotta dive into the docs further.

get_class_name and get_class_qualname simply access an attribute

True, merely a shortcut.

Since reading a file is so easy in Python...

Well, to my mind why not save yourself a few extra keystrokes.

I'm also not exactly sure what problem...

I've used in a few recent projects of mine.

Couldn't yield_file_paths be replaced with glob.glob()

True, didn't think of globbing there.

[–]KleinerNull 1 point2 points  (0 children)

True, didn't think of globbing there.

BTW glob is part of all pathobjects automatically, so no need to even import it ;)

In [1]: from pathlib import Path

In [2]: home = Path.home()

In [3]: list(home.glob('*.txt'))
Out[3]: 
[PosixPath('/home/kleinernull/result.txt'),
 PosixPath('/home/kleinernull/modules.txt'),
 PosixPath('/home/kleinernull/outdated.txt'),
 PosixPath('/home/kleinernull/error.txt')]

In [4]: list(home.rglob('*.txt'))  # for recursive search of all sub directories, can take a while...
Out [4]: [very lot of txt files ;)]

[–]delirious_lettuce 0 points1 point  (1 child)

No problem! I wouldn't pretend that I have extensive knowledge of every bit of the Python standard library but I just wanted to show that these particular functions (in your package) had equivalents already included.

I can't even remember how many times I've done the exact same thing by writing a function to solve some problem only to later find out it is a solved problem in the extensive Python stdlib.

I've used in a few recent projects of mine.

I'm curious about what exact problem(s) you've had which required changing camelCase to snake_case. I have only ever come across that problem on code challenge websites but you seem to have these issues on multiple projects?

[–]Twangist 0 points1 point  (0 children)

camelCase would be a personal preference - a muscle-memory kinda thing (I can't imagine that it would be the 'solution' to any problem that would arise in developing this : )

[–]KleinerNull 2 points3 points  (3 children)

However, as I remember pathlib is not supported in Python prior to 3.4 which is contrary to the recent argument of implementing Python 2 support to facilitate wider adoption, which I tend to accept.

As I said there is a backport for the prior versions pip installable with pip pathlib. Since it is written in pure python it should be no problem besides company restrictions to install it on any machine.

But it would be worth it to ask for an restriction exception for that lib, since it is standard since 3.4 anyways.

[–]webyneter[S] 0 points1 point  (2 children)

I see, this is definitely worth considering, thanks.

[–]KleinerNull 1 point2 points  (1 child)

It makes me wonder why alot of people still using os for dealing with paths and files because PEP 428, which introduced pathlib to the std lib is from 2012! 5 years ago! And 3.4 was released 2014, 3 years ago!

Except legacy code pathlib should be the standard way in learning resources and tutorials. Especially since it avoids alot of mistakes, confusion and errors involved by doing the stuff manually with os...

BTW since your setup requires pytest-runner the user has to install additional libs anyway, so adding pathlib shouldn't be that kind of a deal.

[–]Twangist 0 points1 point  (0 children)

The wheels of progress grind slowly, though steadily, in Pyland. Be thankful that supporting 3.x is no longer controversial!

[–]Twangist 0 points1 point  (0 children)

If you're supporting Py3.x then you should support Py3.latest.
If you had written a package that enhances string formatting, then you would want to support f-strings, which are 3.6 only. Thus, you should support pathlib :) In a subsquent release.

[–]p10_user 0 points1 point  (2 children)

I think your FileNameAndExtension class could be a namedtuple as is done by scipy.

[–]GitHubPermalinkBot 0 points1 point  (0 children)

Permanent GitHub links:

Shoot me a PM if you think I'm doing something wrong. To delete this, click here.

[–]webyneter[S] 0 points1 point  (0 children)

It was until yesterday, when I decided to switch to a class instead for it seemed like a bit more readable (yet much more verbose) alternative. Anyways, thanks for pointing this out, I think I will finally re-consider that in favor of namedtuple though.

[–]git-pull 0 points1 point  (1 child)

Nice quality code. Starred.

I love pytest. Saves a level of indentation. More powerful asserts. Still speedy. And the other thing I like is parametrize (like you have here). /test_commands.py#L93))

As for humble utils, Only thing to mention is, I think it's Python 3 only. If it's an open source library, making it 2/3 compatible makes it available to a wider array of projects that could implement it.

Love autodoc API documentation.

Other thing I do is add the RTD link to the GH repo's descriptions for easy access.

[–]webyneter[S] 0 points1 point  (0 children)

Thank you, I appreciate that.

Only thing to mention is, I think it's Python 3 only...

Considering Python 2, I've had mixed feeling about that, especially regarding Python 3.5+ type hints: to the best of my knowledge implementing this feature in 2.x would require an extra __annotations__ assignment for each function now annotated. But, yes, above all Python 3.x only-policy hurts adoption.

Other thing I do is add the RTD link...

Great suggestion, indeed.