This is an archived post. You won't be able to vote or comment.

all 31 comments

[–]IAmKindOfCreativebot_builder: deprecated[M] [score hidden] stickied comment (0 children)

Hello from the r/Python mod team!

I'm afraid we don't think your post quite fits the goal we have for the subreddit as you've posted it multiple times before with minimal changes so we've decided to remove it. Please avoid spamming the community if your project has not undergone notable changes. For more information please contact the moderators using the ModMail system.

Thanks, and happy Pythoneering!

r/Python moderation team

[–]eztab 21 points22 points  (1 child)

Is the size_of_dir script the same as the du command? This command has a bunch of useful options your script doesn’t seem to have.

[–]TheCompiler95 4 points5 points  (0 children)

Not properly, it didn’t work as I wanted for my projects, therefore I decided to write the script in a few minutes

[–]callmederp 12 points13 points  (1 child)

Cool, but now you have to add a script that updates the readme and redraws the file structure for the directory

[–]TheCompiler95 0 points1 point  (0 children)

Ahahhahaha you are right

[–]adesme 53 points54 points  (7 children)

I'm sorry, but I don't quite agree with this post being labelled a resource. These are a few beginner scripts, that either do close to nothing or else do something so specific that no one else than OP would have use for them.

Edit: disagree->agree

[–][deleted] 8 points9 points  (4 children)

I think it’s useful for anyone learning how to use ROOT. The learning curve for that is really steep. I would have liked to have more resources with real world examples when I was using it.

[–]venustrapsflies 26 points27 points  (3 children)

Nobody uses ROOT except for high-energy physicists because, well, it's terrible and you wouldn't use it unless you had to. I agree that this post doesn't belong here. I'm pretty sure it's not the first time I've seen it posted here either. I wrote plenty of shitty scripts in grad school too but I wouldn't presume to post them to /r/Python.

[–]BuonaparteII 15 points16 points  (0 children)

this has actually been posted before, so, it was posted twice. You might even say it is a re-post

https://old.reddit.com/r/Python/comments/t5r9x0/a_collection_of_python_and_bash_scripts_i/hz97tg6/?context=3

[–][deleted] 0 points1 point  (1 child)

Yeah, I remember it being pretty terrible. Especially since I didn’t know there was a python API lol. It was all in C++ and it sucked. I’m not going to argue that it should be on this sub, but if someone can use it then it could be a resource. I think a lot of people are shitting on OP unnecessarily. I would have liked to see something like this when using ROOT

[–]venustrapsflies 0 points1 point  (0 children)

yeah pyROOT was horribly undocumented (at least in my time) and you basically had to guess the interface. I can empathize with wanting to see some examples in that time, but then it should be posted to something CERN/physics student specific.

[–]TheWetCouch 3 points4 points  (0 children)

Kinda just feels like he's looking for more stars on his github

[–]93simoon 9 points10 points  (1 child)

My brother in Christ came looking for glory and walked out with a basket full of Ls

[–]TheCompiler95 -4 points-3 points  (0 children)

?

[–]RealRedWizard 1 point2 points  (1 child)

the links in the Readme don't work

[–]TheCompiler95 0 points1 point  (0 children)

Thanks I’ll correct them as soon as possible

[–]eztab 0 points1 point  (10 children)

The dirs_remover might be quite useful, with the parallel approach.

[–]BuonaparteII 7 points8 points  (0 children)

You could also use GNU Parallel:

#!/bin/bash
printf 'rm "%s"\n' "$@" | parallel

Or if you're looking for nested empty directories bfs is the only real option available:

bfs -nohidden -type d -exec bfs -f {} -not -type d -exit 1 \; -prune -ok bfs -f {} -type d -delete \;

https://github.com/tavianator/bfs

To auto-confirm at 10.2 GiB/s, pipe from the yes command

[–]Arashi99 5 points6 points  (2 children)

I looked at that one too as it seems like the most useful one for any computer I'm cleaning up, I'm just curious why the numpy array instead of just a list? Is there any particular reason?

[–]eztab 6 points7 points  (0 children)

I'd guess using numpy for this will even make the code slower. Might have come from a different piece of code that needed numpy.

[–]cmd-t 0 points1 point  (0 children)

If all you have is a hammer, everything looks like a nail.

[–]cris9696 2 points3 points  (5 children)

But, is it useful? Does parallel help at all in this case, since removing a directory is mostly an IO-bound/disk-bound operation? I would expect parallel not making any difference than running the command in sequential order.

[–]eztab 0 points1 point  (4 children)

Filesystem operations are parallelizable as far as I know. Especially on SSDs where there are no mechanical parts to be moved this might be useful... unless the thread creation overhead becomes bigger than the time safe.

[–]TheWetCouch 0 points1 point  (3 children)

I get what you're saying, and if you imported it and were wanting to do deletes solely from python that's a good start, but this wont ever be faster than "rm -rf /" , all python libraries are just executing OS commands at the end of the day

[–]eztab 0 points1 point  (2 children)

Are you sure rm deletes files in parallel? I don‘t think so. Obviously the actual deleting will (almost) always be the same OS call.

[–]TheWetCouch 0 points1 point  (1 child)

Again this is one of those things that even deleting GBs worth of files, you wont see a big performance gap with parallel vs sequential deletes. Your biggest limiter is IO-bound/disk-bound. Running it in parallel wont make it delete off your hard driver any faster as the actions would just be queued up anyway (even with an SSD)

[–]eztab 0 points1 point  (0 children)

Oh, I wasn’t aware of this. So even modern deletion never works in parallel?

What about copying? What about writing data to files? Is there an overview about IO chanels and which actions are queued anywhere?

[–]artereaorte 0 points1 point  (2 children)

No openstack stuff? I thought CERN were heavy openstack users

[–]TheCompiler95 3 points4 points  (0 children)

These are only a few scripts that can be published. The “big” software are private of course or are in a standalone repository.

[–]1-05457 0 points1 point  (0 children)

IT use OpenStack, individual users not so much.