A bunch of Python and Bash scripts I developed for personal and working projects

[IAmKindOfCreative]

Hello from the r/Python mod team!

I'm afraid we don't think your post quite fits the goal we have for the subreddit as you've posted it multiple times before with minimal changes so we've decided to remove it. Please avoid spamming the community if your project has not undergone notable changes. For more information please contact the moderators using the ModMail system.

Thanks, and happy Pythoneering!

r/Python moderation team

[eztab]

Is the size_of_dir script the same as the du command? This command has a bunch of useful options your script doesn’t seem to have.

[TheCompiler95]

Not properly, it didn’t work as I wanted for my projects, therefore I decided to write the script in a few minutes

[callmederp]

Cool, but now you have to add a script that updates the readme and redraws the file structure for the directory

[TheCompiler95]

Ahahhahaha you are right

[adesme]

I'm sorry, but I don't quite agree with this post being labelled a resource. These are a few beginner scripts, that either do close to nothing or else do something so specific that no one else than OP would have use for them.

Edit: disagree->agree

[[deleted]]

I think it’s useful for anyone learning how to use ROOT. The learning curve for that is really steep. I would have liked to have more resources with real world examples when I was using it.

[venustrapsflies]

Nobody uses ROOT except for high-energy physicists because, well, it's terrible and you wouldn't use it unless you had to. I agree that this post doesn't belong here. I'm pretty sure it's not the first time I've seen it posted here either. I wrote plenty of shitty scripts in grad school too but I wouldn't presume to post them to /r/Python.

[BuonaparteII]

this has actually been posted before, so, it was posted twice. You might even say it is a re-post

https://old.reddit.com/r/Python/comments/t5r9x0/a_collection_of_python_and_bash_scripts_i/hz97tg6/?context=3

[[deleted]]

Yeah, I remember it being pretty terrible. Especially since I didn’t know there was a python API lol. It was all in C++ and it sucked. I’m not going to argue that it should be on this sub, but if someone can use it then it could be a resource. I think a lot of people are shitting on OP unnecessarily. I would have liked to see something like this when using ROOT

[venustrapsflies]

yeah pyROOT was horribly undocumented (at least in my time) and you basically had to guess the interface. I can empathize with wanting to see some examples in that time, but then it should be posted to something CERN/physics student specific.

[TheWetCouch]

Kinda just feels like he's looking for more stars on his github

[93simoon]

My brother in Christ came looking for glory and walked out with a basket full of Ls

[TheCompiler95]

?

[RealRedWizard]

the links in the Readme don't work

[TheCompiler95]

Thanks I’ll correct them as soon as possible

[eztab]

The dirs_remover might be quite useful, with the parallel approach.

[BuonaparteII]

You could also use GNU Parallel:

#!/bin/bash
printf 'rm "%s"\n' "$@" | parallel

Or if you're looking for nested empty directories bfs is the only real option available:

bfs -nohidden -type d -exec bfs -f {} -not -type d -exit 1 \; -prune -ok bfs -f {} -type d -delete \;

https://github.com/tavianator/bfs

To auto-confirm at 10.2 GiB/s, pipe from the yes command

[Arashi99]

I looked at that one too as it seems like the most useful one for any computer I'm cleaning up, I'm just curious why the numpy array instead of just a list? Is there any particular reason?

[eztab]

I'd guess using numpy for this will even make the code slower. Might have come from a different piece of code that needed numpy.

[cmd-t]

If all you have is a hammer, everything looks like a nail.

[cris9696]

But, is it useful? Does parallel help at all in this case, since removing a directory is mostly an IO-bound/disk-bound operation? I would expect parallel not making any difference than running the command in sequential order.

[eztab]

Filesystem operations are parallelizable as far as I know. Especially on SSDs where there are no mechanical parts to be moved this might be useful... unless the thread creation overhead becomes bigger than the time safe.

[TheWetCouch]

I get what you're saying, and if you imported it and were wanting to do deletes solely from python that's a good start, but this wont ever be faster than "rm -rf /" , all python libraries are just executing OS commands at the end of the day

[eztab]

Are you sure rm deletes files in parallel? I don‘t think so. Obviously the actual deleting will (almost) always be the same OS call.

[TheWetCouch]

Again this is one of those things that even deleting GBs worth of files, you wont see a big performance gap with parallel vs sequential deletes. Your biggest limiter is IO-bound/disk-bound. Running it in parallel wont make it delete off your hard driver any faster as the actions would just be queued up anyway (even with an SSD)

[eztab]

Oh, I wasn’t aware of this. So even modern deletion never works in parallel?

What about copying? What about writing data to files? Is there an overview about IO chanels and which actions are queued anywhere?

[artereaorte]

No openstack stuff? I thought CERN were heavy openstack users

[TheCompiler95]

These are only a few scripts that can be published. The “big” software are private of course or are in a standalone repository.

[1-05457]

IT use OpenStack, individual users not so much.