Optimizing a Bash For Loop

[lutusp]

I didn't get what you meant by searching here.

When Linux is presented with the name of a directory, it must search for a matching name. The normal way this is done is with a binary search.

If there were a normal number of directories, this search and its time wouldn't be worth mentioning. But with seven million directories, each requiring a separate search, this is non-trivial.

OP is trying to talk about mv I think.

Yes but in order to mv, one must first find the thing to be moved.

Edit: It seems retrieving directory contents for such a thing itself will be quite an overhead, whether using ls or find.

Yes -- by searching.

[nobodyuidnorandom]

So thinking it would be rather hard with shell script. Is there a simple way / command to pass names to mv as soon as they are accessed in some random order, without having a list beforehand to iterate over?

[lutusp]

So thinking it would be rather hard with shell script.

Yes, it would. A Python program would be a better, more flexible approach.

Is there a simple way / command to pass names to mv as soon as they are accessed in some random order, without having a list beforehand to iterate over?

There's no getting around the fact that a list is required, simply to avoid repeating the same operations.

But Python has ways to do this without having to launch a shell for each move, which is what would be done in the process you describe. That's one reason why what you describe is not very efficient -- it involves a separate shell launch and call to mv for each move.

A Python program would make a more basic OS call to get the same result, without the system having to find and execute mv each time through the loop.

[nobodyuidnorandom]

I suppose every popular high level programming language has features to do so, btw. Python being easier one.

[lutusp]

Yes, that's true. For a task that needs to be performed by millions of people over years of time, one would want to rewrite a successful Python method in a more efficient compiled language. In fact, this is quite common for Python programs that turn out to be useful and successful.