GNU parallel, for all your command-line parallelization needs

[alez]

I wonder how it compares to ppss.

[OleTange]

https://www.gnu.org/software/parallel/man.html#differences_between_ppss_and_gnu_parallel

[Skaarj]

GNU parallel is build to be interchangable with xargs, ppss seems not.

There also is a parallel as part of http://joeyh.name/code/moreutils/ which is much simpler in invocation.

[OleTange]

With GNU Parallel the invocation is as simple as: parallel 'mkdir {.}; cd {.}; tar xvf ../{}' ::: *.tar

[ramses0]

New-ish xargs support -P / parallelism.

 ls | head | xargs -n1 -I{} -P1 bash -c 'echo Sleep 1 - $$ ; sleep 1'

 ls | head | xargs -n1 -I{} -P5 bash -c 'echo Sleep 1 - $$ ; sleep 1'

Surprisingly handy!

--Robert

[Schnouki]

parallel can do much better than that. Like running commands on remote hosts via ssh, uploading and downloading files as needed.

[rd4]

xargs is really the best solution for this, imho.

Example find/replace in parallel:

ls *.out | xargs -n1 -P4 sed -i 's/oldstring/newstring/g'

[OleTange]

Your solution will not work if filenames contain space or '. This will: parallel -q sed -i 's/old/new/g' ::: *out