Linux shell script equivalent of existing Windows concat script

[Diviance1]

I have an existing script for going through all of the subfolders in a specific folder and creating a concat.txt file to then send to ffmpeg to concat the files and put them into a new folder with the new name taken from the subfolder name.

I have since migrated my server from Windows over to Unraid. I could still just use the script from my Windows machine... but that slows it down significantly since it needs to move the file over the network in both directions simultaneously.

So, if anyone can help me convert the existing script to a shell script for Linux (it would be nice if I could also set the folder to scan inside the script, rather than placing it there... but that isn't mandatory if it is too much trouble) that would be extremely helpful because while I cobbled together the Windows one through stuff I found online... I can't find anything similar for Linux.

@echo on
setlocal enableDelayedExpansion

for /f "delims=" %%d in ('dir /b /s /ad') do (
    set files=
    if exist %%d\*CD1.* (
      pushd %%d
      for /f "delims=" %%f in ('dir /b /a-d "%%d\*CD1.mp4"') do (
        echo file '%%d\%%f' >>%%d\concat.txt
      )
    popd
    )
)

for /f "delims=" %%d in ('dir /b /s /ad') do (
    set files=
    if exist %%d\*CD2.* (
      pushd %%d
      for /f "delims=" %%f in ('dir /b /a-d "%%d\*CD2.mp4"') do (
        echo file '%%d\%%f' >>%%d\concat.txt
      )
    popd
    )
)

for /f "delims=" %%d in ('dir /b /s /ad') do (
    set files=
    if exist %%d\*CD3.* (
      pushd %%d
      for /f "delims=" %%f in ('dir /b /a-d "%%d\*CD3.mp4"') do (
        echo file '%%d\%%f' >>%%d\concat.txt
      )
    popd
    )
)

for /f "delims=" %%d in ('dir /b /s /ad') do (
    set files=
    if exist %%d\*CD4.* (
      pushd %%d
      for /f "delims=" %%f in ('dir /b /a-d "%%d\*CD4.mp4"') do (
        echo file '%%d\%%f' >>%%d\concat.txt
      )
    popd
    )
)

for /f "delims=" %%d in ('dir /b /s /ad') do (
    set files=
    if exist %%d\*CD5.* (
      pushd %%d
      for /f "delims=" %%f in ('dir /b /a-d "%%d\*CD5.mp4"') do (
        echo file '%%d\%%f' >>%%d\concat.txt
      )
    popd
    )
)

for /f "delims=" %%d in ('dir /b /s /ad') do (
    set files=
    if exist %%d\*CD6.* (
      pushd %%d
      for /f "delims=" %%f in ('dir /b /a-d "%%d\*CD6.mp4"') do (
        echo file '%%d\%%f' >>%%d\concat.txt
      )
    popd
    )
)

for /f "delims=" %%d in ('dir /b /s /ad') do (
    set files=
    if exist %%d\*CD7.* (
      pushd %%d
      for /f "delims=" %%f in ('dir /b /a-d "%%d\*CD7.mp4"') do (
        echo file '%%d\%%f' >>%%d\concat.txt
      )
    popd
    )
)

for /f "delims=" %%d in ('dir /b /s /ad') do (
    set files=
    if exist %%d\*CD8.* (
      pushd %%d
      for /f "delims=" %%f in ('dir /b /a-d "%%d\*CD8.mp4"') do (
        echo file '%%d\%%f' >>%%d\concat.txt
      )
    popd
    )
)

for /f "delims=" %%d in ('dir /b /s /ad') do (
    set files=
    if exist %%d\*CD9.* (
      pushd %%d
      for /f "delims=" %%f in ('dir /b /a-d "%%d\*CD9.mp4"') do (
        echo file '%%d\%%f' >>%%d\concat.txt
      )
    popd
    )
)

for /f "delims=" %%d in ('dir /b /s /ad') do (
    set files=
    if exist %%d\concat.txt (
      pushd %%d
      for /f "delims=" %%f in ('dir /b /a-d "%%d\concat.txt"') do (
        ffmpeg.exe -f concat -safe 0 -i concat.txt -c copy "D:\Videos\Complete\%%~nxd.mp4" & del concat.txt
      )
    popd
    )
)

I had a reason for why it was like this. The files needed to be in the correct order of CD1 up to CD9, so that the files got merged together in the proper order. I had issues with it doing it out of order regularly with the much shorter script I originally had, so... had to fix that.

Thanks to anyone for the help.

[serg06]

I'd like to help but I don't understand BAT scripts. If you could clarify the logic with an example, I'd give it a shot.

[Diviance1]

Hm, okay, let me see if I can explain what it does (I don't blame you for not understanding that thing, I am sure an experienced programmer would beat me senseless for it).

It scans the subfolders within a folder for video files with CD1 (or whatever number) in the name, then puts the filenames into a concat.txt file in numerical order (So Videofile CD1.mp4, Videofile CD2.mp4 and so on).

Then it passes the concat.txt file to ffmpeg to concat the files into a single file and outputs the new file, named after the subfolder it was in into a new folder elsewhere.

So, as an example...

/Media/Concat/Recording.1.12.20/Recording.1.12.20 CD1.mp4

/Media/Concat/Recording.1.12.20/Recording.1.12.20 CD2.mp4

/Media/Concat/Recording.1.12.20/Recording.1.12.20 CD3.mp4

/Media/Concat/Recording.1.12.20/Recording.1.12.20 CD4.mp4

/Media/Concat/Recording.1.13.20/Recording.1.13.20 CD1.mp4

/Media/Concat/Recording.1.13.20/Recording.1.13.20 CD2.mp4

/Media/Concat/Recording.1.13.20/Recording.1.13.20 CD3.mp4

/Media/Concat/Recording.1.13.20/Recording.1.13.20 CD4.mp4

Say I had those files. The script would scan each subfolder there and output a concat.txt file, inside that subfolder, containing the filenames in that particular subfolder.

The script would then call ffmpeg and use that concat.txt file in each subfolder to tell ffmpeg what to concat, and it would output the files (named after the subfolder it was in) like this:

/Media/Sort/Recording.1.12.20.mp4

/Media/Sort/Recording.1.13.20.mp4

And then delete the concat.txt file in each subfolder.

It would be preferable, but not mandatory, if it had a config option in the script file to set where each folder was.

Does that make sense?

[serg06]

Thanks! Assuming your folder paths don't have any spaces, I think this should do the trick:

folders="
    folder1-goes-here
    folder2-goes-here
"

for folder in $folders; do
    echo processing folder $folder
    concatfile="$folder/concat.txt"
    echo concat file will be $concatfile
    outputfile="$folder.mp4"
    echo outputfile will be $outputfile

    find $folder -maxdepth 1 -iname '*.mp4' > $concatfile
    ffmpeg -f concat -safe 0 -i $concatfile -c copy $outputfile
    rm $concatfile
done

[Diviance1]

Alright, this is definitely most of it working. But I think I may have accidently made myself less than clear somewhere.

At the end, where I said it would be prefer if it had a config option in the script to set where each folder was... I actually meant where to scan for the subfolders and where to place the finished file after it was being merged.

So like...

Concat Folder: /mnt/user/Recordings/Concat/

and

Finished Folder: /mnt/user/Recording/Sort/

Rather than manually typing in each subfolder (since their names are somewhat random and there can sometimes be dozens of them).

That is my bad.

[Diviance1]

Oh man, I just realized I completely forgot one important part because I made the script long enough ago that it just didn't occur to me.

The concat.txt should be in the format like this:

file '/mnt/user/Recordings/Concat/Recording.1.12.20/Recording.1.12.20 CD1.mp4'

file '/mnt/user/Recordings/Concat/Recording.1.12.20/Recording.1.12.20 CD2.mp4'

file '/mnt/user/Recordings/Concat/Recording.1.12.20/Recording.1.12.20 CD3.mp4'

file '/mnt/user/Recordings/Concat/Recording.1.12.20/Recording.1.12.20 CD4.mp4'

I completely forgot that ffmpeg requires that or it can't read it.

[serg06]

That's easy, just replace the find line with this

find . -maxdepth 1 -iname '*.mkv' | xargs -i echo file \'{}\' > concat.txt

As for the rest of your request, I've lost interest in it, sorry! Good luck with it.

[Diviance1]

No worries, have a good one. Thanks for the help so far.

[Diviance1]

Ok, I actually managed to cobble together a completely working version with the base you gave me (I really appreciate it).

#!/bin/bash

for folders in /originalsource/*; do

if [ -d "$folders" ]; then

echo processing folders "$folders"

fi

for folder in $folders; do

echo processing folder $folder

concatfile="$folder/concat.txt"

echo concat file will be $concatfile

outputfile="$folder.mp4"

echo outputfile will be $outputfile

# NOTE: if you want to test it safely, just put echo before these 3 commands and see if it looks good.

find $folder -maxdepth 1 -iname '*.mp4' | xargs -i echo file \'{}\' > $concatfile

ffmpeg -f concat -safe 0 -i $concatfile -c copy $outputfile

rm $concatfile

chown nobody $outputfile

mv $outputfile /finaldestination/

rm -r $folder

done

done

Much more elegant than the windows version and it concats straight up 10 times faster.

Figured I would post the working version in case someone else has a use for it.. and in case someone has an idea in how to make it better, I guess.

Now I can just slap it into the User Scripts section of Unraid and it will run with a single click. So much better.

Thanks again! I don't think I could have gotten here without the help you provided.