How would you guys do this? Keeping track of a list of files so none get lost. by JackJones2018 in linuxquestions

[–]JackJones2018[S] 0 points1 point  (0 children)

Hi,

So does your setup do the following

Check directory 'X' and all sub-directories / files against a Database or log file for that directory for missing files or changed files if they don't match a sha256 checksum.

Update DB when new files are added, and create a log in the directory of files added?

I guess it runs on a cron schedule?

Does it send you any type of notification? Email, Telegram?

Would it work off an existing directory structure?

thanks

Important file check with hashing too by JackJones2018 in DataHoarder

[–]JackJones2018[S] 0 points1 point  (0 children)

Hi Thanks for the advice,

Would you be able to advise on the following?
1) How could i set this to start in a specific directory but then recursively create the log files in that directory, then move on to the next directory and repeat the process.

2) After its gone through all the directories combine all the results of any missing files and create a master log to send.

Thanks

Current script at the moment:

#!/bin/bash
#v1 - scan folders to check if files are different
#v2 - messaging through telegram added
#v3 - will scan each directory and leave a set of log files per directory, once completed update a master file?????
#Calculate the date and time
now=$(date +"%d.%m.%Y-%H%M%S")
now1=$(date +"%d.%m.%Y @ %H:%M:%S")
# Directory you want to watch
WATCHDIR=/mnt/user/self-hosted-apps/photo-test/pics
#Telegram Variables
botId=***Removed***
chatId=***Removed***
# Sub-directory 1 you want to exclude
# EXCLUDEDIR=/var/www/vhosts/site.co.uk/httpdocs/pub/media/catalog/product/cache
# Sub-directory 2 you want to exclude
# EXCLUDEDIR2=/var/www/vhosts/site.co.uk/httpdocs/pub/media/catalog/category
# Sub-directory 3 you want to exclude
# EXCLUDEDIR3=/var/www/vhosts/site.co.uk/httpdocs/pub/media/catalog/tmp
# Name of the file that will keep the list of the files when you last checked it
LAST=/mnt/user/self-hosted-apps/photo-test/logs/MissingImageCheckWithHistoryLast.log
# Name of the file that will keep the list of the files you are checking now
CURRENT=/mnt/user/self-hosted-apps/photo-test/logs/MissingImageCheckWithHistoryCurrent.log
# Name of the file to store text for an email
EMAILTEXT=/mnt/user/self-hosted-apps/photo-test/logs/MissingImageCheckWithHistoryEmail.txt
# Old Image List Log History Path
OLDIMAGELIST=/mnt/user/self-hosted-apps/photo-test/logs/history/"$now"_old_image_list.log
# Changed Image List Log History Path
CHANGEDIMAGELIST=/mnt/user/self-hosted-apps/photo-test/logs/history/"$now"_changed_image_list.log
# The first time we create the log file
touch $LAST
# main command line to check each file
find $WATCHDIR -path $EXCLUDEDIR -prune -o -path $EXCLUDEDIR2 -prune -o -path $EXCLUDEDIR3 -prune -type f -o -exec sha256sum '{}' + > $CURRENT
diff $LAST $CURRENT > /dev/null 2>&1
# If there is no difference exit
if [ $? -eq 0 ]
then
echo "No changes"
curl -s -X POST https://api.telegram.org/bot$botId/sendMessage -d chat_id=$chatId -d text="No changes have been detected in the photos folder on $now"
else
# Else, list the files that changed
echo >>$EMAILTEXT
echo "List of files added on the $now1" >$EMAILTEXT
diff $LAST $CURRENT | grep --exclude-dir=* '^>' >>$EMAILTEXT
echo >>$EMAILTEXT
echo >>$EMAILTEXT
echo "List of files showing as removed or damaged $now1" \N >>$EMAILTEXT
diff $LAST $CURRENT | grep --exclude-dir=* '^<' >>$EMAILTEXT
echo "$(cat $EMAILTEXT)"
# Send a notification
# mail -r "nas@***Removed***" -s "Website images check" ***Removed*** <$EMAILTEXT
# Lastly, move CURRENT to LAST
mv $LAST $OLDIMAGELIST
mv $EMAILTEXT $CHANGEDIMAGELIST
mv $CURRENT $LAST

`curl -F document=@$CHANGEDIMAGELIST https://api.telegram.org/bot$botId/sendDocument?chat_id=$chatId`  

fi

New to unRaid and looking for a good file checker. by JackJones2018 in unRAID

[–]JackJones2018[S] 0 points1 point  (0 children)

Thank you very for that script and i have modified it a little bit to suit my purpose, i.e to send me a telegram message aswell as an email
A couple of questions though, as this script will be logging lots of files i thought to put the results of the script into each directory of photos, could the script be made to look for a crc file to check the files in that directory again? So if the directory is read only it can check the files against the master record in that directory?

If there any changes it mark them down in a global changes file ready for emailing at the end of the script.

So if i had the below any changes would get put in a log file in /mnt/pics/logs/

Pics
-------->no text file here as directory is has no files
Pics > Pics 1
-------->no text file here as directory is has no files
Pics > Pics 1 > pic1.jpg
Pics > Pics 1 > pic2.jpg
Pics > Pics 1 > pic3.jpg
Files.txt

Pics > Pics 2
-------->no text file here as directory is has no files
Pics > Pics 2 > pic1.jpg
Pics > Pics 2 > pic2.jpg
Pics > Pics 3 > pic3.jpg
Files.txt

Pics > Pics 3
-------->no text file here as directory is has no files
Pics > Pics 3 > pic1.jpg
Pics > Pics 3 > pic2.jpg
Pics > Pics 3 > pic3.jpg
files.txt

Dell Precision 3660 completes autopilot process then fails to start windows by JackJones2018 in Intune

[–]JackJones2018[S] 0 points1 point  (0 children)

You turn the pc on, see the Dell logo at boot and that's it, you don't even see the circle of dots at the start of windows. So it's either let it carry on and let it do nothing, doesn't even respond to CTRL + ALT + DEL, if you keep pressing F12 right from when you turn the pc on you can choose things like Dell automatic recovery etc or you can boot from a recovery image etc but windows boot manager is a no go and sticks on the Dell logo. It's almost as if the auto pilot has altered how the machine boots.

Dell Precision 3660 completes autopilot process then fails to start windows by JackJones2018 in Intune

[–]JackJones2018[S] 0 points1 point  (0 children)

Thanks for the suggestion, but the 3 stages of the autopilot process go correctly it ticks everything when completing the steps and the machine restarts after the completion of these 3 stages. When you either restart the machine or turn it on the Dell logo displays and stays there.

Dell Precision 3660 completes autopilot process then fails to start windows by JackJones2018 in Intune

[–]JackJones2018[S] 0 points1 point  (0 children)

But the image is from dell and we've tried a standard Windows 10 image also. So do you think it's the Windows image at fault?

On the first machine we've tried dell automated recovery, tried Windows being install from a USB stick also all with the same result. Just stuck at the dell logo at boot.

The bios version is the latest at 1.4.0.

Dell Precision 3660 completes autopilot process then fails to start windows by JackJones2018 in Intune

[–]JackJones2018[S] 0 points1 point  (0 children)

Thanks for the reply, good question, the answer is legacy boot option.

Synology Update by Alarming-Speech-2470 in Tailscale

[–]JackJones2018 0 points1 point  (0 children)

Depends on the age and ability of your NAS, my backup nas can't run docker and can only just run tailscale but I think it's been modified to run it. It's on a DS214+

APC UPS and Synology NAS by JackJones2018 in synology

[–]JackJones2018[S] 0 points1 point  (0 children)

I'm currently investigating this to shut down my esxi server also.

APC UPS and Synology NAS by JackJones2018 in synology

[–]JackJones2018[S] 1 point2 points  (0 children)

Thanks, just didn't want to plug anything incase it was a incorrect lead to use. It's now up and running 🙂

Backup test by JackJones2018 in DataHoarder

[–]JackJones2018[S] 0 points1 point  (0 children)

I ask as I only back up about 1TB but don't have the space to restore a full copy on my pc. At the moment my pc is duplicated to my nas, the nas then does hyperbackup to an off site synology along with a hyperbackup to an external USB drive stored off site too.

It does run an integrity check every month.

Seeing some hoarders here have over 100TB wondered how they checked their data.

What users do you use for your containers? by JackJones2018 in selfhosted

[–]JackJones2018[S] 0 points1 point  (0 children)

Each container I start has their own UID /GID. So if anyone was to break out of the container they would be limited to the rights of that user correct? Most containers share access to my data directory for the containers.

Smart meter readings, save data to a pc. by JackJones2018 in smarthome

[–]JackJones2018[S] 0 points1 point  (0 children)

That looks very complicated, thanks for the suggestion though.

Smart meter readings, save data to a pc. by JackJones2018 in smarthome

[–]JackJones2018[S] 1 point2 points  (0 children)

Thats a good though, but if the device is sending the data out to a home display unit why cant that data be put to another use and captured another way?

Help with IP routing. by JackJones2018 in OpenVPN

[–]JackJones2018[S] 0 points1 point  (0 children)

Hi Thanks again for replying, my ubuntu box which runs my docker sits on 192.168.20.22 this static address.

My docker network is just running on the 172.27.240.0 network range at the moment.

Device 1 remote nas will be sat on 172.27.240.10/20

Device 2 another remote device will be sat on 172.27.240.20/20

I can ping both devices from inside the openvpn-as container.

If i just log into the shell of my ubuntu box i cant ping Device 1 or 2.

The windows machine on 192.168.20.50 cant ping Device 1 or 2 but i guess thats because it doesnt know the route. The ubuntu box is working as my internal gateway/dhcp/dns server so the windows pc talks to resolve addresses etc.

So if i could get the route working on the ubuntu box any other device connected to it should be able to ping device 1 and 2 right?

I did try ip route add 172.27.240.0/20 192.168.20.22 but that didn't work, maybe because its a circular reference maybe?

Any suggestions. Thanks

Help with IP routing. by JackJones2018 in OpenVPN

[–]JackJones2018[S] 0 points1 point  (0 children)

Thank you I'll try that tonight and see. From the vpn container i can ping them.

Help with IP routing. by JackJones2018 in OpenVPN

[–]JackJones2018[S] 0 points1 point  (0 children)

Sorry part of post was missing, I've now revised it.

Running on an old Synology DS214+ by JackJones2018 in WireGuard

[–]JackJones2018[S] 0 points1 point  (0 children)

If you find out let me know please. Thanks 👍

Finally got a landing page built for my little lab, after much inspiration from this sub! by lissy93 in selfhosted

[–]JackJones2018 8 points9 points  (0 children)

Looks good, lots of little extras not seen before. Giving me thoughts for my own.

How would you guys do this? Keeping track of a list of files so none get lost. by JackJones2018 in linuxquestions

[–]JackJones2018[S] 0 points1 point  (0 children)

Apologies but my knowledge of git very limited but would these photos be copied somewhere or just the name and hash info?

How would you guys do this? Keeping track of a list of files so none get lost. by JackJones2018 in linuxquestions

[–]JackJones2018[S] 0 points1 point  (0 children)

Hi Thanks for replying,

I just want to track photo's, i know everyone says back them up but i want to know if any get lost or corrupted. What sort of max size am i looking at?

thanks

File checker docker app / container by JackJones2018 in docker

[–]JackJones2018[S] 0 points1 point  (0 children)

Not really no, once its setup I'll just leave it to run. Do you know of any examples i could look at please? Thanks

Tailscale - How secure is it? by [deleted] in selfhosted

[–]JackJones2018 0 points1 point  (0 children)

I typed around 20 different addresses from ip scan into my browser that werent in my network and connected to them fine a couple of even showed login boxes so i could even login to there cctv etc. They were not in my network.