hoarding.me

rclone, scripts and automation with crontab.

Automating rclone is what really makes this setup special. We’re going to be creating scripts to automate uploading and remounting.


cd /home/scripts && mkdir logs

The logs folder will be for tracking the scripts when they run.
Lets make the scripts themselves.


touch mount-m.cron mount-tv.cron upload-m.cron upload-tv.cron

And lets make our mountchecks. These are the files our scripts will be looking for when checking the mount is still active. We’ll upload these in a few minutes to test.


touch /home/plex/tv-r/mountcheck /home/plex/m-r/mountcheck

The mountchecks will be automatically encrypted and uploaded when you setup the upload crons, it’s very important you double check and ensure these are in your mounts as otherwise the mountcheck/re-mount won’t work.

Now for the scripts themselves, I’ve used the scripts created by ajkis with some minor modifications to suit our needs.

NOTE: To set these up without enabling automation immediately, comment the cron entries before exiting out, otherwise they’ll immediately start uploading the contents of their respective folders. This is also following on from the bottom of the previous page, remember, these mount scripts are mounting from /mnt/plexdrive/yourencryptedfolder to /home/plex/tv-gd (or /home/plex/m-gd).

Mounting the movies folder.

nano mount-m.cron

#!/bin/bash
# Change paths for mount and log file & create mountcheck file.
# Add script to crontab by typing crontab -e and pasting the line below (without # in front )
# * * * * * /home/scripts/mount-m.cron >/dev/null 2>&1
# Make script executable with: chmod a+x /home/scripts/mount-m.cron

logfile="/home/scripts/logs/mount-m.cron.log"

if pidof -o %PPID -x "mount-m.cron"; then
echo "$(date "+%d.%m.%Y %T") EXIT: mount-m.cron already running."
exit 1
fi

if [[ -f "/home/plex/m-gd/mountcheck" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: Check successful, /home/plex/m-gd mounted." | tee -a "$logfile"
exit
else
echo "$(date "+%d.%m.%Y %T") ERROR: Drive not mounted, remount in progress." | tee -a "$logfile"
# Unmount before remounting
fusermount -uz /home/plex/m-gd | tee -a "$logfile"
rclone mount \
        --read-only \
        --allow-non-empty \
        --allow-other \
        --max-read-ahead 2G \
        --acd-templink-threshold 0 \
        --checkers 16 \
        --quiet \
        --stats 0 \
dec-m:/ /home/plex/m-gd/&

if [[ -f "/home/plex/m-gd/mountcheck" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: Remount successful." | tee -a "$logfile"
else
echo "$(date "+%d.%m.%Y %T") CRITICAL: Remount failed." | tee -a "$logfile"
fi
fi
exit

What this script will do is check for the mountfile within your mountpoint (/home/plex/m-gd) and if it isn’t present, remount the drive and then write it’s output to the logfile. The max-read-ahead can be changed to your own preferences.

You can do the same process with the mount-tv.cron script.

Mounting the TV Folder.

#!/bin/bash
# Change paths for mount and log file & create mountcheck file.
# Add script to crontab by typing crontab -e and pasting the line below (without # in front )
# * * * * * /home/scripts/mount-tv.cron >/dev/null 2>&1
# Make script executable with: chmod a+x /home/scripts/mount-tv.cron

logfile="/home/scripts/logs/mount-tv.cron.log"

if pidof -o %PPID -x "mount-tv.cron"; then
echo "$(date "+%d.%m.%Y %T") EXIT: mount-tv.cron already running."
exit 1
fi

if [[ -f "/home/plex/tv-gd/mountcheck" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: Check successful, /home/plex/tv-gd mounted." | tee -a "$logfile"
exit
else
echo "$(date "+%d.%m.%Y %T") ERROR: Drive not mounted, remount in progress." | tee -a "$logfile"
# Unmount before remounting
fusermount -uz /home/plex/tv-gd | tee -a "$logfile"
rclone mount \
        --read-only \
        --allow-non-empty \
        --allow-other \
        --max-read-ahead 2G \
        --acd-templink-threshold 0 \
        --checkers 16 \
        --quiet \
        --stats 0 \
dec-tv:/ /home/plex/tv-gd/&

if [[ -f "/home/plex/tv-gd/mountcheck" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: Remount successful." | tee -a "$logfile"
else
echo "$(date "+%d.%m.%Y %T") CRITICAL: Remount failed." | tee -a "$logfile"
fi
fi
exit

Creating the movie upload script.

The upload scripts are done the same way. Here is the upload-m.cron. Note: If you run into bandwidth issues you can limit the uploads via adding this to the end of the line ‘rclone move etc’--bwlimit 25M (or whatever speed you wish).

#!/bin/bash
# RCLONE UPLOAD CRON TAB SCRIPT
# Type crontab -e and add line below (without # )
# * * * * * root /home/scripts/upload-m.cron >/dev/null 2>&1

if pidof -o %PPID -x "upload-m.cron"; then
exit 1
fi

LOGFILE="/home/scripts/logs/upload-m.cron.log"
FROM="/home/plex/m-r/"
TO="m-gd:/"

# CHECK FOR FILES IN FROM FOLDER THAT ARE OLDER THEN 15 MINUTES
if find $FROM* -type f -mmin +5 | read
then
echo "$(date "+%d.%m.%Y %T") RCLONE UPLOAD STARTED" | tee -a $LOGFILE
# MOVE FILES OLDER THEN 5 MINUTES
rclone move $FROM $TO -c --no-traverse --transfers=300 --checkers=300 --delete-after --min-age 5m --log-file=$LOGFILE
echo "$(date "+%d.%m.%Y %T") RCLONE UPLOAD ENDED" | tee -a $LOGFILE
fi
exit

Creating the TV upload script.

And here is the upload-tv.cron

#!/bin/bash
# RCLONE UPLOAD CRON TAB SCRIPT
# Type crontab -e and add line below (without # )
# * * * * * root /home/scripts/upload-tv.cron >/dev/null 2>&1

if pidof -o %PPID -x "upload-tv.cron"; then
exit 1
fi

LOGFILE="/home/scripts/logs/upload-tv.cron.log"
FROM="/home/plex/tv-r/"
TO="tv-gd:/"

# CHECK FOR FILES IN FROM FOLDER THAT ARE OLDER THEN 15 MINUTES
if find $FROM* -type f -mmin +15 | read
then
echo "$(date "+%d.%m.%Y %T") RCLONE UPLOAD STARTED" | tee -a $LOGFILE
# MOVE FILES OLDER THEN 15 MINUTES
rclone move $FROM $TO -c --no-traverse --transfers=300 --checkers=300 --delete-after --min-age 15m --log-file=$LOGFILE
echo "$(date "+%d.%m.%Y %T") RCLONE UPLOAD ENDED" | tee -a $LOGFILE
fi
exit

Automating the scripts

To add these to crontab do this;

crontab -e

Add all four entries

 

* * * * * /home/scripts/mount-m.cron >/dev/null 2>&1
* * * * * /home/scripts/mount-tv.cron >/dev/null 2>&1
* * * * * /home/scripts/upload-m.cron >/dev/null 2>&1
* * * * * /home/scripts/upload-tv.cron >/dev/null 2>&1

 

Now lets make them all executable.

chmod a+x mount-m.cron mount-tv.cron upload-m.cron upload-tv.cron

Now here’s what happens.

Ombi
>Couchpotato
>Sonarr

While you can add tv/movies via Sonarr/Radarr/Couchpotato directly, let’s assume everything is being added by Ombi for now and we’ll use Radarr as an example.

A movie gets requested from Ombi and gets sent to Radarr which then gets automatically added to rtorrent. When it’s finished downloading Radarr will process it into the folder /home/plex/m-r/. Every minute our crontab will search this folder for files that are more than 5 minutes old and if it finds anything that satisfies this, will upload it via your encrypted remote to GD. From here plexdrive mounts these to /mnt/plexdrive/ and rclone then mounts them again into /home/plex. It’s these two folders then that Plex reads and indexes. Plex can also automatically scan your library and will add new media it finds when it searches.

What about the files Sonarr/Radarr downloads? They need to be able to tell what previous episodes were downloaded so it doesn’t grab the same content when it refreshes its library. Because our rclone script moves the files from /home/plex/tv-r we need to use a different workaround. Hence why I specified the home for TV within sonarr to be /home/plex/fuse-tv. To create a fuse of both /home/plex/tv-gd (already uploaded and archived TV) with /home/plex/tv-r (downloaded but yet to be archived TV);
 

First install fuse apt-get install fuse and unionfs apt-get install unionfs-fuse

 

Creating the TV-Fuse mount.

The fuse tv script is done the same way as the others, here is the fuse-mount-tv.cron.

#!/bin/bash
# Change paths for mount and log file & create mountcheck file.
# Add script to crontab by typing crontab -e and pasting the line below (without # in front )
# * * * * * /home/scripts/fuse-mount.cron >/dev/null 2>&1
# Make script executable with: chmod a+x /home/scripts/fuse-mount.cron

logfile="/home/scripts/logs/fuse-mount-tv.cron.log"

if pidof -o %PPID -x "fuse-mount.cron"; then
echo "$(date "+%d.%m.%Y %T") EXIT: fuse-mount.cron already running."
exit 1
fi

if [[ -f "/home/plex/fuse-tv/mountcheck" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: Check successful, fuse mounted." | tee -a "$logfile"
exit
else
echo "$(date "+%d.%m.%Y %T") ERROR: Drive not mounted, remount in progress." | tee -a "$logfile"
# Unmount before remounting
fusermount -uz /home/plex/fuse-tv | tee -a "$logfile"
/usr/bin/unionfs-fuse -o cow,allow_other /home/plex/tv-r=RW:/home/plex/tv-gd=RO /home/plex/fuse-tv

if [[ -f "/home/plex/fuse-tv/mountcheck" ]]; then
echo "$(date "+%d.%m.%Y %T") INFO: Remount successful." | tee -a "$logfile"
else
echo "$(date "+%d.%m.%Y %T") CRITICAL: Remount failed." | tee -a "$logfile"
fi
fi
exit

Same thing in cron, crontab -e & * * * * * /home/scripts/fuse-mount-tv.cron >/dev/null 2>&1 Don’t forget to also chmod a+x fuse-mount.cron

This fuse-mount means that when tv-gd does unmount (which it inevitably will), your fuse mount at /home/plex/fuse-tv will also get remounted and fixed. Note: user_allow_other in /etc/fuse.conf needs to be uncommented for fuse to work without IF you’re not running these as root.

Creating the Movies-Fuse mount.

And here is the radarr fuse script.


#!/bin/bash
# Change paths for mount and log file & create mountcheck file.
# Add script to crontab by typing crontab -e and pasting the line below (without # in front )
# * * * * *  /home/scripts/fuse-mount-m.cron >/dev/null 2>&1
# Make script executable with: chmod a+x /home/scripts/fuse-mount.cron

logfile="/home/scripts/logs/fuse-mount-m.cron.log"

if pidof -o %PPID -x "fuse-mount-m.cron"; then
   echo "$(date "+%d.%m.%Y %T") EXIT: fuse-mount-m.cron already running."
   exit 1
fi

if [[ -f "/home/plex/fuse-m/mountcheck" ]]; then
   echo "$(date "+%d.%m.%Y %T") INFO: Check successful, /home/plex/fuse-m mounted." | tee -a "$logfile"
   exit
else
   echo "$(date "+%d.%m.%Y %T") ERROR: Drive not mounted, remount in progress." | tee -a "$logfile"
   # Unmount before remounting
   fusermount -uz /home/plex/fuse-m | tee -a "$logfile"
   /usr/bin/unionfs-fuse -o cow,allow_other /home/plex/m-r=RW:/home/plex/m-gd=RO /home/plex/fuse-m

   if [[ -f "/home/plex/fuse-m/mountcheck" ]]; then
      echo "$(date "+%d.%m.%Y %T") INFO: Remount successful." | tee -a "$logfile"
   else
      echo "$(date "+%d.%m.%Y %T") CRITICAL: Remount failed." | tee -a "$logfile"
   fi
fi
exit

Same thing in cron, crontab -e & * * * * * /home/scripts/fuse-mount-m.cron >/dev/null 2>&1 Don’t forget to also chmod a+x fuse-mount-m.cron

That’s everything. When you do need to restart the server you’ll need to re-mount the mounts, for now you can do it the dirty way via invoking the scripts directly. cd /home/scripts and ./mountplex.sh, ./mount-tv.cron & ./mount-m.cron

I’ve tested out with a few different @reboot crons but never found something that worked correctly. I’ll update this guide if I do. In the meantime, enjoy your new unlimited plex server!

That’s pretty much everything, if you’ve any feedback or issues you can reach out via /r/hoardingme!

[Homepage, Page Index.]