This tutorial describes how to do scripted passwordless encrypted automatic rsync backups from an Ubuntu server to a Synology NAS. It is supposed to be a general guide, which can be used with other Linux variants as well. Only minor changes should be required.

I did this first of all in order to help other Synology NAS users perform efficient and secure backups to their NAS, but also to get get my scratch pad like personal installation notes in a more reasonable shape, so I can recall later on myself what I did, how, and why.

1. Setup prerequisites for rsync on source and destination host
2. Create home directory for DSM user (to hold .ssh directory)
3. Setup ssh and avoid password entry
4. Create a script and use cron to automate execution
5. Use another module than the default “NetBackup”

For this tutorial, the following hardware/software was used:

Source host:
OS : ubuntu-16.04-server-amd64.iso running in a VM.
ip : 192.168.1.167 (not relevant for this tutorial)
hostname : vmtest
user : geohei

Destination host:
hardware : DS1817+
OS : DSM 6.1.3-15152 Update 3 running in a VM (Virtual Machine Manager)
ip : 192.168.1.174
hostname : vm-neelix
user : janeway

The following targets should be achieved:

• rsync from client (Ubuntu server) to server (NAS)
• encrypted and passwordless rsync data transfer
• automatic script execution using cron
• script should catch previous unterminated script execution and avoid starting it again

 

1. Setup prerequisites for rsync on source and destination host

Destination host DSM rsync service needs to be enabled on destination host:
DSM : File Services > rsync > Enable rsync service : enable
This automatically creates the shared folder “NetBackup”.

rsync data transfer should ideally be accomplished to a destination user having minimal permissions. Unfortunately however, this user must be a member of the “administrators” group, since rsync requires shell login at the destination host. Only users of the “administrators” group generate the /etc/passwd field 7 (last field) /bin/sh. Users not being member of “administrators” group have /sbin/nologin, which doesn’t allow (rsync) logins. Basically, the user created during initial DSM setup is just fine. He has all the necessary permissions required for rsync and belongs to the “administrators” group.

Perform a first manual rsync from source geohei@vmtest to destination janeway@vm-neelix. Note that destination directory “rsyncfolder” is created by rsync if necessary (only one directory will be created, not multiple).

geohei@vmtest:~$ rsync -rlptogRv -e ssh "/home/geohei/data" janeway@192.168.1.174::NetBackup/rsyncfolder
The authenticity of host '192.168.1.174 (192.168.1.174)' can't be established.
ECDSA key fingerprint is SHA256:###########################################.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.174' (ECDSA) to the list of known hosts.
janeway@192.168.1.174's password:
Could not chdir to home directory /var/services/homes/janeway: No such file or directory
sending incremental file list
created directory rsyncfolder
/home/
/home/geohei/
/home/geohei/data/
/home/geohei/data/bar_file_1.txt
/home/geohei/data/foo_directory/
/home/geohei/data/foo_directory/bar_file_2.txt

sent 298 bytes  received 113 bytes  18.27 bytes/sec
total size is 0  speedup is 0.00

Transferred data of source host (uid:gid shown as numeric IDs):

geohei@vmtest:~$ ls -ldn /home/
drwxr-xr-x 3 0 0 4096 Dec 31  2016 /home/
geohei@vmtest:~$ ls -ln /home/
total 4
drwxr-xr-x 6 1000 1000 4096 Aug 27 19:38 geohei
geohei@vmtest:~$ ls -ln /home/geohei/
total 4
drwxrwxr-x 3 1000 1000 4096 Aug 27 19:32 data
geohei@vmtest:~$ ls -ln /home/geohei/data/
total 4
-rw-rw-r-- 1 1000 1000    0 Aug 21 20:41 bar_file_1.txt
drwxrwxr-x 2 1000 1000 4096 Aug 22 12:47 foo_directory
geohei@vmtest:~$ ls -ln /home/geohei/data/foo_directory/
total 0
-rw-rw-r-- 1 1000 1000 0 Aug 22 12:47 bar_file_2.txt

Transferred data of source host (uid:gid shown as numeric IDs):

janeway@vm-neelix:~$ ls -ln /volume1/NetBackup/rsyncfolder/
total 0
drwxr-xr-x 1 0 0 12 Dec 31  2016 home
janeway@vm-neelix:~$ ls -ln /volume1/NetBackup/rsyncfolder/home/
total 0
drwxr-xr-x 1 1000 1000 216 Aug 27 19:38 geohei
janeway@vm-neelix:~$ ls -ln /volume1/NetBackup/rsyncfolder/home/geohei/
total 0
drwxrwxr-x 1 1000 1000 54 Aug 27 19:32 data
janeway@vm-neelix:~$ ls -ln /volume1/NetBackup/rsyncfolder/home/geohei/data/
total 0
-rw-rw-r-- 1 1000 1000  0 Aug 21 20:41 bar_file_1.txt
drwxrwxr-x 1 1000 1000 28 Aug 22 12:47 foo_directory
janeway@vm-neelix:~$ ls -ln /volume1/NetBackup/rsyncfolder/home/geohei/data/foo_directory/
total 0
-rw-rw-r-- 1 1000 1000 0 Aug 22 12:47 bar_file_2.txt

rsync preserves data permissions and uid:gid at the destination host. However this only occurs if source rsync command includes -og options, and if the destination rsync daemon is being run as super-user. See rsync(1) and rsyncd.conf(5) for details. On top, the destination host needs to be addresses by destination module name ::NetBackup, as opposed to a destination path (/volume1/NetBackup). In the latter case, the uid:gid would be inherited from the user used by the rsync command (janeway:users in the example above).

To make the mess complete, Synology customized its rsync binary. At this point it really (!) gets complicated, especially because there is barely any information available about the changed code. If interested in details, please have a look at this.

 

2. Create home directory for DSM user (to hold .ssh directory)

rsync prints the message (no error, just an information) after the password was entered: "Could not chdir to home directory /var/services/homes/janeway: No such file or directory". This is normal since DSM doesn’t create a home directory when adding new users (--no-create-home). The same message shows when logging into shell using any DSM user. If desired (not required for now!), create the home directory. First, sudo to user root. The sudo password is the one of user janeway. Then create the home directory for user janeway and finally the required symlink.

janeway@vm-neelix:/$ sudo -i
Password:
root@vm-neelix:~# mkdir /home
root@vm-neelix:~# mkdir /home/janeway
root@vm-neelix:~# chown janeway:users /home/janeway/
root@vm-neelix:~# mkdir /var/services/homes
root@vm-neelix:~# ln -s /home/janeway /var/services/homes/janeway

ssh and rsync logins should not complain anymore about a missing home directory.

While until now, the generation of a home directory for user janeway was cosmetic, it becomes a requirement for chapter 3 below.

 

3. Setup ssh and avoid password entry

First, generate an RSA 2048 bit key for user geohei@vmtest using ssh-keygen. Accept default file name suggestion for id_rsa. Leave the passphrase empty (you will be asked twice). Below, I used ssh-keygen -t rsa -b 2048, but the -t and -b options are defaults, hence ssh-keygen would have been sufficient.

geohei@vmtest:~$ ssh-keygen -t rsa -b 2048
Generating public/private rsa key pair.
Enter file in which to save the key (/home/geohei/.ssh/id_rsa):
Created directory '/home/geohei/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/geohei/.ssh/id_rsa.
Your public key has been saved in /home/geohei/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:########################################### geohei@vmtest
The key's randomart image is:
+---[RSA 2048]----+
|   E      . Eo   |
|        +  =.o.  |
|   =      o=+.   |
| o.   .  ..o*    |
|     o +S.o+o.   |
|    . =o=.o*+    |
|  X  =.+. += o   |
|    o oo. X+o    |
|   .  ...*o+.    |
+----[SHA256]-----+

Next, you need to copy the just generated id_rsa.pub from the source host to the destination host. Now we need the home directory of the destination host user (janeway) since the content of id_rsa.pub is stored in this directory (/home/janeway/.ssh/authorized_keys). The command ssh-copy-id should be used to transfer the key. This command requires login at the destination host. That’s another reason (besides the rsync ccommand) why user janeway needs to be part of the DSM “administrators” group (see above).

geohei@vmtest:~$ ssh-copy-id -i /home/geohei/.ssh/id_rsa.pub janeway@192.168.1.174
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/geohei/.ssh/id_rsa.pub"
The authenticity of host '192.168.1.174 (192.168.1.174)' can't be established.
ECDSA key fingerprint is SHA256:###########################################.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
janeway@192.168.1.174's password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'janeway@192.168.1.174'"
and check to make sure that only the key(s) you wanted were added.

From now on, geohei@vmtest can login (ssh, rsync, …) to janeway@vm-neelix without password entry. This also permits scripted rsync as shown below.

The bottom line with the RSA key exchange can be summarized as follows. The user, invoking the ssh transfer (geohei@vmtest) must generate the RSA key. This RSA key must be transferred to the home directory of the destination user@host (janeway@vm-neelix).

If for some reason, ssh-copy-id cannot be used, then copy the RSA key manually from source host /home/geohei/.ssh/id_rsa.pub to destination host /home/janeway/.ssh/authorized_keys. A situation where this is required would be for example a destination user which doesn’t have a password (as it is for DSM user root as seen in chapter 5 below), as ssh-copy-id requires entering destination user password.

 

4. Create a script and use cron to automate execution

The following script will automate the rsync process.

geohei@vmtest:~$ cat backup.sh
#!/bin/bash

server='192.168.1.174'

script=$(basename $0)
lock="/tmp/$script.pid"
pid=$$

echo $lock

# open $lock for writing on file descriptor 200
exec 200> $lock
# copy write file descriptor 1 (stdout, which is pid in this case) to file descriptor 200
echo $pid 1>&200

echo "INFO : rsync started."
if flock -n -x 200; then
  # if $script is not already active
  rsync -rlptRv -e ssh "/home/geohei/data" janeway@192.168.1.174::NetBackup/rsyncfolder
  echo "INFO : rsync finished."
sleep 10
  rc=0
else
  # if $script is active
  echo "ERROR : rsync not possible since $script is still busy !!!"
  rc=1
fi
rm $lock
echo "INFO : rsync finished."
exit $rc

geohei@vmtest:~$ ls -l backup.sh
-rwxr-xr-x 1 geohei geohei 659 Aug 26 00:57 backup.sh

Make sure that permissions are set to 755.

The script includes comments, which explain the basic operation. It works with file descriptors, which permit to check if the script is still active or not. This design avoids running the script twice, possibly causing mutual interence.

cron takes care of daily trigger. I run it as user geohei, but it could also be user root. Just make sure the RSA keys of the user invoking cron, are transferred to the destination host@user’s home directory.

0 3 * * * ~/backup.sh

 

5. Use another module than the default “NetBackup”

First of all, this chapter was added after the great (!) help of user “nixjps” in the Synology forum. It is based on this thread “rsync issues and questions” and greatly simplified below. If you really want to explore it … get yourself some coffee!

Ok … let’s say you don’t like the default folder “NetBackup” (which is automatically created when the DSM rsync service is enabled) as rsync directory, and you prefer something like “rsync Backups”. The first thing you need to know is that the shared folder “NetBackup” is hardcoded into the customized Synology rsync binary. It is not just another rsync module. It has a special status. I didn’t dig through the (outdated 3.0.9) rsync source code, but it can apparently be found at SourceForge.

My rsync objective is to keep all file/directory permissions as well uid:gid throughout the entirely copied filesystem tree. Using the DSM rsync version, this can only be done on the client side with the rsync -og (tells the rsync server to keep uid:gid) and --numeric-ids (avoids name mapping) options, while on the server side, rsync must be started by a user of the “administrators” group in daemon mode (which occurs when a destination module name is used on the client side, as opposed to a destination path).

The only possibility to make DSM rsync server behave like for the “NetBackup” folder, is to create a desired new module and to use user root as login from the client side to the DSM rsync server.

The original /etc/rsyncd.conf looks like this:

#motd file = /etc/rsyncd.motd
#log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
use chroot = no

The modified version would be something like that:

#motd file = /etc/rsyncd.motd
#log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
use chroot = no

[rsyncB]
path = /volume1/rsync Backup
comment = module for rsync backups
read only = false
uid = root

The above assumes of course that the shared folder “rsync Backup” exists on the DSM.

Restart of the rsyncd service is not required since /etc/rsyncd.conf is re-read when newly invoked.

Listing all (listable) rsync destination modules and paths from the client looks like this:

geohei@vmtest:~$ rsync 192.168.1.174::
rsyncB          module for rsync backups
NetBackup       System default shared folder
rsync Backup
home            ~

If passwordless access is desired (required for script operation – see above), then RSA key from geohei@vmtest also needs to be copied to the directory of root@vm-neelix. This works like explained here.

Be aware that DSM user root has per default no password. The command synouser --setpw root [root_pwd] would generate a password, but this is not recommended. Nevertheless, if done, delete it later on again out of /etc/shadow.

The better solution is to manually copy the RSA key. Below shows permissions, uid:gid and content (one RSA key per line) of /root/.ssh/authorized_keys.

janeway@vm-neelix:/$ sudo -i
Password:
root@vm-neelix:~# ls -ld .ssh/
drwx------ 2 root root 4096 Sep  3 17:21 .ssh/
root@vm-neelix:~# ls -l .ssh/
#total 4
-rw------- 1 root root 396 Sep  3 17:13 authorized_keys
root@vm-neelix:~# cat .ssh/authorized_keys
ssh-rsa ... geohei@vmtest

Now, there are 2 possibilities to invoke rsync on the client side:

geohei@vmtest:~$ rsync -rlptogvDR -e "ssh -l root" --numeric-ids "/home/geohei/" janeway@192.168.1.174::rsyncB

geohei@vmtest:~$ rsync -rlptogvDR -e ssh --numeric-ids "/home/geohei/" root@192.168.1.174::rsyncB

Both seem to produce identical results in all respects.

Like that, shared folder “rsync Backup” can be used exactly the same way as “NetBackup”, hence cirumnavigating the Synology rsync “NetBackup” folder restrictions.