Encrypted, streaming backups with tar and gpg with fifos
I wanted to back up a server to a USB drive earlier.
I wanted the backup to be encrypted and didn’t want to waste any time writing an unencrypted tarball to file and then encrypt it. GPG is really good for encrypting files so initially I tried its symmetric mode (tar -c / | gpg –symmetric -o backup.tar.gpg) but it seems this mode can’t accept streaming input (it takes the key on standard in and I can’t find a way to have it read the key from somewhere else).
I realised (too late for myself, unfortunately) that this is a perfect time to use a FIFO! I was excited for FIFOs when I first discovered them in school but have never really had a use for them myself.
A FIFO is a special file which allows a single person to open it for write and a single person to open it for read & pipes the data straight from the writer to the reader.
- First, create a FIFO file.
mkfifo /mnt/backup.fifo
- Now start the writing end of the fifo (in the background). This is creating the archive.
tar -zcvf /mnt/backup.fifo --exclude=/proc --exclude=/mnt --exclude=/sys / &
- Now start the reading end of the fifo. This is doing the encryption.
gpg --symmetric /mnt/backup.fifo -o /mnt/usb/backup.tar.gz.gpg