Backing up encrypted and compressed VM snapshot to Azure Cloud

For some time now I was thinking about a good backup solution for our root server. We are using our hard drives in RAID0 mode which means that the two hard drives are not mirrored. Therefore we could use the complete 5TB space. In this scenario, complete data loss is quite likely from time to time, in case one of the two hard drives gets corrupted.
One way to solve this issue is a remote backup of the single vm images. Using LVM it’s possible to take a snapshot from a running virtual machine image. So we can safely compress and transfer the image at a specific state.
Since my home server wouldn’t have enough space to store the backup, I was looking for a cheap “cloud storage”. Besides Amazon AWS there’s also Microsoft Azure. The price per gigabyte is quite good for a low latency and low redundancy option. To register at Azure you’ll need a valid credit card. After that, you can test the service in trail mode for free.

Create backup

Transferring large files to Azure is a bit tricky. I had difficulties using the offical client software called azcopy. I found an other version of this tool, an not yet released preview: azcopy-v10. Using this version, I was able to copy bigger files with 500gb+ successfully. I created an AUR package, so it is easy to install in ArchLinux.
Together with LVM and GnuPG, I combined several commands, so that I could compress, encrypt and transfer the VM snapshot at once in a single step :D Considering the active image you want to backup is called “mail” and resides in a volume group “vg0”. You can create a snapshot with this command:

lvcreate -s -n mail_snap -L 20G /dev/vg0/mail

Install azcopy-v10 and start the transfer:

pacaur -S azcopy-v10
pv -cN source /dev/vg0/mail_snap | gpg --batch --passphrase "my_secret_password" --symmetric --compress-algo zlib | azcopy cp "https://myaccount.blob.core.windows.net/mycontainer/mail_$(date +"%Y-%m-%d").img.gpg?sas"

This is what the command does:

  • With the command pv, we are piping the contents of the snapshot to gpg and we’ll have an additional progress bar in our terminal.
  • GPG is encrypting the snapshot with a specific passphrase, which you’ll have to define. Please note that this usage is considered unsafe because you should never type or provide your passwords in plain text. Please consult the gpg manual on how to setup asymetric encryption for better security. Further, gpg is using zlib to compress the archive.
  • The last part in this chain is azcopy, which will read our encrypted and compressed data stream from STDIN. There you’ll have to define the URL to your storage account on Azure, the destination filename and the one time session key called SAS. This information can be found in the Azure portal where you can create your blob storage account. Be sure to also replace “myaccount” and “mycontainer” in the url with the correct names.
SAS Token inside the Azure portal

After the transfer is complete, you can remove the snapshot from LVM:

lvremote /dev/vg0/mail_snap

Restore backup

To restore a backup, just use azcopy as well:

azcopy cp "https://myaccount.blob.core.windows.net/mycontainer/mail.img.gpg?sas" /mnt/playground.img.gpg
gpg -o /mnt/playground.img -d /mnt/playground.img.gpg

Gpg will ask for the passphrase you specified before.

No Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

* Checkbox GDPR is required

*

I agree

Software
Jellyfin media server on Archlinux ARM

In this post, I want to share some insights on building Jellyfin media server for Archlinux ARM. The PKGBUILD for Jellyfin one can find on the AUR, is specifically made for 64 bit architectures. Nevertheless Microsoft released the dotnet runtime, which Jellyfin relies on instead of Mono, also for Linux …

Software
2
Voice control Archlinux with Amazon Alexa

I was interested to see how commercial voice recognition software would behave on an usual Linux laptop and tried to deploy an Amazon Alexa instance. There are some installation scripts and tutorials available for Ubuntu and Raspberry PI using the avs-device-sdk from Github. Even though some of them are official …

Software
3
Auto update Archlinux and user repository

In this post I’m going to describe on how to setup automatic package upgrades / system updates. In the first part, a systemd service script and timer triggers the package manager pacman to automatically sync the repositories and upgrade the packages every hour: The last command enables (on boot) and …