Backing up encrypted and compressed VM snapshot to Azure Cloud

For some time now I was thinking about a good backup solution for our root server. We are using our hard drives in RAID0 mode which means that the two hard drives are not mirrored. Therefore we could use the complete 5TB space. In this scenario, complete data loss is quite likely from time to time, in case one of the two hard drives gets corrupted.
One way to solve this issue is a remote backup of the single vm images. Using LVM it’s possible to take a snapshot from a running virtual machine image. So we can safely compress and transfer the image at a specific state.
Since my home server wouldn’t have enough space to store the backup, I was looking for a cheap “cloud storage”. Besides Amazon AWS there’s also Microsoft Azure. The price per gigabyte is quite good for a low latency and low redundancy option. To register at Azure you’ll need a valid credit card. After that, you can test the service in trail mode for free.

Create backup

Transferring large files to Azure is a bit tricky. I had difficulties using the offical client software called azcopy. I found an other version of this tool, an not yet released preview: azcopy-v10. Using this version, I was able to copy bigger files with 500gb+ successfully. I created an AUR package, so it is easy to install in ArchLinux.
Together with LVM and GnuPG, I combined several commands, so that I could compress, encrypt and transfer the VM snapshot at once in a single step :D Considering the active image you want to backup is called “mail” and resides in a volume group “vg0”. You can create a snapshot with this command:

lvcreate -s -n mail_snap -L 20G /dev/vg0/mail

Install azcopy-v10 and start the transfer:

pacaur -S azcopy-v10
pv -cN source /dev/vg0/mail_snap | gpg --batch --passphrase "my_secret_password" --symmetric --compress-algo zlib | azcopy cp "https://myaccount.blob.core.windows.net/mycontainer/mail_$(date +"%Y-%m-%d").img.gpg?sas"

This is what the command does:

  • With the command pv, we are piping the contents of the snapshot to gpg and we’ll have an additional progress bar in our terminal.
  • GPG is encrypting the snapshot with a specific passphrase, which you’ll have to define. Please note that this usage is considered unsafe because you should never type or provide your passwords in plain text. Please consult the gpg manual on how to setup asymetric encryption for better security. Further, gpg is using zlib to compress the archive.
  • The last part in this chain is azcopy, which will read our encrypted and compressed data stream from STDIN. There you’ll have to define the URL to your storage account on Azure, the destination filename and the one time session key called SAS. This information can be found in the Azure portal where you can create your blob storage account. Be sure to also replace “myaccount” and “mycontainer” in the url with the correct names.
SAS Token inside the Azure portal

After the transfer is complete, you can remove the snapshot from LVM:

lvremote /dev/vg0/mail_snap

Restore backup

To restore a backup, just use azcopy as well:

azcopy cp "https://myaccount.blob.core.windows.net/mycontainer/mail.img.gpg?sas" /mnt/playground.img.gpg
gpg -o /mnt/playground.img -d /mnt/playground.img.gpg

Gpg will ask for the passphrase you specified before.

No Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

* Checkbox GDPR is required

*

I agree

Software
2
Bye bye Play Store! Mirroring Play Store Apks to private F-Droid repo

This small tutorial will show you how to setup your own F-Droid repository containing automatically mirrored Play Store apps. Using this private repo, you don’t have to relay on Google Play Store or third party stores anymore to keep non-free apps up-to-date. All you need to have is a Linux …

Software
Ebook reader app for Nextcloud

Because of lack of altenatives I decided to write a small ebook reader plugin for Nextcloud. The task of creating such an app was quite easy: I just forked the files_pdfviewer extension and replaced PDF.js with the Epub.js library. In the app template file, I used the reference ebook reader …

Software
Hacking replay gain audio normalization into Jellyfin

There is already a feature request for audio normalization in Jellyfin media server. This is important if you want to listen to your music collection while always having the same loudness level. Usually, in different recordings or music genres some tracks are louder and others are more quiet. The standard …