Building a Ubuntu Server - Part 4 - Multi-part Series

After struggling over the week-end with my new Ubuntu Server (Xenial Xerus), I have come up with the below solution to satisfy my setup requirements.

First off, let me describe what I was attempting to do. The system has 3 hot swap drives, each at 500G. The system came pre-installed and I wanted to start from scratch as Xenial Xerus had just been released and the system from Zareason had the previous release Wily Werewolf installed.

For the setup I wanted to do a RAID 1 (mirror) with a spare of the base system. I also wanted to setup LVM and encryption. All available via the installer.

So, first instinct was to load the installer, flatten the drives and install from USB. No luck. GPT Woes and failed to install grub, etc.

I tried many times, and if I removed LVM and encryption, and just went RAID, it worked fine. Testing the same setup with no encryption yielded a working system as well. It turns out the installer lets you encrypt the entire partition and then install the /boot partition within the encrypted drive.

I now realize that while this was allowed in the installer, this wasn't going to work. The /boot partition needs to be accessible non-encrypted in order for the installer to load grub, etc.

Now that I knew this, I was able to plan things out with the required EFI and /boot partitions separate from my encrypted RAID/LVM partition for /.

First, since the drives have existing partitions, I needed to clear those out. So, from a Live CD I opened a terminal and executed the following to remove the existing partition tables. This step shouldn't be necessary if you have brand new and clean drives to start.

sudo sgdisk -Z /dev/sda
sudo sgdisk -Z /dev/sdb
sudo sgdisk -Z /dev/sdc

After I reset the tables, I created my partitions on first drive, also from the Live CD. This just made it easier to setup the drives ahead of time and let the installer see those partitions later.

sudo sgdisk -n 1:0:+512M -t 1:ef00 -c 1:"EFI System" /dev/sda 
sudo sgdisk -n 2:0:+512M -t 2:fd00 -c 2:"Linux RAID" /dev/sda
sudo sgdisk -n 3:0:+64G -t 3:fd00 -c 3:"Linux RAID" /dev/sda

The above will create 3 partitions:

• 512MB EFI
• 512MB RAID
• 64GB RAID

Next, I copied the structure over to the second drive (mirror) and third drive (spare). The below command will randomize the UUID as well on the copy.

sudo sgdisk /dev/sda -R /dev/sdb -G
sudo sgdisk /dev/sda -R /dev/sdc -G

The first partition of each drive sd[abc]1 will be used for hosting the EFI mount point. This cannot be RAIDed and will need to be cloned (dd) to the other disks and added to the boot chain (see further down).

The second partition will be for my boot partition, and will be a plain RAID 1 mirror on sd[ab]2 with sdc2 being a spare. No encryption and no LVM.

The 3rd and final partition will be for the rest of the system and will be installed in LVM on top of an encrypted RAID 1 array sd[ab]3 with spare sdc3.

Now we need to prepare the EFI partition to ensure the installer can see and use it.

sudo mkfs.fat -F 32 /dev/sda1
mkdir /tmp/sda1
sudo mount /dev/sda1 /tmp/sda1
sudo mkdir /tmp/sda1/EFI
sudo umount /dev/sda1

At this point I was able to safely hop back into the server installer and configure the partitions.

I chose to use the manual partitioning so i could select the EFI partitions on each drive and set them as ESP, build both RAID1 sets (sd[ab]1+sdc1 (spare) and sd[ab]2+sdc2 (spare)), create the encrypted volume on MD1 (64GB RAID1 from partition 2), then add LVM on the encrypted volume as one volume group with two logival volumes (17GB swap and 25GB for /).

With this setup, the installer was finally happy with me, and I was able to complete my install. All that was left at this point was to mirror the EFI partition and update the boot chain.
To do this, I rebooted into my new server and logged in tomy admin account and executed the following:

dd if=/dev/sda1 of=/dev/sdb1
dd if=/dev/sda1 of=/dev/sdc1
efibootmgr -c -g -d /dev/sdb -p 1 -L "Ubuntu (Mirror)" \
 -l '\EFI\Ubuntu\grubx64.efi'
efibootmgr -c -g -d /dev/sdc -p 1 -L "Ubuntu (Spare)" \
-l '\EFI\Ubuntu\grubx64.efi'

All that was left was to test the drives, rebuilding the array from simulated failure, etc. But Ill leave that for a future discussion.

Now, off to secure my server, configure my UPS, etc.

Reference article: how to install ubuntu 14.04 64 bit with a dual boot raid 1 partition on an uefi

Building a Ubuntu Server - Part 3
Building a Ubuntu Server - Part 5