It’s time to try btrfs (again)

I’m following btrfs for a long time and made my first experiences with it using CoreOS. But they removed it in favour of ext4+overlay in early 2015. Personally I’ve never had a single problem running btrfs but there were lots of not so good articles around this time.

Now some time has passed and some guys told me good things about btrfs. Facebook, Synology and others are using btrfs in production these days. So … time to give it another try.

Why btrfs?

• Ease of use
• Copy on Write
• RW/RO Snapshots
• Bootable subvolumes (snapshots)
• mostly self-healing
• RAID included
• Easy to add/remove devices, online
• Differential Snapshot send: Backups
• And more like compression, checksums and others

@see https://btrfs.wiki.kernel.org

Setup

Considerations

A Snapshot refers to a subvolume and does not include other subvolumes mounted as childs.
The root volume (subvolid=0) is a bit special and does not have a name.
Grub2 can directly boot into subvolumes using the kernel parameter rootflags=subvol=<name>.

This leads to a subvolume mounted as rootfs on / instead of the btrfs top-level volume.
I’ve come to the following structure:

/mnt/btrfs_root           # subvolid=0
/mnt/btrfs_root/rootfs
/mnt/btrfs_root/home
/mnt/btrfs_root/opt

Rule of thumb: always use the latest kernel

Hands on

Assumptions:

• Two HDDs (sda, sdb) same size for a mirroring RAID
• Partitioned sdX1 as SWAP and sdX2 for btrfs

Filesystem setup

Protipp: You can easily copy the GPT to another Disk using sgdisk -R <New_Disk> <Existing_Disk> or for msdos partition tables: sfdisk -d <Existing_Disk> | sfdisk -f <New_Disk>

Protipp: You can convert your existing ext2/3/4 to btrfs using btrfs-convert

# Create the filesystem on sda 
mkfs.btrfs /dev/sda2
# Mount it
mount /dev/sda2 /mnt/btrfs_root
# Add sdb2
btrfs device add /dev/sdb2 /mnt/btrfs_root
# Make it RAID1
btrfs balance start -dconvert=raid1 -mconvert=raid1 /mnt/btrfs_root

Of course this can be achieved by running a single command mkfs.btrfs -m raid1 /dev/sda2 /dev/sdb2 but I want you to show how easy you can add devices and convert your setup to raid1.

btrfs filesystem show /mnt/btrfs_root
# Label: none  uuid: f4479ed6-67a8-4a76-b87e-f8a0c8111d91
#        Total devices 2 FS bytes used 17.81GiB
#        devid    1 size 2.72TiB used 20.03GiB path /dev/sda2
#        devid    2 size 2.72TiB used 20.03GiB path /dev/sdb2

btrfs filesystem df /mnt/btrfs_root
# Data, RAID1: total=19.00GiB, used=17.41GiB
# System, RAID1: total=32.00MiB, used=16.00KiB
# Metadata, RAID1: total=1.00GiB, used=406.47MiB
# GlobalReserve, single: total=41.25MiB, used=0.00B

Looks good, let’s create the subvolumes:

cd /mnt/btrfs_root
btrfs subvolume create rootfs
btrfs subvolume create home
btrfs subvolume create opt
btrfs subvolume list -a .
# ID 1155 gen 32084 top level 5 path rootfs
# ID 1157 gen 32084 top level 5 path opt
# ID 1164 gen 29410 top level 5 path home

Let’s look at the UUIDs:

blkid -s UUID /dev/sda2 /dev/sdb2
# /dev/sda2: UUID="f4479ed6-67a8-4a76-b87e-f8a0c8111d91"
# /dev/sdb2: UUID="f4479ed6-67a8-4a76-b87e-f8a0c8111d91"

You see it’s the same device so it makes no difference if we mount sda2 or sdb2.

OS Setup

Depending on your installer you could now use the subvolumes as mountpoints:

mount -o subvol=rootfs /dev/sda2 /
mount -o subvol=home /dev/sda2 /home

A proper fstab would look like:

/dev/sda1  none swap sw 0 0

# btrfs root volume
UUID=f4...1d91 /mnt/btrfs_root  btrfs  defaults,subvolid=0    0 0
# subvolumes
UUID=f4...1d91 /                btrfs  defaults,subvol=rootfs  0 0
UUID=f4...1d91 /opt             btrfs  defaults,subvol=opt     0 0
UUID=f4...1d91 /home            btrfs  defaults,subvol=home    0 0

Tell Grub2 to boot into rootfs subvolume by adding the kernel parameter.

# rootfs is mounted to / here!
# /etc/default/grub
GRUB_CMDLINE_LINUX="... rootflags=subvol=rootfs"

# Regenerate grub2.conf
grub2-mkconfig -o /boot/grub2/grub.cfg

Assuming your installation crashed - e.g. by performing a dist upgrade - you can now boot into your rootfs-before-upgrade Snapshot.
Hopefully you did btrfs snapshot / /mnt/btrfs_root/rootfs-before-upgrade :)

---

reboot

Now check if your syslog / journal if there are any btrfs errors and validate the mounts:

mount
# ...
btrfs subvolume show /
# ...
btrfs subvolume show /mnt/btrfs_root
# /mnt/btrfs_root is toplevel subvolume

Protipp: You can abbrev most of the btrfs commands like btrfs filesystem => btrfs fi

Protipp: Setup bash autocompletion for btrfs

wget http://www.spinics.net/lists/linux-btrfs/attachments/bin5A44eGuZ7T.bin -O /etc/bash_completion.d/btrfs
bash

btrfs <tab><tab>
# balance           filesystem        property ...

Protipp: df -h is not accurate for btrfs, use btrfs fi df or btrfs fi show

Congratulations to your new btrfs based linux box.
Next time we’ll dig into Snapshots and Backups.