Booting FreeBSD from a root zfs pool using a standard MBR and partition table
Posted by Benjamin Close on May 8, 2009 under FreeBSD | 
5 Comments to Read
This page documents the current state of play for booting the root filesystem (/) off a zfs zpool under FreeBSD, using a standard master boot record (MBR) and a standard partition table. The aim was to be able to have a dual boot system for my laptop using the standard FreeBSD quick selection boot loader.
Note, this does not cover using the gpt based partition tables. If you want to use these, please refer to the following page: http://lulf.geeknest.org/blog/freebsd/Setting_up_a_zfs-only_system/ or booting zfs as root using a small ufs boot partition as provided by the instructions at: http://wiki.freebsd.org/ZFSOnRoot.
Below is the steps required to be able to setup the root zpool
- Download a FreeBSD -current fixit cdrom snapshot later than 200901, as these have loader ZFS support
- Burn the CD
- Boot the CD
- Setup any partitions you want – note you must setup the ‘a’ partition to cover the entire device as the loader will use this.
- Select Fixit from the menu, and use the CDrom as a source
- Create the pool and install the loader (see below)
- Copy the required files to boot (see below)
Creating the root zpool and installing the Loader
The fixit cd has everything required to create a zpool, however by default none of the required modules are loaded. Hence they need to be loaded first:
cd /mnt2/boot/kernel
kldload ./opensolaris.ko
kldload ./zfs.ko
Once the modules have been loaded all the zfs tools (zpool,zfs,zdb) should now work. Let assume you want to install FreeBSD to /dev/ad4s2 (second partition on a sata disk). You can do this using:
zpool create somename /dev/ad4s2
Where somename is the name of the pool you want to create. This creates a single zfs filesystem and a zfs pool of storage. To install the boot loader you need to do:
# dd if=/mnt2/boot/zfsboot of=/dev/da0s1 count=1
# dd if=/mnt2/boot/zfsboot of=/dev/da0s1 skip=1 seek=1024
The first line installs boot1, the second line installs boot2. However, boot2 is responsible for loading boot3 (aka the loader – found in /boot/loader). Hence that must be put in place.
Copying the required files to boot
The easiest way to get things to the point where things are ready to boot is to copy all the files from /dist – the live distribution. Before you do this, you might like to take advantage of zfs and create some subfilesystems so you can snapshot, monitor space, etc.
For instance creating a /usr and /var filesystem is often very handy:
#zfs create somename/usr
#zfs create somename/var
Now you can copy the base system:
cp -a /dist/* /somename
This will install among other things:
- /somename/boot/kernel/kernel – FreeBSD kernel
- /somename/boot/kernel/opensolaris.ko – zfs dependency
- /somename/boot/kernel/zfs.ko – module understanding zpools/zfs
- /somename/boot/loader – the FreeBSD loader
At this point you need to replace the loader with one that understands zfs. You can download the loader from: (To be advise – see cavet below)
And if you have a usb stick copy it in place using:
mount_msdos /dev/da0s1 /mnt cp /mnt/loader /somename/boot
Finally you have to tell FreeBSD where to mount filesystems on a standard boot:
zfs set mountpoint=/var somename/var zfs set mountpoint=/usr somename/usr
At this point any command you type will now probably fail indicating it’s missing some shared library. This is because /usr has now changed. You can get around this by telling the loader where to find valid libraries:
export LD_LIBRARY_PATH=/mnt2/lib
Next we need to build the zfs cache. This is used by zfs mount to automatically mount zfs filesystems by /etc/rc.d/zfs at boot time. It’s also used to determine if a filesystem is local to the system or belongs to an exported pool.
mkdir /boot/zfs mkdir /somename/boot/zfs cd / zfs export somename zfs import -f somename cp /boot/zfs/zfs.cache /somename/boot/zfs/
Finally we tell the loader where we want to boot from and set the init scripts to automatically start all zfs filesystems:
echo 'zfs_enable="YES"' > /somename/etc/rc.conf
echo 'zfs_load="YES"' > /somename/boot/loader.conf
echo 'vfs.root.mountfrom="zfs:somename"' >> /somename/boot/loader.conf
And set the root filesystem to a legacy mountpoint (so zfs mount -a won’t try and mount an already mounted filesystem)
zfs set mountpoint=legacy somename
At this point you can reboot and things should now boot!
Update: 20090809
There has been a lot of updates to instructions along the way. There are now official ZFS on Root instructions available using GPT/MBR/other available at: http://wiki.freebsd.org/RootOnZFS
dante said,
you forgot to put a “kldload ./geom_mbr.ko” at the beginning. Without it you can’t read mbr partition table !
Henri Hennebert said,
Now your problem is solved.
With 7.2-STABLE after r193696 with a patch on /usr/src/lib/libstand/nfs.c (see post http://docs.freebsd.org/cgi/getmsg.cgi?fetch=589861+0+archive/2009/freebsd-stable/20090607.freebsd-stable – note the patch of SOPEN_MAX is now in stable).
I put my pool on ad0s2 and set this partition active.
The boot run smoothly ONLY IF the partition type (sysid) is NOT 0xa5 (FreeBSD/NetBSD/386BSD). I use fdisk to set sysid to 0x82 (Linux swap or Solaris x86) just to make a tribute to sun!
If the sysid is 0xa5 /usr/src/sys/boot/i386/libi386/biosdisk.c try to bee too smart…
Henri
Henri Hennebert said,
Correction to my previous post:
I don’t no why, the previous solution work only once. Or maybe I dream it work …
The loader can’t be loader from a slice other than with the sysid 0xa5.
So I just put a bsdlabel on ad0s2 with ad0s1a starting at 0 and extending over the complete slice (so part a == part c) and put the zfsboot on ad0s2a and create the pool on ad0s2a.
Everything really run smoothly now.
Henri
Quoi tu n’es pas au courant ? | GCU-Squad! said,
[…] pour autant avoir à tout changer chez toi, ni à planquer tes vieilleries, désormais, c’est simple et […]
Quoi tu n’es pas au courant ? | GCU-Squad! said,
[…] pour autant avoir à tout changer chez toi, ni à planquer tes vieilleries, désormais, c’est simple et […]
Add A Comment