Note, this does not cover using the gpt based partition tables. If you want to use these, please refer to the following page: http://lulf.geeknest.org/blog/freebsd/Setting_up_a_zfs-only_system/ or booting zfs as root using a small ufs boot partition as provided by the instructions at: http://wiki.freebsd.org/ZFSOnRoot.

Below is the steps required to be able to setup the root zpool

1. Download a FreeBSD -current fixit cdrom snapshot later than 200901, as these have loader ZFS support
2. Burn the CD
3. Boot the CD
4. Setup any partitions you want – note you must setup the ‘a’ partition to cover the entire device as the loader will use this.
5. Select Fixit from the menu, and use the CDrom as a source
6. Create the pool and install the loader (see below)
7. Copy the required files to boot (see below)

Creating the root zpool and installing the Loader

The fixit cd has everything required to create a zpool, however by default none of the required modules are loaded. Hence they need to be loaded first:

cd /mnt2/boot/kernel
kldload ./opensolaris.ko

kldload ./zfs.ko

Once the modules have been loaded all the zfs tools (zpool,zfs,zdb) should now work. Let assume you want to install FreeBSD to /dev/ad4s2 (second partition on a sata disk). You can do this using:

    zpool create somename /dev/ad4s2

Where somename is the name of the pool you want to create. This creates a single zfs filesystem and a zfs pool of storage. To install the boot loader you need to do:

    # dd if=/mnt2/boot/zfsboot of=/dev/da0s1 count=1
    # dd if=/mnt2/boot/zfsboot of=/dev/da0s1 skip=1 seek=1024

The first line installs boot1, the second line installs boot2.  However, boot2 is responsible for loading boot3 (aka the loader – found in /boot/loader). Hence that must be put in place.

Copying the required files to boot

The easiest way to get things to the point where things are ready to boot is to copy all the files from /dist  - the live distribution. Before you do this, you might like to take advantage of zfs and create some subfilesystems so you can snapshot, monitor space, etc.

For instance creating a /usr and /var filesystem is often very handy:

    #zfs create somename/usr
    #zfs create somename/var

Now you can copy the base system:

    cp -a /dist/* /somename

This will install among other things:

• /somename/boot/kernel/kernel  - FreeBSD kernel
• /somename/boot/kernel/opensolaris.ko – zfs dependency
• /somename/boot/kernel/zfs.ko – module understanding zpools/zfs
• /somename/boot/loader  - the FreeBSD loader
  

At this point you need to replace the loader with one that understands zfs. You can download the loader from: (To be advise – see cavet below)
And if you have a usb stick copy it in place using:

   mount_msdos /dev/da0s1 /mnt
   cp /mnt/loader /somename/boot

Finally you have to tell FreeBSD where to mount filesystems on a standard boot:

   zfs set mountpoint=/var somename/var
   zfs set mountpoint=/usr somename/usr

At this point any command you type will now probably fail indicating it’s missing some shared library. This is because /usr has now changed. You can get around this by telling the loader where to find valid libraries:

   export LD_LIBRARY_PATH=/mnt2/lib

Next we need to build the zfs cache. This is used by zfs mount to automatically mount zfs filesystems by /etc/rc.d/zfs at boot time. It’s also used to determine if a filesystem is local to the system or belongs to an exported pool.

   mkdir /boot/zfs
   mkdir /somename/boot/zfs
   cd /
   zfs export somename
   zfs import -f somename
   cp /boot/zfs/zfs.cache /somename/boot/zfs/

Finally we tell the loader where we want to boot from and set the init scripts to automatically start all zfs filesystems:

    echo 'zfs_enable="YES"' > /somename/etc/rc.conf
    echo 'zfs_load="YES"' > /somename/boot/loader.conf
    echo 'vfs.root.mountfrom="zfs:somename"' >> /somename/boot/loader.conf

And set the root filesystem to a legacy mountpoint (so zfs mount -a won’t try and mount an already mounted filesystem)

     zfs set mountpoint=legacy somename

At this point you can reboot and things should now boot!
Update: 20090809

There has been a lot of updates to instructions along the way. There are now official ZFS on Root instructions available using GPT/MBR/other available at: http://wiki.freebsd.org/RootOnZFS

The Problem

After installing Windows Vista Enterprise Edition, I very quickly wanted to get back to FreeBSD. As per previous versions of Windows, Vista does a great job of overwriting the Master boot record (MBR). Unlike previous versions of windows, you cannot just install a different boot loader in the MBR – so I found out the hard way. (It used to be sooo easy fdisk /mbr would fix any broken boot sectors, then just install your favourite boot loader).

Vista writes a unique disk id in the MBR. This unique id (UID) is used by Vista to determine what disk is what. It does make some sense, Hardware can shift around, but the UID should always be the same. This allows Vista to always know where it’s files are and to appropriate label drive C: as the correct drive.

Installing another boot loader, however (ie Grub, lilo, the FreeBSD boot0 / boot loader /boot manager) will overwrites the UID in the MBR and Vista fails to load with an error:

"unable to access \windows\winload.exe"

The error message is semi appropriate but very misleading. What it really should say is ”I can’t work out which of my the drives has my files on it hence I can’t find winload.exe”.

• ‘Full details about the UID can be found at: http://multibooters.co.uk/mbr.html

Existing Solutions

Out on the web there is many solutions to the dreaded winload.exe. Many of them are misleading, some I believe are not even tested. Also many apply to Vista Release Candidate 1 (rc1) which I believe didn’t required the UID to be present in order to successfully boot. Below is some of the solutions that people say work (but Ifound didn’t).

Using Grub

• Ref: http://ubuntuforums.org/showthread.php?t=358175
• Ref: http://blogs.sun.com/moinakg/entry/vista_and_solaris_express_dual

Some say grub loading windows with something like:

Title Vista
  root (hd0,0)
  makeactive
  chainloader +1

will work – but grub still over writes the MBR hence the same issue will occur – though you may be lucky as the grub MBR is really small (and might leave the UID) untouched.

Using gag

There’s reports on a FreeBSD mailing list that gag will work, thought I’m running FreeBSD/amd64 which gag or grub for that matter won’t compile on. I know I could always install the lib32 compatability distribution to use it but I wanted the default FreeBSD boot loader.

Using the Vista Boot Loader

• Refs: http://www.tech-recipes.com/rx/2014/vista_using_bcdedit_boot_up_options_dual_boot_xp_prof’

Many people have given up on trying to use a normal boot loader, and instead use the Vista boot loader.
The WinXP/NT bootloader (ntldr & config file boot.ini) have been replaced with files winboot.exe and bcedit.exefor configuration.

Theses commands could be setup to boot other operating systems though your milege may vary.

Commercial Software

Many people have decided it’s all too hard and gone and purchased software to handle it for them – there’s lots out there

• VistaBoot (http://vistabootpro.org)
• EasyBCD

Utilities

• Ref: http://www.sysint.no/en/Download.aspx

There is many utilities that claim to fix the issue. One such util is MBRFix.exe, this fixes the fact that vista won’t boot but it will clobber any boot loader you have got installed.

The Fix (That Worked)

• Obtained from: http://forum.onmac.net/showthread.php?t=1324

The simple fix is to tell Vista that the drive it boots off of will always be the drive it can find its files on. This way you can clobber the UID in the MBR as much as you want and Vista won’t care. To do this you run:

BCDEDIT /set {bootmgr} device boot
BCDEDIT /set {default} device boot
BCDEDIT /set {default} osdevice boot

This can be run from either the Vista repair cd, from within Vista, or from something likehttp://www.ultimatebootcd.com (For Windows).

Conclusion

Hence it IS possible to run Vista with any boot loader that you like – in my case the FreeBSD boot loader is the one of choice.