Transferring Zone Delegated Datasets

You're minding your own business, re-deploying a zone based on the most up-to-date base image, and suddenly find yourself needing to transfer the contents of a delegated ZFS dataset from old to new.

We've all been there.

Sure you could manually re-create the dataset(s) in the new zone and copy all of the contents between them, but there are two big reasons that may be a very bad idea:

• You could have lots of data.  At home, I use a SmartOS Zone as a file server that contains several TB worth of camera RAWs and PSD files.  Copying all of this data off of one dataset and on to another seems like a complete waste of time and energy.  Since my HP N54L already doesn't like ZFS scrubs (one of the downsides to using SHA256 checksums), I shudder to think of how long it will take when I ask it to bulk copy data to and from its little 4 disk array.  It's an operation that would likely be measured in weeks or days, rather than seconds or minutes if we're considering the latency of wetware.
• You could have lots of datasets.  In a data-driven world of nails, ZFS is the ultimate hammer.  Like many of the people who have been bitten by the ZFS bug, I make liberal use of ZFS features whenever possible.  Each user in my home directory has their own dataset (as well as the home directory being its own dataset).  Certain large regions of my home directory dataset are also organized with their own datasets.  I automate the creation of ZFS snapshots with cron, and to ensure that none of this awesomeness gets lost, I use ZFS send/recv for online replication.  Recreating all of my datasets from scratch would mean disrupting all of these operations, not to mention create tons of unnecessary cruft in Zpool history.

Fortunately, there is another solution to the problem: Directly moving the dataset between zones.  It should be a simple one-liner with zfs rename.  Let's give it a shot:

[root@gz ~]# zfs rename zones/<src>/data zones/<dst>/data
cannot rename 'zones/<src>/data': dataset is used in a non-global zone

Well, that didn't work at all.

It turns out Illumos prevents datasets from being moved in this manner if it's been delegated to a zone.  So we'll have to unmount the dataset, remove the delegation, rename the dataset, re-delegate it to its new home, and then remount it.  This might sound complicated, but it's really just a few more steps.  Lets set the stage and go through it.

First of all, we're going to shut down the zones we're going to operate on and verify the state of our datasets before proceeding.

[root@gz ~]# vmadm stop <src>
Successfully completed stop for VM <src>
[root@gz ~]# vmadm stop <dst>
Successfully completed stop for VM <dst>
[root@gz ~]# zfs list -o name,zoned,mounted,mountpoint
NAME                                  ZONED  MOUNTED  MOUNTPOINT
zones/<src>                             off      yes  /zones/<src>
zones/<src>/data                         on       no  /zones/<src>/data
zones/<src>/data/home                    on       no  /home
zones/<src>/data/home/admin              on       no  /home/admin
zones/<src>/data/home/user1              on       no  /home/user1
zones/<src>/data/home/user1/moredata     on       no  /home/user1/moredata
zones/<src>/data/home/user2              on       no  /home/user2
zones/<dst>                             off      yes  /zones/<dst>
zones/<dst>/data                         on       no  /zones/<dst>/data

Specifically, all of the delegated datasets need to be unmounted before you continue.  If they are not, mount points will form in your global zone and significant disruption to your hypervisor may occur.  zfs unmount can help here if your zfs list output doesn't look like the one above.

Next, we're going to disable the zoned attribute on our destination dataset, move the data over, and re-enable the zoned attribute.  I personally prefer the use of zfs inherit over zfs set to clear the zoned attribute, however, zfs set zoned=off should work just as well here.

[root@gz ~]# zfs inherit zoned zones/<src>/data

Now you can delete the destination dataset and replace it with the source dataset in its entirety.

[root@gz ~]# zfs destroy zones/<dst>/data
[root@gz ~]# zfs rename zones/<src>/data zones/<dst>/data

Or if you prefer, you can transfer child datasets from the source dataset:

[root@gz ~]# zfs rename zones/<src>/data/home zones/<dst>/data/home

Next step is to verify that none of your mount points are occluded by other directories and files within the destination zone.  This would be an issue in my example, which can be solved by either deleting or renaming the home directory within the zone.

[root@gz ~]# mv /zones/<dst>/root/home /zones/<dst>/root/home_old

After that, bring up your zones using vmadm.  Don't worry about changing the zoned property back on again, SmartOS takes care of that for you.

[root@gz ~]# vmadm start <src>
Successfully started VM <src>
[root@gz ~]# vmadm start <dst>
Successfully started VM <dst>

And there you have it.  A quick and easy way to transfer custody of your delegated datasets between zones.  And we didn't even get our hands dirty with zoneadm.  An exercise which I will save for another day.