Installing SmartOS

Reader's Note: I will be making an effort this year to post more regularly about what I'm doing with technology. We'll see how well this goes.

While SmartOS was designed by Joyent to be installed on thousands of machines simultaneously at datacenter scales, it can also be used in much smaller deployments of a handful, or even just a single machine.

This is excellent in the home visualization space, as SmartOS integrates the hypervisor and storage subsystem into a single component, removing the need for any storage infrastructure, and making it even more attractive for use in the home or small office.

It also allows for the deployment of Illumos Zones OR full Virtual Machines (contained within zones) allowing you to get both maximum performance and flexibility out of your hardware.

What follows is a brief guide on setting up and configuring SmartOS for use in a stand-alone environment like this. It specifically outlines the tweaks I personally made to my installation at home. And really that's the only point of it.

SmartOS Architecture

SmartOS boots from a live image which provides the Illumos kernel and a root ramdisk. The zones ZFS pool is imported, SMF manifests are imported, and the system (optionally) starts previously running zones and VMs. If the zones pool is not found, SmartOS will go through it's installation process.

Notice: Despite looking like a (relatively) normal UNIX environment, the global zone is quite specialized, and should not be used as a general purpose operating environment. It should only be reserved for tasks that are ill-suited for running within a Zone or VM.

Hardware

I've installed SmartOS on my HP N54L Microserver, which supports everything but KVM. Since I also use one of the Zones as my home router, I've added an additional Network Interface, as described in my previous blog post.

Downloading

The latest SmartOS live images are available at smartos.org. Click on the large download link to the right. The ISO is nice and simple, but can get costly to download and re-burn each time you decide to upgrade to the latest build image. I recommend using the USB image, especially in my case since the N54L has an on-board USB port for booting from.

Ubuntu has an excellent disk imager utility which I recommend using to burn USB images to USB drives on Windows.

Installation

Installing SmartOS is as simple as burning the image to the medium and booting from it. Depending on the read speed of your CD or USB drive, booting can take a while, and since SmartOS was designed for unattended use, it's not overly verbose on the console while it's booting.

Configuration

If there is no configured zones pool on your hardware, you will be taken to a configuration menu to provide required information.

-----------------------------------------------------------------------
SmartOS Setup  
Copyright 2011, Joyent, Inc.            http://wiki.smartos.org/install  
-----------------------------------------------------------------------

You must answer the following questions to configure the system.  
You will have a chance to review and correct your answers, as well as a  
chance to edit the final configuration, before it is applied.

Would you like to continue with configuration? [Y/n]  

Provide basic networking information, select the disks to be used to for the storage pool, and specify a root password.

2016 Update: The configuration options for a new installation of SmartOS has changed over the last year. Now, in addition to basic network information, static DNS servers and NTP servers can be set, as well as a default search domain. Additionally, you can specify your zpool configuration (raidz2, mirrored, or manual)

You will have a chance to edit the configuration file before it's written to the storage pool, and then the configurator will create the storage pool, several critical datasets, populate them with files and write the configuration to the pool, then reboot.

Tweaks

While it's easy to get started right away using SmartOS, this is exactly the time to ensure that your experience is going to be the best that it can be.

Checksum

By default, SmartOS uses Fletcher4 as it's checksum.

[root@gz ~]# zfs get checksum
NAME           PROPERTY  VALUE     SOURCE  
zones          checksum  on        default  
zones/archive  checksum  on        default  
zones/config   checksum  on        default  
zones/cores    checksum  on        default  
zones/dump     checksum  noparity  local  
zones/opt      checksum  on        default  
zones/swap     checksum  on        default  
zones/usbkey   checksum  on        default  
zones/var      checksum  on        default  

While this is normally just fine, I prefer the security of a secure hashing algorithm, and fortunately ZFS delivers.

[root@gz ~]# zfs set checksum=sha256 zones

2016 Update: SmartOS now ships with several new hashing algorithms, namely sha512, skein, and edonr, all of which are superior to fletcher in collision resistance and sha256 in performance. I recommend using one of these instead of sha256, especially if you value disk performance.

We will leave the noparity local option alone for zones/dump.

Compression

I'm a fan of always using compression on disk. The lz4 compression algorithm is a high-performance replacement for lzjb, which features significantly faster compression and decompression, as well as a moderately higher compression ratio than it's predecessor. Lets see how SmartOS makes use of this compression algorithm by default.

[root@gz ~]# zfs get compression
NAME           PROPERTY     VALUE     SOURCE  
zones          compression  off       default  
zones/archive  compression  lzjb      local  
zones/config   compression  off       default  
zones/cores    compression  gzip      local  
zones/dump     compression  off       local  
zones/opt      compression  off       default  
zones/swap     compression  off       default  
zones/usbkey   compression  off       default  
zones/var      compression  off       default  

Unfortunately, not at all. Lets fix that.

[root@gz ~]# zfs set compression=lz4 zones
[root@gz ~]# zfs inherit compression zones/archive

Besides enabling lz4 for everything, we ensure that zones/archive inherits it from zones (as lz4 > lzjb). However, we do leave the zones/cores with gzip and zones/dump without compression.

2016 Update: If the lz4_compress feature is enabled on your storage pool, lz4 will be the default compression algorithm used, and will no longer need to be explicitly set.

ARC and Swap

If there's not enough free memory, the kernel will start paging memory out to persistent storage. If that happens, we don't need to cache it in free memory as well (seeing as we won't have much of that left). Because of this, I always limit ARC caching of swap to metadata only:

[root@gz ~]# zfs set primarycache=metadata zones/swap

Conclusion

And thats what I do to a SmartOS installation before downloading images. Please let me know what you do to your SmartOS installations in the comments section below.