Replacing disk in zpool (potentially with larger disks)

ZFS documentation

Replacing disk in zpool (potentially with larger disks)
Note: use "parted" > "mklabel GPT" > "q" to format the disk.
If the above doesn't cause the zpool to expand, try using "$ zpool online -e <pool>" or "$ zpool online -e <pool> <disk>"

Actual steps taken

Notes before we get started.

• Each disk in a zfs pool is actually single physical disk setup as a RAID0 virtual disk, this is because the RAID controller will not pass the physical disks to the OS (lame).
• This means that each time we replace a disk in a zfs pool we will need to destroy the original raid configuration on the old disk (optional) and then create a new raid configuration on the new disk (mandatory).
• While this can be done from the RAID Controller on boot, it's much easier to use omconfig/omreport on the command line.

Steps to take:

1. Check what disks are in the pool:
   $ zpool status pool
2. Set disk that needs to be replaced offline in zfs pool:
   $ zpool offline pool disk
3. Optional: Destroy disk's raid configuration:
   $ omconfig storage vdisk action=deletevdisk controller=<controller #> vdisk=<vdisk #>
   Can find the controller number and disk number using omreport (Ex: omreport storage vdisk controller=0)
4. Optional: Turn off machine (might not be possible if you don't want to lose services the machine offers)
   $ shutdown -h now
5. Replace disk with new disk
6. Clear foreign raid configurations, i.e., any raid configuration that is already on the new disk:
   $ omconfig storage controller action=clearforeignconfig controller=<controller #>
7. Create new vdisk:
   $ omconfig storage controller action=createvdisk controller=<controller #> raid=r0 pdisk=<pdisk #> size=<size of disk> name=<unique name, no space, no quotes>
   Can find the pdisk # using:
   $ omreport storage pdisk controller=<controller #>
8. Replace the disk in the zpool:
   $ zpool replace pool olddisk /dev/newdisk
   Ex: zpool replace testpool sdb /dev/sdb
9. Set disk to online in zpool (this step may be unnessecary):
   $ zpool online pool disk
   If the disk is larger than the previous one and you'd like to use the new space, you may need to use:
   $ zpool online -e pool disk
   Note that the new space will not be used until the autoexpand variable is set.
10. If you'd like the pool to expand to use the new space, use the following once all the necessary disks have been replaced:
    $ zpool set autoexpand=on pool
11. Turn off autoexpand for safety, i.e. what if you want to replace a disk with a larger one temporarily:
    $ zpool set autoexpand=off pool