Dell Deployment Toolkit

Dell provide a toolkit for automating configuration of their systems. A Linux environment can be booted from CD or via PXE which includes the drivers and syscfg and raidcfg tools allowing for BIOS and RAID settings to be customized. The toolkit provides several methods for scripting and automating deployments.

Security

Note that the DTK automatically (at least when PXE booted) brings up eth0 and starts an sshd there with no password for root. This seems ok on low risk nodes when eth0 is on the local network (for example worker nodes on the ROCKS clusters), but could be an issue for other environments.

Get It

Download from ftp://ftp.us.dell.com/sysman/ Look for the newest version of DTK Linux ISO.

Setup for PXE booting

It is easy PXE boot DTK from the ROCKS PXE infrastructure.

First mount the ISO and get the files isolinux/SA.1 and isolinux/SA.2 These are the Linux kernel and the initrd/ramdisk containing drivers and Dell tools.

I created a directory at /tftpboot/pxelinux/dtk to hold these files. /tftpboot/pxelinux is the base area for tftp in ROCKS.

Then a node's pxelinux.cfg file should look like this, to PXE boot DTK:

default dtk
prompt 0
label dtk
        kernel dtk/SA.1
        append initrd=dtk/SA.2 ramdisk_size=65536

The DTK Linux environment will automatically initialize eth0 using dhcp. It also starts an open sshd server --- you can connect as root without a password.

ROCKS Commands to Add bootaction

In ROCKS 5.2, the bootaction and boot commands have been changed from earlier versions --- the pxeaction command has gone to bit heaven.

rocks add bootaction action=dtk kernel="kernel dtk/SA.1" ramdisk="dtk/SA.2" args="ramdisk_size=65536"
rocks set host runaction cc-117-1 action=dtk
rocks set host boot cc-117-1 action=run

To set the host back normal and do an install:

rocks set host runaction cc-117-1 action=os
rocks set host boot cc-117-1 action=install

Note you can also set installaction, normally this shouldn't be changed (always use the same installer and options), but to set it to the normal setting, do:

rocks set host installaction cc-117-1 action=install
rocks set host boot cc-117-1 action=install

Scripts

You can give the Linux environment a script to automate a deployment action. The script can be fetched from and NFS share or via tftp.

TFTP

Here is an example using tftp to fetch script dtk/raid0.sh

We want the pxelinux.cfg to look like:

default dtk
prompt 0
label dtk
        kernel dtk/SA.1
        append initrd=dtk/SA.2 ramdisk_size=65536 share_type=tftp share_location=dtk share_script=raid0.sh tftp_ip=10.10.128.12

The ROCKS commands to setup are:

rocks add bootaction action=dtk kernel="kernel dtk/SA.1" ramdisk="dtk/SA.2" args="ramdisk_size=65536 share_type=tftp share_location=dtk share_script=raid0.sh tftp_ip=10.10.128.12"
rocks set host runaction cc-117-1 action=dtk
rocks set host boot cc-117-1 action=run

NFS

Given an NFS share at /exports/dtk want this pxelinux.cfg:

default dtk
prompt 0
label dtk
        kernel dtk/SA.1
        append initrd=dtk/SA.2 ramdisk_size=65536 share_type=nfs share_location=10.10.128.12:/exports/dtk share_script=raid0.sh

The ROCKS commands to setup are:

rocks add bootaction action=dtk kernel="kernel dtk/SA.1" ramdisk="dtk/SA.2" args="ramdisk_size=65536 share_type=nfs share_location=10.10.128.12:/export/dtk share_script=raid0.sh"
rocks set host runaction cc-117-1 action=dtk
rocks set host boot cc-117-1 action=run

Script example

Here is the script, it uses raidcfg to setup "RAID0" on the two drives in a compute node. Tested on r610 with SAS6i controller.

#!/bin/bash

cat > /tmp/raid0.ini <<EOF
[vdisk0]
controllerid=0
;size=278784
raid=0
readpolicy=nra
writepolicy=wt
stripesize=64
cachepolicy=
adisk=0:0:1,0:1:1
EOF

raidcfg -ctrl -c=0 -ac=rst
raidcfg -i=/tmp/raid0.ini

Note that the first radicfg command resets the controller. This removes all RAID config from it, putting the disks in to "JBOD" mode.

Side note on using omconfig from OS

Our normal ROCKS builds include raidcfg and omconfig programs. For some reason raidcfg isn't working for me, but omconfig is. The two have similar (indentical?) functionality.

It is possible to redo the RAID config from a running ROCKS client, but doing so of course destroys the OS install. However, with ROCKS builts being so reliable and easy, this is a workable method for changing RAID configs and is probably easier than the above for existing ROCKS clients. This is tested for SAS6i.

For a host that is using RAID, you can switch to JBOD by issuing a controller reset:

omconfig storage controller action=resetconfig controller=0

For a node that is in JBOD mode, you can switch to RAID0 (creating vdisks with other RAID configs should work as well):

omconfig storage controller action=createvdisk controller=0 raid=r0 adisk=0:0:0,0:0:1 size=max

Note that after either of the above two commands are run, the OS will be unable to read anything more from disk --- you won't be able to issue anymore commands to the system. Not even a shutdown. Switch the node's boot action to install in ROCKS and power cycle the node via IPMI or PDU to reinstall.

BIOS Flashing

Some of the recent BIOS packages for Linux are not working from a RHEL5.3+ system. This presents an issue as we normally do BIOS updates from the production Linux installs. For example, the system BIOS can not be applied from SL54 on the R610 nodes.

The DTK Linux environment should be able to run any of the Dell updates. With the above PXE boot of dtk, given only the SA.1 kernel and SA.2 initrd, the BIOS update will run. As a simple test, do the PXE boot, ssh in and scp the update from somewhere on local net and run it. Works.

Scripting BIOS updates

Want to script a file transfer in... Here will use tftp (remember that the tftp is provided by busybox, options slightly different). On the tftp server, have the root location /tftpboot/pxelinux with files in a directory dtk/files. Have a symlink named "r610.bios" pointing to actual bios package.

On the node running DTK, can do following:

-bash-3.00# tftp -g -r dtk/files/r610.bios -l bios.sh 10.10.128.15
-bash-3.00# sh bios.sh -q
-bash-3.00# reboot

So stick that in a script and run as above...

#!/bin/bash

tftp -g -r dtk/files/r610.bios -l bios.sh 10.10.128.15
sh bios.sh -q
sleep 3600
reboot

Note the long sleep --- want to try to avoid reboot loops. Alternately, use "shutdown", or just don't do anything and after putting bootaction back to normal, manually induce a reboot (ssh, ipmi, pdu, etc.).

Add the ROCKS DB entries (note that ROCKS is very slow to run the "set host runaction" command, it seems to do a "set host boot" on every cluster node after this, which takes several seconds):

rocks add bootaction action="dtk r610bios" kernel="kernel dtk/SA.1" ramdisk="dtk/SA.2" args="ramdisk_size=65536 share_type=tftp share_location=dtk share_script=r610bios.sh tftp_ip=10.10.128.15"
rocks set host runaction cc-115-1 action="dtk r610bios"
rocks set host boot cc-115-1 action=run

The above goes fine for doing the one update type on the one hardware type. If we want to do more variations, some work is needed...

-- TomRockwell - 05 Dec 2009