If a line starts with $ it is a command to be run as a normal user, if it starts with # it is a command to be run as root.

To run a stress test on nodes before reincluding them in condor:

• $ cd /net/data07/tests/stress_test/
• $ nohup ./stress_test.sh ZW_3lepnu_1898_X.job myemail@umich.edu &
• $ nohup ./stress_test.sh ZW_3lepnu_1899_X.job myemail@umich.edu &

Replace "X" with the number 1-4. If you are testing multiple nodes at once, the number for each node must be unique. The tests will for 9 hours and an email will be sent to "myemail@umich.edu" when each one finishes with a summary of results. If an email address is not given, it will default to umrocks.logs@gmail.com. If you have trouble recieving mail from the clusters, try using this default account instead. Contact Jeff for the password. The actual output files will be overwritten the next time the test is run. If the job crashes, an email should be sent at that point, but if the whole machine hangs you'll have to check in the morning.

-- JeffGregory - 14 Feb 2006

Steps to compile custom AFS rpms for RHEL4/SL4/SLC4

• // get all SOURCES, SPECS and SRPMS from http://www.openafs.org/release/latest.html
• wget -r http://www.openafs.org/dl/openafs/1.4.0/rhel-4/
• rpm -ivh SRPMS/openafs-1.4.0-rhel4.1.src.rpm
• rpmbuild -ba /usr/src/redhat/SPECS/openafs.spec --define "kernvers `uname -r`"
• // remove old rpms, you might want to run rpm -qa | grep openafs first to make sure there's nothing you want to keep.
• for rpms in `rpm -qa | grep openafs`; do sudo rpm -e --nodeps $rpms; done
• rpm -ivh /usr/src/redat/RPMS/openafs-1.4.0-1.1.x86_64.rpm
• // do for each of openafs-kpasswd-1.4.0-1.1.x86_64.rpm, openafs-kernel-smp-1.4.0-2.6.12_3FastNFSv4smp_1.x86_64.rpm, openafs-client-1.4.0-1.1.x86_64.rpm, openafs-krb5-1.4.0-1.1.x86_64.rpm, openafs-1.4.0-1.1.x86_64.rpm
• mkdir /afs
• chkconfig -add openafs-client
• /etc/init.d/openafs-client start

-- JeffGregory - 16 Feb 2006

Alternative method for making custom AFS rpms

• // Get SRPM from http://www.openafs.org/release/latest.html
• cd /usr/src/redhat/SRPMS
• wget -r http://www.openafs.org/dl/openafs/1.4.0/rhel-4/openafs-1.4.0-rhel4.1.src.rpm
• // Note that i686 was choosen below, could also be x86_64 or ia64 as required
• rpmbuild --rebuild --target i686 --define "build_userspace 1" --define "build_modules 1" /usr/src/redhat/SRPMS/openafs-1.4.0-rhel4.1.src.rpm
• // By default just the kernel module rpm would be built without the above --define entries
• // remove old rpms, you might want to run rpm -qa | grep openafs first to make sure there's nothing you want to keep.
• for rpms in `rpm -qa | grep openafs`; do sudo rpm -e --nodeps $rpms; done
• rpm -ivh /usr/src/redat/RPMS/openafs-1.4.0-1.1.x86_64.rpm
• // do for each of openafs-kpasswd-1.4.0-1.1.x86_64.rpm, openafs-kernel-smp-1.4.0-2.6.12_3FastNFSv4smp_1.x86_64.rpm, openafs-client-1.4.0-1.1.x86_64.rpm, openafs-krb5-1.4.0-1.1.x86_64.rpm, openafs-1.4.0-1.1.x86_64.rpm
• mkdir /afs
• chkconfig -add openafs-client
• /etc/init.d/openafs-client start

-- ShawnMcKee - 29 Mar 2006

Resize partitions using 'parted'

• Make sure the partition to resize PRECEDES the empty area on the disk
• Make sure you are using EXT2 or EXT3
• Run 'parted /dev/xxx' where xxx = sda or hda or whatever
• At the (parted) prompt type 'print'
• You can see the current start of the partition and you need to determine the end (in megabytes)
• If you want to resize partition 3 you would issue a command like: resize 3 0.031 98456

Configure NFS and IPTABLES to work together

Create the file /etc/sysconfig/nfs and use it to set the ports of different services:

#START OF NFS
LOCKD_TCPPORT=32777
LOCKD_UDPPORT=32777
RQUOTAD_PORT=991
MOUNTD_PORT=1001
#END OF NFS

Change /etc/sysconfig/iptables to allow these ports:

# START OF IPTABLES
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -p 50 -j ACCEPT
-A RH-Firewall-1-INPUT -p 51 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp --dport 5353 -d 224.0.0.251 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 631 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 7001 -j ACCEPT
# START OF LINES ADDED FOR NFS
-A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 2049 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 2049 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 111 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 111 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 991 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 991 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 32777 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 32777 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m udp -p udp --dport 1001 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 1001 -j ACCEPT
# END OF LINES ADDED FOR NFS
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT
#END OF IPTABLES

-- JeffGregory - 07 Mar 2006

Configure number of NFS daemons

To increase the number of nfs daemons, edit the file /etc/sysconfig/nfs. Look for a line like:

RPCNFSDCOUNT=64 

If the line or file do not exist, add it and change 64 to the number of daemons you would like to have running. This can also be set in /etc/sysconfig/network.

This file is sourced by almost all of the network related services (NFS, ypserv, networking, etc) and will not be overwritten when the system updates. The startup scripts however are frequently overwritten and changes to them should be avoided.

-- JeffGregory - 26 Apr 2006

Adding new users with AFS home directories

• First I get a random password from this site to use as a default: http://motdepasse.site.voila.fr/index.html
• # ssh root@linat03
• # kinit admin
• # /root/adduser.krb
• Enter the new users username and password when prompted

-- JeffGregory - 15 May 2006

Adding and updating the local repository

• First install vsftpd or httpd on the server machine.
• Make a pub/SLC/4x/custom directory in either the ftp or http root (/var/ftp or /var/www/html). To enable both, "ln -s /var/ftp/pub /var/www/html". ftp won't follow symlinks)
• Copy rpms for each architure into their own directory (i386, i686, x86_64)
• Run createrepo /var/ftp/pub/SLC/4x/custom

To have the repo update automatically each night, create /etc/cron.daily/createrepo

#! /bin/sh
/usr/bin/createrepo /var/ftp/pub/SLC/4x/custom

The machines need a repo file to tell them to access this repo, such as /etc/yum.repos.d/umatlas.repo

[umatlas]
name=UM-ATLAS
baseurl=http://linat05.grid.umich.edu/pub/SLC/4x/custom/
enabled=1

-- JeffGregory - 23 May 2006