Athena Software Setup at Michigan

Overview

We have installed an Atlas software "mirror" and the Athena "Multi" installation of all 12.X.Y Athena versions on the Michigan cluster. The mirror is the software repository from which ATLAS installation kits may installed. The kit installations have the Athena software which users run. One can install ATLAS installation kits using the UM mirror to other machines directly from the UM mirror (see Pacman Notes for instructions on how to install Athena using pacman).

The Athena installation kits provide a run-time environment for Athena and related software like CMT and pool. They do not provide CVS access to edit/update Athena code at CERN. However, it is possible to checkout code from the CERN Athena repository and compile/run locally (see Athena Code Checkout).

Software Location

/data07/athena/multi

This is an Athena "Multi" installation which has multiple Athena versions installed in many subdirectories, such as Athena project directories which have names beginning with the prefix "Atlas", for example AtlasAnalysis. These are the same directories as used in the "builds" area on lxplus or BNL. There are are other directories like CMT and usr for other software installed by the kit.

/afs/atlas.umich.edu/atlas/software/multi

This is another Athena "Multi" installation, identical to the one above, except on AFS. Originally, we intended to have all software on AFS, but we had some AFS problems we decided to put an installation on NFS (i.e the /data07 location above). This installation is currently (Mar, 2007) equivalent to the one on NFS, and we hope the AFS problems are gone.

/afs/atlas.umich.edu/atlas/software/athena

This directory has single kit Athena installations labelled by Athena version numbers. These single kit installations were made during the period of AFS problems when the multi installation was not getting updated properly. Eventually this directory will be eliminated when we are sure the multi installation works correctly.

/afs/atlas.umich.edu/atlas/software/mirror/ATLAS.mirror

This is the ATLAS mirror (basically a repository of ATLAS software in compressed tar files). The program pacman is used to install Athena installations from a mirror. In pacman the UM mirror can be accessed via the abbreviation am-UM, by its direct filename /afs/atlas.umich.edu/atlas/software/kits/ATLAS.mirror (if you have /afs access), or by the URL http://gate01.grid.umich.edu/am-UM/ATLAS.mirror. See Pacman Notes for instructions on how to install Athena using pacman.

You can quickly check which Athena versions are installed by looking in any of the "Atlas" project and looking at the numbered subdirectories. These subdirectories are 10 less than then Athena version (e.g. subdirectory 2.0.2 = Athena 12.0.2 [however, this correspondence may eventually be broken])

One-time Athena set up.

Athena is an immensely complex program and not easily summarized. Here I give some basic instructions. For more details see the ATLAS Computing Workbook.

1. Create an "Athena working area", i.e. some directory for use with Athena. The default name is $HOME/testarea, but it can have any name as long as that is reflected in your requirements file (see below). Each specific Athena release you want to work with will have a subdirectory called AtlasOffline-X.Y.Z (X.Y.Z = Athena version, e.g. 12.0.6) in this working area. These subdirectory names can be over-ridden in the requirements file if desired. You can make multiple Athena work areas for different purposes, each with their own cmthome directory and requirements files. Its also a good idea to make your Athena working area on an NFS filesystem rather than AFS, because if you want to run batch jobs with condor as it turns out you can only write to NFS filesystems and not AFS in condor jobs.

2. Create a cmthome directory for your requirements file. The requirements file and the setup scripts are used by the CMT which is the program used to manage Athena code. Creating a requirements file is rather tricky (see AtlasLogin twiki page), so I recommend you use the handy script create-cmthome.sh which will create the requirements files and set up scripts for you. This script has been placed in all the Athena installation directories. Here is what you do:

cd <Athena installation directory> (e.g. /data07/athena/multi)
./create-cmthome.sh

This script will create a directory $HOME/cmthome containing files requirements, setup.(c)ch, and cleanup.(c)sh. This step only needs to be repeated when the CMT version is changed, which does not happen very often. I usually copy these files to another "cmthome" directory since I often have multiple Athena working areas with different requirements files.

You can edit your requirements file to customize your setup. For example, the default requirements file assumes that your working area is $HOME/testarea and you must edit the ATLAS_TEST_AREA line if it is not. If you do change your requirements file you must re-create the setup.(c)sh files as follows:

cd cmthome
source <Athena installation directory>/cmtsite/setup.sh
cmt config

Each-time you use a specific Athena release

1. Configure CMT to use a specific release, X.Y.Z:

cd <Athena working directory>
source cmthome/setup.(c)sh -tag=X.Y.Z

After this command is number of environmental variables are defined such as CMTPATH (directory path to look for source code) and SITEROOT (location of Athena code).

2. Run the Athena runtime script

source <Athena working directory>/AtlasOffline/X.Y.Z/AtlasOfflineRunTime/cmt/setup.(c)sh

A handy way to run this command is to make use of the fact that CMT setup script also defines environmental variable AtlasArea which is defined as <Athena working directory>/AtlasOffline/X.Y.Z. Thus the above command can be run as:

source $AtlasOffline/AtlasOfflineRunTime/cmt/setup.(c)sh

Of course, you can put the commands in (1) and (2) in a little script which you source to configure Athena, rather than typing everything out.

3. Create a "run" directory below your "AtlasOffline-X.Y.Z" directory for your Athena options file e.g:

cd $HOME/testarea/AtlasOffline-X.Y.Z
mkdir run
cd run

This "run" directory (which can have any name) is the directory from which you run Athena. Typically you need an "options" file which tells Athena what to do. For testing purposes you can use the HelloWorldOptions.py options file which does a short test run of Athena. You can get this file with the get_files command which searches for a specific file in the Atlas builds directories. (Note one stupid feature of this program - it does not inform you if it fails to find a file, you just have to notice that it did not work). Anyway do this:

get_files HelloWorldOptions.py
athena HelloWorldOptions.py

and you get should get various "HelloWorld" messages printed to the screen.

Sample Athena Setup script

Here is a simple script, run_athena.csh, to automate the Athena setup and optionally run Athena. It can also be used as input script for condor. The script must be customized for an individual users setup. This script must be sourced to run Athena interactively.

-- EdwardDiehl - March 7, 2007