Athena Code Checkout

Overview

At Michigan we have Athena "kits" installed which have only Athena binaries located at /afs/atlas.umich.edu/atlas/software/kits . However, it is possible to check out code from the CERN Athena repository which can be compiled and run using the local kit installation. In order to check out code you must access the CVS server at CERN which can be done by using kserver or ssh. Here the kserver method is discussed.

Note that you only need to check out Athena code if you want to modify it and recompile it. If you are running a straight Athena version you only need to source the CMT setup script and the Athena runtime script as discussed in Athena Software Setup at Michigan. Then just run Athena with some options file. In old Athena versions it was common to check out RecExCommon just to run Athena, which still can be done, but is not required.

Preliminaries

You MUST work on an SLC3 machine (only choice: linat01) to checkout code/compile from CERN. Hopefully, this will change in the near future, since Athena 12.0.5 (and 12.3.0) and up should compile on SLC4, but up to now (Jan 2007) I have only gotten checkout and compile to work on linat01. However, after compilation you should be able to run on any of our machines.

Checking out code from CERN requires 2 things:

1. A Kerberos ticket for CERN.CH.
2. Setting environment variable CVSROOT=:kserver:atlas-sw.cern.ch:/atlascvs (which seems to be automatically set on Linat01)

To get cell a Kerberos ticket for CERN.CH use the kinit command as shown below. Note that "CERN.CH" is a Kerberos realm and must be capitalized. The "-4" means to use Kerberos 4 authentication which is what CERN uses.

$ kinit -4 <username>@CERN.CH

After you run this command type klist to see a list of your Kerberos tickets. You might also want to get an AFS token for CERN.CH. With this you can directly copy/examine files in the CERN AFS area. Use command tokens to see a list of your AFS tokens. Do this to get tokens:

$ klog -cell CERN.CH

One-Time Setup.

Set up an Athena working area on an NFS disk, e.g. /atlas/data08. Choose NFS, because if you eventually want to run batch jobs on condor you need to use NFS because condor jobs cannot write to AFS. Ask Bob Ball to create this directory if it does not already exist. Make an Athena working area called "testarea" (the name "testarea" is arbitrary - it can be anything, as long as its reflected in the requirements file).

$ cd /atlas/data08/<username>
$ mkdir testarea
$ cd testarea

Make a directory for a specific Athena release X.Y.Z, say 12.0.4. Here I use the "standard" name of AtlasOffline-X.Y.Z. This is default name for an Athena working directory and allows for a simpler (and static) requirements file. It is possible to use arbitrary names for the working directory, but this requires extra flags in the requirements file. This directory contain subdirectories with the source code you check out, another for libraries you compile from the checked out code, as well as the "run" directory from which you run Athena.

$ mkdir AtlasOffline-12.0.4

Make a "cmthome" directory for requirements file. In the cmthome directory put in the following requirements file:

#
#  Athena Kit Installation Setup  
#
set CMTSITE STANDALONE
set SITEROOT /data07/athena/multi
macro ATLAS_DIST_AREA ${SITEROOT}
#  Edit this line
macro ATLAS_TEST_AREA /atlas/data08/<username>/<working_directory>
use AtlasLogin AtlasLogin-* $(ATLAS_DIST_AREA)

I believe this is the simplest possible requirements file, and it will work for any Athena release. Of course, many additional flags can be used for more sophisticated setups. Now we run cmt to create the setup script needed to configure cmt for using a particular release. Do this:

$ cd cmthome
$ source /data07/athena/multi/cmtsite/setup.csh
$ cmt config

The last command creates setup.csh which you'll use to configure cmt to use a specific release. It also creates cleanup.csh, and Makefile, though I as far as I know these are never used. I believe the requirements file is only used at this step. You'll only need to repeat this step when the CMT version is changed which is infrequent. The requirements file itself should work for any Athena release.

Code Check out

Before checking out (or later on running) code you must source the CMT setup scripts you created in the cmthome directory. These scripts will configure CMT to use a specfic Athena release. Do this:

$ source <path-to-cmthome>/setup.csh -tag=12.0.4

Now check the CMTPATH environment variable. It should look something like this:

$ echo $CMTPATH

/atlas/data08/testarea/AtlasOffline-12.0.4/Control/AthenaExamples/AthExHelloWorld/cmt

CMTPATH tells CMT where to look for source code, in this case, first in your own working directory, and next in the CERN code repository.

Now you are ready to check out code. First "cd" to your working area:

$ cd <working-directory>/AtlasOffline-12.0.4

In order to check out a package you must know the name of the package, optionally its "tag" (version number), and its directory tree in the Athena code repository. If you do not specify the tag you will get the latest version of the package which is not necessarily compatible with the Athena version you are using. The check out command is:

$ cmt co -r <package_name-tag> <directory-tree>

As far as I can tell it is a bit of a pain to figure out all these details. First off there is a command get_tag which can find the "tag" of a given package if you know the directory tree of the package. However, get_tag only works on lxplus and BNL (I am in the process of adding it to UM - E.D.). It works as follows:

$ get_tag < directory-tree> -r < release>

This mode of get_tag finds the tags which goes with a specific Athena release (if you omit the -r < release> part you will get a listing of all versions of the package). Notice that you need to know the directory tree of the package. Here are the ways I know how to figure out the directory tree, none of which is very graceful.

1. Use the lxr source code browser and do a search for the package.
2. Use the CVS browser and poke around.
3. Look in the directory structure of the kits area at UM or builds area as lxplus (/afs/cern.ch/atlas.software/builds)

Here I will check out the HelloWorld package as an example. Looking in LXR I find its directory tree and so I run get_tag on lxplus:

$ get_tag Control/AthenaExamples/AthExHelloWorld -r 12.0.4

2.0.4;AtlasCore;/Control/!AthenaExamples/AthExHelloWorld;AthExHelloWorld-00-01-16

Finally, I check out the package:

$ cmt co -r AthExHelloWorld-00-01-16 Control/AthenaExamples/AthExHelloWorld

# ================= working on package AthExHelloWorld version AthExHelloWorld-00-01-16 path Control/AthenaExamples
in /atlas/data08/diehl/testarea/AtlasOffline-12.0.4/Control/AthenaExamples/AthExHelloWorld
# get top files
cvs update: Updating .
Creating setup scripts.
Creating cleanup scripts.

Now the source code is checked out and so you can modify/compile/run it. First cd down to the bottom of the package directory tree:

$cd Control/!AthenaExamples/!AthExHelloWorld

Here you will find the following directories (plus others not described here):

• src Source code of the package
• cmt Directory in which you compile the code; has a requirements file which tells CMT how to compile the package.
• <packagename> Contains include files of the package.
• share Contains options files which can be used to run the package
• <compiler-options> A directory for the compiled libraries created by gmake (see below). The name is determined by the compiler options e.g. "i686-slc3-gcc323-opt".

First go into src and makes some modifications to HelloWorld.cxx, like adding unclear and ominous error messages. Then go into cmt and do the following to compile the code:

cmt config (possibily unnecessary)
source setup.csh
gmake

This is the standard procedure for compiling any Athena package. After compilation the subdirectory will be created with the shared object library of the compile code in it (lib<packagename>.so) plus a bunch other other files. Athena will now use this private version of the shared object file rather than the default one. Note that Athena does not directly look in this directory, but rather follows the symlinks which are put in

<working-directory>/AtlasOffline-X.Y.Z/InstallArea/<compiler-options>/lib

Running the code

To run the code you need to create a "run" directory for the options file. In old (pre 11 versions, I think) the run directory was always at the bottom of the package directory tree, parallel to src, and cmt directories mentioned above. However, the "run" directory can be anywhere at all and can have any name. Really a "run" directory is just a place to park your Athena options file. I like to make a run directory off my working directory like so:

mkdir <working-directory>/AtlasOffline-12.0.4/run
cd <working-directory>/AtlasOffline-12.0.4/run

Now you need an Athena options file. Packages typically come with options file which are needed by the package (some packages only have options, and no source code). These are put in the package share directory, and also referred to by symlinks in directory:

<working-directory>/AtlasOffline-12.0.4/InstallArea/jobOptions/<package>

However, these are the "low-level" options files which set defaults on all the options (called "properties" in Athena jargon) which apply to each package. There is also the directory

<working-directory>/AtlasOffline-12.0.4/InstallArea/share;

which seems to have "top-level" options

-- EdwardDiehl - 18 Dec 2006