Copying files to/from CERN

AFS copying from AFS with 'cp'

If you get AFS tokens (with klog -cell CERN.CH) you can copy files directly from CERN AFS space with speeds around 70 kB/s. Example: cp /afs/cern.ch/user/d/diehl/Comp/12.0.4/calib/CalibrationNtuple.root .

Copying with sftp

With sftp you can get copy rates of around 500 kB/s between lxplus/Mars and UM, and 40 MB/s between Mars and lxplus. For large file transfers between lxplus/UM FDT is recommended.

Copying with FDT

FDT is Fast Data Transfer from the Mona Lisa project. With FDT you can copy with rates of 500 - 5000 kB/s (possibly faster if you really know what you are doing). FDT is a java program and currently (Feb 2007) java does not work on lxplus (64 bit SLC4). To run FDT at CERN use lx32slc4.cern.ch (32 bit SLC4). You can also run FDT on the UM Mars machines, but only in client mode. For some reason (probably CERN firewalls) an FDT server run on MARS cannot be contacted by outside clients.

FDT Installation

First install the software on both computers (soon we should install FDT to a common location on both computers so the user won't have install FDT)

mkdir ~/fdt
cd ~fdt
wget http://monalisa.cern.ch/FDT/lib/fdt.jar
wget http://monalisa.cern.ch/FDT/scripts/fdtServer.sh
wget http://monalisa.cern.ch/FDT/scripts/fdtClient.sh
chmod u+x fdtClient.sh
chmod u+x fdtServer.sh

Copying files with FDT.

FDT has a client-server model. You run a "server" on one machine and a "client" on the other. It does not matter which machine is client or server - you can copy both directions it either case. Start the server on UM machine (e.g. umt3int01):

~/fdt/fdtServer.sh

Run the client at CERN. You can copy either direction:

FDT Copy from CERN to UM

syntax:

./fdtClient.sh -c <fdt_server> <lxplus_file> -d <destination_dir>

example (copy CERN file /tmp/diehl/file.root to UM directory /atlas/data08/diehl/):

./fdtClient.sh -c umt3int01.physics.lsa.umich.edu  /tmp/diehl/file.root -d /atlas/data08/diehl

FDT Copy from UM to CERN (you must "pull" it)

./fdtClient.sh -c <fdt_server> -pull <um_file> -d <destination_dir>

example (copy UM file /atlas/data08/diehl/fred.dat to CERN /tmp/diehl):

./fdtClient.sh -c umt3int01.physics.lsa.umich.edu  /atlas/data08/diehl/fred.dat  -d /tmp/diehl

It would be wonderful if FDT could be used to copy files directly to/from CASTOR at CERN. As far as I know this is not possible since CASTOR must be accessed with special "rf" commands (e.g. rfdir, rfcp) which can only be run from the command line.

FDT copying lists of files

In the above examples you can substitute the <source_file> with list of files via the -fl <filelist> flag, where <filelist> is a list of files to copy. For example to copy a list of files from UM to CERN (running client at CERN):

./fdtClient.sh -c umt3int01.physics.lsa.umich.edu -pull -fl list.txt -d /tmp/diehl

where list.txt is:

/atlas/data08/diehl/sector13/CalibNtuple_12.0.4_100330.root
/atlas/data08/diehl/sector13/CalibrationNtuple.root

This command copies both files to CERN /tmp/diehl, in but subdirectories according to the full path of the files in the list. That is to say, the copied files will end up in /tmp/diehl/atlas/data08/diehl/sector13/.

FDT copying speed

As shown above FDT will copy at the same speed as sftp, about 500 kB/s. To go faster do add flags "-P >n< -ss >m<M" to specify using "n" TCP streams and a buffer size of "m" megabytes. For example using "-P 5 -ss 2M" I was able to copy at 4 MB/s.

FDT Errors

FDT does not always give clear error messages. Sometimes it will claim to be copying, but at 0 MB/s. If this happens, you probably have a typo either the file name or destination directory and FDT is not actually doing anything. Kill FDT with ^C and try again.

Copy with Grid tools

(..add later)

-- EdwardDiehl - 16 Jan 2007