How To Add New Pools to Dcache

This page is obsolete for dCache/Chimera. See HowToAddNewPoolsToChimera instead.

On our existing dcache system ,I add some New pools to it..All the new pools are distributed on the new computer nodes.Each new computer node devote one or tow of its harddisk partition to the pool space.. On each node,there are a partition such as /dcache (only one pool on this node), or /dcache and /dcache1 (tow pools on this node)..these nodes are all installed with the dcache-server rpm..and configured as the pools of dcache..what we need to do is to put the pools information on the headnodes, which make the headnodes being aware of the existence of new added pools and manage these pools.Flowing jobs should be done to the new nodes..

1. Modify site-info.def on umopt1.

1-1) adding new pools information to site-info.def, take node c-5-20 for example, add the corresponding entries:

## if you don't want any of the following services(GSIFTP,GSIDCAP,DCAP,POOLS) to be activated on this node, just uncomment the corresponding entries.

DCACHE_DOOR_GSIFTP=c-5-20.aglt2.org
DCACHE_DOOR_GSIDCAP=c-5-20.aglt2.org
DCACHE_DOOR_DCAP=c-5-20.aglt2.org
DCACHE_POOLS=c-5-20.aglt2.org:all:/dcache c-5-20.aglt2.org:all:/dcache1

1-2) overwrite /home/install/extras/site-info.def with the new site-info.def

(resouces: to avoid adding all the nodes information manually , there are some scripts to do this work..Jut prepare a list of hostnames of newnodes and the original site-info.def, then run the mod_site_info.pl ,we will get all the corresponding information added to the new site-info.def.. scripts are located on umopt1: /home/install/extras/add_newnodes_2_dcache/mod_site_info/)

2. add new pool node to export list on head02..(this is neccessary when the new pools need to mount the pnfs on head02..)

for each node(take c-5-20 for example), touch a newfile whose name is the IP address of the node(192.41.230.62),and add an entry on this file.

root@head02 /pnfs/fs/admin/etc/exports# more 192.41.230.62
/pnfsdoors    /0/root/fs/usr           0 nooptions

(resources: script to do this job automatically ,umopt1: /home/install/wuwj_extras/add_newnodes_2_dcache/trust_nodes/head2trustip.pl , which takes new_node_ip as the input..)

3 .If this node has dcap doors ,add it to dcache.conf on head02.

(this is optional, but on doing this, you can run dccp command without specify the dcap door endpoint..the dcap door are appointed from this file..eg, you can run dccp /pnfs/aglt2.org/../../somefile /tmp/somefile instead of dccp dcap://head01.aglt2.org:22125/pnfs/aglt2.org/../../somefile /tmp/somefile .. in the first pattern, dcache will choose a proper dcap door for the request.)

3-1) edit /pnfs/fs/admin/etc/config/dCache/dcache.conf

(Warn: this file can not be writen, you need to touch a new file, then cp the new file to /pnfs/fs/admin/etc/config/dCache/dcache.conf to overwrite it.)

adding the corresponding entries.(take c-5-20 for example)

c-5-20.aglt2.org:22136
c-5-20.aglt2.org:22137
c-5-20.aglt2.org:22138
c-5-20.aglt2.org:22139
c-5-20.aglt2.org:22140

(resources: script to do this work --umopt1 :/home/install/wuwj_extras/add_newnodes_2_dcache/add_dcap_door/add_dcap_door.pl)

4. changes on PoolManager

4-1). modify the PoolManager configuration file on head01

edit /opt/d-cache/config/PoolManager.conf on head01

For each new nodes(we have tow pools on each node), take node c-5-20 for example

Add the following entry:

psu create pool c-5-20_1
psu create pool c-5-20_2
psu addto pgroup ResilientPools c-5-20_1
psu addto pgroup default c-5-20_2

In our case, we have tow kinds of pools, resilient pools whose group name is ResilientPools, which means files stored in these pools have 2-3 copies for each.. the other is Non-resilient pools whose group name is default, which means there is only one copy for each file stored in these pools.For thoese nodes which have tow pools, we set one to be resilient pool ,and the other to be non-resilient pool.

(resources:Also there is a script to generate a pool list information , umopt1: /home/install/extras/add_newnodes_2_dcache/mod_site_info/genpoollist.pl)

4-2) run the script install the rpm on the node or install dcache via rocks building appliance.

on the node, run

 /home/install/extras/bin/dcache/ install_dcache_manual.sh 

Note:: this script would install and configure dcache on this node, but it wouldnot start dcache if the /etc/grid-security/hostcert.pem and hostkey.pem are not present..

on the rocks head node (either umopt1.aglt2.org for UM , or msurox.aglt2.org for MSU), run

perl /home/install/extras/install_hostcert.pl c-5-20

Note this would install the GSI related files in /etc/grid-security and start dcache service on the node.. -- WenjingWu - 22 Oct 2007