Creating a gbasf2 (gb2) tool¶

Lets create a gb2_tool.
- Name of the tool: gb2_test_tool
- Funtionality of the tool:

Get Replicas from an LFN/LPN:

$ gb2_test_tool --get_repicas <LFN/LPN>

Get the Storage Element status from the Resource Status System (RSS):

gb2_test_tool --se_status <se_name>

- Replica information is stored in the File Catalog (FC). Currently we use the Rucio FC.

The connection to the FC goes via DIRAC through FC

- Metadata information are stored in Metadata Catalog. Currently we use AMGA as metadata catalog.

- Status of storage Element is store in RSS. We have four status of SE.

ReadAccess : SE availability for Read query.

WriteAcess : SE availability for Write query.

RemoveAccess : SE availability for Remove (delete) query.

CheckAccess : SE availability for check query.

Code¶

Now lets start Coding.

1st we need to comunicate to relevant system/resources. This comminucation must go through manager. In directory gbasf2/lib you will see differnt folder like ds, job, prod e.t.c. Each of which contains manager.py in it.

Create a directory test inside gbasf2/lib

$ cd BelleDIRAC/gbasf2/lib
$ mkdir test
$ cd test

Create files name manager.py and __init__.py under BelleDIRAC/gbasf2/lib/test
```
$ touch manager.py
$ touch __init__.py
```

Add following lines of code in manager.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-
from DIRAC import S_OK, S_ERROR
from DIRAC.Resources.Catalog.FileCatalog import FileCatalog
from DIRAC.ResourceStatusSystem.Client.ResourceStatus import ResourceStatus

class Manager(object):
    def __init__(self, fileCatalog=None , resourceStatus=None):

        self.__fileCatalog = fileCatalog
        if not self.__fileCatalog:
            self.__fileCatalog = FileCatalog()

        self.__resourceStatus = resourceStatus
        if not self.__resourceStatus:
            self.__resourceStatus = ResourceStatus()

    def getReplicas(self, lpns):
        """ Get replicas of LFNs/LPNs from FC.

        :param list lpns: [lpn1, lpn2]

        :return dict: S_OK()/S_ERROR()
        """
        # getReplicas in FC provides the info we need
        ret = self.__fileCatalog.getReplicas(lpns)
        if not ret['OK']:
            # ret = {'OK': False, 'Message': 'ssss'}
            return ret
        # ret = {'OK': True , 'Value': {'Successful': {'lpn': ''}, 'Failed': {'lpn': ''}}}}
        return ret

    def getSeStatus(self, ses):
        """ get Storage Element status from RSS.

        :param list ses : [se1 , se2]

        :return dict : S_OK()/S_ERROR()
        """
        # getElementStatus in resourceStatus gives staus of ses.
        ret = self.__resourceStatus.getElementStatus(ses,"StorageElement")
        if not ret['OK']:
            return ret
        return ret

Lets make the args options --get_replicas and --se_status.
Generally we recycle the args options between different gb2_tools.

These are available in BelleDIRAC/Client/helpers/optutil.py
- Add the following line of code to BelleDIRAC/Client/helpers/optutil.py .
  def addLfns(self): self._parser.add_argument('--get_replicas', type=str, help = 'LFN/LPN') def addSeStatus(self): self._parser.add_argument('--se_status', type=str, help = 'Name of SE')

lets create a controller for our gb2_test_tool .

Controller: This is where you manage commandline input, call appropriate functions from Manager and layout the ouput structure.

In the directory BelleDIRAC/Client/controllers , you will see different type of controllers like datasetCLController/projectCLController e.t.c

Lets make a new controller called testCLController.

$ cd BelleDIRAC/Client/controllers
$ touch testCLController.py

Add following line of code in testCLController.py.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
from DIRAC import S_OK, S_ERROR, exit as DIRACExit
#import the class Manager that we made in gbasf2/lib/test/Manager.py
from BelleDIRAC.gbasf2.lib.test.manager import Manager
from BelleDIRAC.Client.helpers.auth import userCreds

#@userCreds
def test(args=None):
    """ Controller for test command

    :param args : args.get_replicas or args.se_status
    """

    #lets make sure at least one the args options exist.
    if not (args.get_replicas or args.se_status):
        print('One of --get_replicas' or '--se_status is required.')
        #exit with a exit code 1.
        DIRACExit(1)

    #initialize Manager
    manager = Manager()
    if args.get_replicas:
        #call the getReplicas from test/Manager.
        lfn = args.get_replicas
        res = manager.getReplicas([lfn])

        # check for error return
        if not res['OK']:
            #return res Message.
            return res['Message']
        print(res['Value'])

    if ags.se_status:
        se = args.se_status
        res = manager.getSeStatus([se])
        if not res['OK']:
            return res['Message']
        print(res['Value'])
    # we are here means all OK. Now just exit.
    DIRACExit(0)

Putting it together as gb2_script.

gb2_script located in BelleDIRAC/Client/gb2_scripts contains script for each of the gb2 commands.

Create a file name same as tool’s name in BelleDIRAC/Client/gb2_scripts/
```
$ touch gb2_test_tool.py
```

Add Following line of code in gb2_test_tool.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-

"""
Some helpul text about the command.
This goes in option --usages of the command.
Examples::

$ gb2_test_tool --get_replicas <LFN>
$ gb2_test_tool --se_status DESY-TMP-SE
"""

import argparse
from BelleDIRAC.Client.helpers.common import setLogger
from BelleDIRAC.Client.helpers.optutil import OptUtil
from BelleDIRAC.Client.controllers.testCLController import test

def getOpt():
    parser = argparse.ArgumentParser()
    option = OptUtil(parser)
    option.addUsage(__doc__)
    # add relevant options of the tool
    option.addLfns()
    option.addSeStatus()
    return parser


if __name__ == "__main__":
    args = getOpt().parse_args()
    setLogger(args.verbose)
    print(test(args))

Lets create a excutable for the command.
Now we create excutable for the command. These are created in BelleDIRAC/Client/bin or BelleDIRAC/Client/pbin or BelleDIRAC/Client/sbin depending upon the fucntionality of command.
- Create a file name gb2_test_tool in ``BelleDIRAC/Client/bin/
  $ cd BelleDIRAC/Client/bin/ $ touch gb2_test_tool $ chmod 777 gb2_test_tool
- Add the Following line of code to the file gb2_test_tool
  #!/bin/bash exec /usr/bin/env python $DIRAC/BelleDIRAC/Client/gb2_scripts/gb2_test_tool.py "$@"

Finnaly lets run the script:

To run the command we need to source the environment and get the proxy.
```
$ source BelleDIRAC/gbasf2/tools/setup.sh && gb2_proxy_init -g belle
```

Run the gb2_test_tool

$ gb2_test_tool --get_replicas /belle/Data/proc/release-04-02-04/DB00000498/bucket11/prod00013724/e0012/4S/r03306/mdst/sub00
$ gb2_test_tool --se_status DESY-TMP-SE