Integration within the EGEE infrastructure

GridWay is an open-source component for meta-scheduling in the Grid Ecosystem. GridWay gives end users, application developers and managers of Grid infrastructures a scheduling functionality similar to that found on local DRM (Distributed Resource Management) systems.

GridWay is completely functional on EGEE, being able to interface with the computing, file transferring and information services available within the EGEE infrastructure. GridWay does not intend to substitute the resource brokers available in the EGEE distribution (gLite-WMS and LCG-RB), but to provide a meta-scheduling alternative with greater functionality and higher performance for given application profiles. GridWay is being successfully used in the NA4 Fusion activities. GridWay is being tested under ETICS, the eInfraestructure for Testing, Integration and Configuration of Software of the EGEE project.

This section of the GridWay web page only provides relevant information about installing and configuring GridWay on the EGEE infrastructure. For general information about GridWay, please refer to the GridWay main page.

The figure shows the integration of GridWay with the EGEE infrastructure services available in the different distributions:

• LCG2
• gLite
• Future gLite

Figure 1-1. Integration of GridWay with the EGEE services.

GridWay provides a meta-scheduling alternative to the resource brokers available in the EGEE distribution (gLite-WMS and LCG-RB) for users, application developers and grid managers interested in the following features:

Table 1-1. Features and benefits.

This installation requires a previous Scientific Linux installation and the User Interface configuration. GridWay installation has been tested under ETICS using:

• Scientific Linux CERN Release 3 and 4
• gLite 3.0.2

The GridWay Installation and Configuration Guide provides complete information about installation of the meta-scheduler. Next steps illustrate specific aspects for installation on an EGEE UI:

• Before unpacking the distribution, in order to prepare the environment, execute (as root) the following:

 # ${GPT_LOCATION}/sbin/gpt-build -force -nosrc -builddir=/tmp/globus_core.$$ \ 
 > gcc32 gcc32pthr gcc32dbg gcc32dbgpthr 

• When invoking the configure command, make sure you add the option “–disable-ws”.
• Compiling GridWay in an EGEE UI had some problems in the past as the gridftp implementation installed in this machines lacked some functions needed by the gridftp transfer mad used by GridWay. It also lacks Globus WS libray support (GT4) so Web Services execution mad can not be built here. Fortunately these mads are not used to interact with EGEE resources as home directories in this clusters are not shared and WN does not necessarily need to have external connection, Pre Web Services execution mad is used to talk to CE's and the dummy transfer mad is used to transfer files. You can now disable gridftp mad building using “–disable-gridftp” and ws mad with “–disable-ws” so you can compile GridWay in an EGEE UI and use it to manage EGEE resources. The only caveat is that you can not use gridftp transfer mad with other non EGEE resources or interact with Web Services enabled resources. If you want to have this functionality you have to install GT4 by yourself and use this installation to compile and run GridWay.
• For a multi-user installation, the “sudoers” file of the sudo command must include also the dummy TM MAD:

  gwadmin ALL=(GW_USERS) NOPASSWD: $GW_LOCATION/bin/gw_tm_mad_dummy *

The GridWay Installation and Configuration guides provides complete information about configuration of the meta-scheduler. Next steps describe the specific configuration of the access drivers for the EGEE infrastructure services (LCG).

In file “$GW_LOCATION/etc/gwd.conf”:

  IM_MAD = mds2_glue:gw_im_mad_mds2_glue:-l etc/im_examples/host_glue.list:dummy:prews_nsh
  EM_MAD = prews_nsh:gw_em_mad_prews::rsl_nsh
  TM_MAD = dummy:gw_tm_mad_dummy:-g

There are two options for the configuration of the TM MAD:

• “-g”: starts a GASS server.
• “-u”: specifies the URL of a GridFTP server running in the client. For example:

   TM_MAD = dummy:gw_tm_mad_dummy:-u gsiftp\://cepheus.dacya.ucm.es 

  (notice the escaped “:”).

There are two options for VO management:

• It is possible to configure a GridWay instance to only use those queues authorized to your VO by filtering them. This is done with option “-q” of the IM MAD. For example (in “gwd.conf”):

  IM_MAD = mds2_glue:gw_im_mad_mds2_glue:-l etc/im_examples/host_glue.list
  -q (GlueCEAccessControlBaseRule=VO\:fusion):dummy:prews_nsh

  Change “fusion” with the VO you belong to.

• The other possibility is to use the parameter QUEUE_ACCESS in the requirement expression of a job. This parameter is set to the list of authorized VOs, each one between colons:

  QUEUE_ACCESS = ":fusion::dteam:"

  For example (in the Job Template or job_template.default):

  REQUIREMENTS = QUEUE_ACCESS = "*:fusion:*"

  to only use those queues authorized for “fusion” VO, or

  REQUIREMENTS = QUEUE_ACCESS = "" | QUEUE_ACCESS = "*:fusion:*"

  if you also want to use those queues with no access restrictions.

• Warning: Make sure of using at least one of the above control access alternatives in order to not cause a lot of authorization failures.

Even if this aspect falls out of the GridWay configuration, in order to use the EGEE resources, the user should iniatilize the proxy signed through the appropriate voms server.

voms-proxy-init --voms YOUR_VO

Note: In case you have no knowledge of the vomses for the EGEE VOs and you get the following info:

$ voms-proxy-init -voms YOUR_VO
Cannot find file or dir: /opt/glite/etc/vomses

The functionality and interfaces provided by GridWay are independent from the underlying Grid infrastructure. Please refer to the guides (user's guide, command reference, programming guide…) available at the Documentation section.