====== Integration within the EGEE infrastructure======

===== Introduction =====


GridWay is an open-source component for meta-scheduling in the Grid Ecosystem. GridWay gives end users, application developers and managers of Grid infrastructures a [[http://www.gridway.org/about/functionality.php| scheduling functionality]] similar to that found on local DRM (Distributed Resource Management) systems.

GridWay is completely functional on EGEE, being able to interface with the computing, file transferring and information services available within the EGEE infrastructure. GridWay does not intend to substitute the resource brokers available in the EGEE distribution (gLite-WMS and LCG-RB), but to provide a meta-scheduling alternative with greater functionality and higher performance for given application profiles. GridWay is being successfully used in the [[http://grid.bifi.unizar.es/egee/fusion-vo/contact.html|NA4 Fusion activities]]. GridWay is being tested under [[http://etics.web.cern.ch|ETICS]], the eInfraestructure for Testing, Integration and Configuration of Software of the EGEE project.

This section of the GridWay web page only provides relevant information about installing and configuring GridWay on the EGEE infrastructure. For general information about GridWay, please refer to the [[http://www.gridway.org|GridWay main page]].

===== Scheduling grid infrastructure on EGEE =====

The figure shows the integration of GridWay with the EGEE infrastructure services available in the different distributions:

  * LCG2
  * gLite
  * Future gLite

**Figure 1-1. Integration of GridWay with the EGEE services.**

{{ documentation:howto:egee.png }}


===== Features and benefits on the EGEE infrastructure =====

GridWay provides a meta-scheduling alternative to the resource brokers available in the EGEE distribution (gLite-WMS and LCG-RB) for users, application developers and grid managers interested in the following features:

**Table 1-1. Features and benefits.**

^ Features ^ Benefits ^
| **Support for DRMAA standard (C and JAVA bindings)** | Compatibility of applications with DRM systems that implement the standard, such as SGE, Condor, Torque, ... |
| **DRM Command Line Interface** | Command Line Interface similar to that provided by local resource managers to submit, kill, migrate, monitor and synchronize single, array and interdependent jobs) |
| **Lightweight middleware** | Higher efficiencies for given application profiles as GridWay reduces the number of submission stages and provides mechanisms, such as opportunistic migration and performance slowdown detection, that improve the usage of the underlying resources |
| **Site-level accounting and scheduling policies** | Analysis of resource utilization, determining trends in usage and monitoring user behavior |
| **Installation minimum requirements** | Easy and fast deployment and maintenance on a wide variety of platforms |
| **Interoperability** | Simultaneous access to different infrastructures: GT WS, pre-WS, LCG, gLite... |


===== Installation on an EGEE UI =====

This installation requires a previous [[https://www.scientificlinux.org/|Scientific Linux]] installation and the [[http://glite.web.cern.ch/glite/packages/R3.0/deployment/glite-     UI/glite-UI.asp|User Interface configuration]]. GridWay installation has been tested under ETICS using:

  * Scientific Linux CERN Release 3 and 4
  * gLite 3.0.2

The [[http://www.gridway.org/documentation/guides.php|GridWay Installation and Configuration Guide]] provides complete information about installation of the meta-scheduler. Next steps illustrate specific aspects for installation on an EGEE UI:

  * Before unpacking the distribution, in order to prepare the environment, execute (as root) the following:  
 <xterm> # ${GPT_LOCATION}/sbin/gpt-build -force -nosrc -builddir=/tmp/globus_core.$$ \ 
 > gcc32 gcc32pthr gcc32dbg gcc32dbgpthr </xterm> 
This step generates files "globus-build-env-$GLOBUS_FLAVOR_NAME.sh" and "globus_config.h".
  * When invoking the **configure** command, make sure you add the option "--disable-ws".
  * Compiling GridWay in an EGEE UI had some problems in the past as the gridftp implementation installed in this machines lacked some functions needed by the gridftp transfer mad used by GridWay. It also lacks Globus WS libray support (GT4) so Web Services execution mad can not be built here. Fortunately these mads are not used to interact with EGEE resources as home directories in this clusters are not shared and WN does not necessarily need to have external connection, Pre Web Services execution mad is used to talk to CE's and the dummy transfer mad is used to transfer files. You can now disable gridftp mad building using "--disable-gridftp" and ws mad with "--disable-ws" so you can compile GridWay in an EGEE UI and use it to manage EGEE resources. The only caveat is that you can not use gridftp transfer mad with other non EGEE resources or interact with Web Services enabled resources. If you want to have this functionality you have to install GT4 by yourself and use this installation to compile and run GridWay.
  * For a multi-user installation, the "sudoers" file of the **sudo** command must include also the dummy TM MAD:<file>gwadmin ALL=(GW_USERS) NOPASSWD: $GW_LOCATION/bin/gw_tm_mad_dummy *</file>



===== Configuration to access to EGEE resources =====

The GridWay [[documentation:release_5.4:ig| Installation]] and [[documentation:release_5.4:ig|Configuration]] guides provides complete information about configuration of the meta-scheduler. Next steps describe the specific configuration of the access drivers for the EGEE infrastructure services (LCG).

In file "$GW_LOCATION/etc/gwd.conf":
<file>
  IM_MAD = mds2_glue:gw_im_mad_mds2_glue:-l etc/im_examples/host_glue.list:dummy:prews_nsh
  EM_MAD = prews_nsh:gw_em_mad_prews::rsl_nsh
  TM_MAD = dummy:gw_tm_mad_dummy:-g
</file>
There are two options for the configuration of the TM MAD:
  * "-g": starts a GASS server.
  * "-u": specifies the URL of a GridFTP server running in the client. For example: <file> TM_MAD = dummy:gw_tm_mad_dummy:-u gsiftp\://cepheus.dacya.ucm.es </file> (notice the escaped ":").

There are two options for VO management:

  * It is possible to configure a GridWay instance to only use those queues authorized to your VO by filtering them. This is done with option "-q" of the IM MAD. For example (in "gwd.conf"):<file>IM_MAD = mds2_glue:gw_im_mad_mds2_glue:-l etc/im_examples/host_glue.list
-q (GlueCEAccessControlBaseRule=VO\:fusion):dummy:prews_nsh</file>Change "fusion" with the VO you belong to.
  * The other possibility is to use the parameter QUEUE_ACCESS in the requirement expression of a job. This parameter is set to the list of authorized VOs, each one between colons:<file>QUEUE_ACCESS = ":fusion::dteam:"</file>For example (in the Job Template or job_template.default):<file>REQUIREMENTS = QUEUE_ACCESS = "*:fusion:*"</file>to only use those queues authorized for "fusion" VO, or<file>REQUIREMENTS = QUEUE_ACCESS = "" | QUEUE_ACCESS = "*:fusion:*"</file>if you also want to use those queues with no access restrictions.

   * **Warning**: Make sure of using at least one of the above control access alternatives in order to not cause a lot of authorization failures. 


===== Accessing the VOMS servers =====

Even if this aspect falls out of the GridWay configuration, in order to use the EGEE resources, the user should iniatilize the proxy signed through the appropriate voms server.
<xterm>
voms-proxy-init --voms YOUR_VO
</xterm>


**Note:** In case you have no knowledge of the vomses for the EGEE VOs and you get the following info:
<xterm>
$ voms-proxy-init -voms YOUR_VO
Cannot find file or dir: /opt/glite/etc/vomses
</xterm>
then insert [[documentation:howto:all_vo|this file]] into the directory ''/opt/glite/etc/vomses'' or $HOME/.glite/vomses

===== Using GridWay =====

The functionality and interfaces provided by GridWay are independent from the underlying Grid infrastructure. Please refer to the guides (user's guide, command reference, programming guide...) available at the [[:documentation|Documentation]] section.