| Home | About | Software | Documentation | Support | Outreach | Ecosystem | Dev | Awards | Team & Sponsors | |
Integration within the OSG infrastructure
Scheduling grid infrastructure on OSG
The figure shows the integration of GridWay with the OSG infrastructure services available in the different distributions:
- OSG v.0.4.0 (pre WS)
- OSG v.0.6.0 (WS)
Figure 1-1. Integration of GridWay with the OSG services.
Features and benefits on the OSG infrastructure
GridWay provides a meta-scheduling solution for users, application developers and grid managers interested in the following features:
Table 1-1. Features and benefits.
| Features | Benefits |
|---|---|
| Support for DRMAA standard (C and JAVA bindings) | Compatibility of applications with DRM systems that implement the standard, such as SGE, Condor, Torque,.. |
| DRM Command Line Interface | Command Line Interface similar to that provided by local resource managers to submit, kill, migrate, monitor and synchronize single, array and interdependent jobs) |
| Lightweight middleware | Higher efficiencies for given application profiles as GridWay reduces the number of submission stages and provides mechanisms, such as opportunistic migration and performance slowdown detection, that improve the usage of the underlying resources |
| Site-level accounting and scheduling policies | Analysis of resource utilization, determining trends in usage and monitoring user behavior |
| Installation minimum requirements | Easy and fast deployment and maintenance on a wide variety of platforms |
| Interoperability | Simultaneous access to different infrastructures: GT WS, pre-WS, LCG, gLite… |
Installation of GridWay
You have to make sure that you compile GridWay with support for ws and/or prews. This will depend on which OSG version the resources you are planning to interact with have installed.
The GridWay Installation and Configuration Guide provides complete information about installation of the meta-scheduler.
Configuration to access to OSG resources
The GridWay Installation and Configuration Guide provides complete information about configuration of the meta-scheduler. Next steps describe the specific configuration of the access drivers for the OSG infrastructure services.
Right now there are two versions of OSG middleware being used. There is version 0.4.0 that uses pre web services Gatekeeper and mds2 information provider and a newer one 0.6.0 that uses web services and provides mds4 information.
Pre-Web Services Configuration
Unfortunately the information provided by mds2 in this middleware version does not contain queue data and it is insufficient for gridway to be usable. You can instead create static information for this nodes. The way to do this is to create a file for each of the nodes you want to configure with the needed information for the scheduler. This is an example of one of this files:
-> HOSTNAME="somehost.com" ARCH="i686" OS_NAME="Linux" OS_VERSION="2.4.21-32.ELsmp"
CPU_MODEL="Intel(R) Xeon(TM) CPU 2" CPU_MHZ=2665 CPU_FREE=189 CPU_SMP=2 NODECOUNT=1
SIZE_MEM_MB=2006 FREE_MEM_MB=964 SIZE_DISK_MB=73964 FREE_DISK_MB=62787
FORK_NAME="jobmanager-fork" LRMS_NAME="jobmanager-fork" LRMS_TYPE="fork"
-> QUEUE_NAME[0]="default" QUEUE_NODECOUNT[0]=1 QUEUE_FREENODECOUNT[0]=1 QUEUE_MAXTIME[0]=0
QUEUE_MAXCPUTIME[0]=0 QUEUE_MAXCOUNT[0]=1 QUEUE_MAXRUNNINGJOBS[0]=1
QUEUE_MAXJOBSINQUEUE[0]=0 QUEUE_STATUS[0]="0" QUEUE_DISPATCHTYPE[0]="Immediate"
This file consists in two lines, one for the host general info and other for queue info. For clarity I've put the example in various lines and arrows indicating where a new line starts, you should delete the arrows. If you are adding more queues you have to add more lines, one for each queue. Variables will be named like “QUEUE_NAME[1]”, “QUEUE_NAME[2]” and so on.
Be careful with the name of the queues and node count.
Afterwards you will need to create another file with each line consisting in a pair of resource name and information file, the one previously described. This is an example of one of this files:
somehost.com etc/osg/somehost.com.attr otherhost.com etc/osg/otherhost.com.attr
The path to the attribute files can be a full path or relative to “GW_LOCATION”.
In “gwd.conf” you can use this lines as an example to add this resources:
IM_MAD = osg_mds2:gw_im_mad_static:-l etc/osg_prews.list:gridftp:osg_prews EM_MAD = osg_prews:gw_em_mad_prews::rsl TM_MAD = gridftp:gw_tm_mad_ftp:
Change the name of the files so they match the ones you have created. You may also change the name of the mads to the name you want.
Web Services Configuration
To configure WS OSG Resources you only need to have a file with a list of hosts you want to access, the information with this version of OSG is enough for gridway to schedule and sends jobs. A typical list can be like this one:
host1.domain.org host2.domain.org
And in gwd.conf you have to add these lines:
IM_MAD = osg_mds4:gw_im_mad_mds4:-l etc/osg_ws.list -p 9443:gridftp:osg_ws EM_MAD = osg_ws:gw_em_mad_ws:-p 9443:rsl2 TM_MAD = gridftp:gw_tm_mad_ftp:
Note that a parameter is added to IM mad and EM mad (-p 9443). This parameter is needed as the Grid services are not located at the standard port (8443), so here we are telling to connect to port 9443 instead.
Using GridWay
The functionality and interfaces provided by GridWay are independent from the underlying Grid infrastructure. Please refer to the guides (user's guide, command reference, programming guide…) available at the Documentation section.