Virtual appliance in Amazon EC2

Using GridWay as a front-end to your cluster, grid-resources, etc. requires an installation and set-up which could keep apart users wanting to test specific features. It also expects some administrator knowledge to follow up the detailed documentation.

The virtual appliance in the most successful cloud provider Amazon EC2, consists of two public images (AMIs). The advantage of cloud computing resides on the possibility of launching on demand a working instance of GridWay 5.5.0 within just a couple of minutes.

On the next sections it is explained, step by step, how to get it working.

Select one of the public GridWay AMIs from the next Table:

Create a security group with all the ports open. Otherwise you should set the GLOBUS_TCP_PORT_RANGE according to your selection.

After setting your key pair and choosing the instance type you are ready to go.

According to your key pair and public-DNS of your running instance, you can get in by ssh:

ssh -i YOUR-PRIVATE-RSA-KEY.pem root@YOUR-INSTANCE-PUBLIC-DNS
The authenticity of host 'ec2-79-125-53-164.eu-west-1.compute.amazonaws.com (79.125.53.164)' can't be established.
RSA key fingerprint is 0d:4e:3e:51:0e:b8:c5:21:26:6f:a4:ce:06:63:df:33.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'ec2-79-125-53-164.eu-west-1.compute.amazonaws.com,79.125.53.164' (RSA) to the list of known hosts.
Linux ec2-79-125-53-164 2.6.21.7-2.fc8xen #1 SMP Fri Feb 15 12:39:36 EST 2008 i686
    ______      __               __  __            ______     _     __
   / ____/___  / /____  _____   / /_/ /_  ___     / ____/____(_)___/ /
  / __/ / __ \/ __/ _ \/ ___/  / __/ __ \/ _ \   / / __/ ___/ / __  /
 / /___/ / / / /_/  __/ /     / /_/ / / /  __/  / /_/ / /  / / /_/ /
/_____/_/ /_/\__/\___/_/      \__/_/ /_/\___/   \____/_/  /_/\____/

GridWay 5.5.0 AMI, built by Alejandro Lorca, based on the Amazon EC2 
Ubuntu 9.04 jaunty AMI built by Eric Hammond.
Distributed System Architecture group, Universidad Complutense de Madrid.
Please check http://gridway.org and http://dsa-research.org for more info.
--------------------------------------------------------------------------
root@ec2-79-125-53-164:~#

Now you are up and running, but you need to adapt the GridWay metascheduler to your needs. By default, GridWay is configured to interact with the training infrastructure Gilda within the EGEE project.

The virtual appliance is configured such that it has a web server running with a working installation of the great project GridWay + Google Maps web interface. You can check the connection to the instance with your favourite browser http://YOUR-INSTANCE-PUBLIC-DNS and going to the HOST tab of the application. It should look like this figure:

Use the default GridWay user gwuser (pass gwuser) only for testing. Should you delete this user, then you won't be able to retrieve the information from the web interface (you can safely lock this account as administrator). This default user has access to standard GridWay usage:

root@ec2-79-125-53-164:~# su - gwuser
gwuser@ec2-79-125-53-164:~$ gwhost
HID PRIO OS              ARCH  MHZ  %CPU    MEM(F/T)     DISK(F/T)      N(U/F/T) LRMS               HOSTNAME                      
0   1    ScientificCERNS i686  2660    0 16384/16384           0/0         0/0/0 jobmanager-pbs     ce-edu.grid.acad.bg           
1   1    ScientificSLBer i686  3001    0   2048/2048           0/0        0/4/20 jobmanager-lcgpbs  ce.hpc.iit.bme.hu             
2   1    CentOSFinal     i686  2193    0   1026/1026           0/0       0/0/104 jobmanager-sge     ce1-egee.srce.hr              
3   1    ScientificSLBer i686  1400    0   2048/2048           0/0       0/24/24 jobmanager-lcgpbs  grid010.ct.infn.it            
4   1    ScientificCERNS i686  1001    0     513/513           0/0       0/14/14 jobmanager-lcgpbs  gilda-01.pd.infn.it           
5   1    ScientificSLSL  i686  2400    0   2024/2024           0/0       0/20/20 jobmanager-lcgpbs  sirius-ce.ct.infn.it          
6   1    Scientific SLSL i686  2600    0   2048/2048           0/0       0/10/20 jobmanager-lcgsge  vega-ce.ct.infn.it            
7   1    ScientificSLBer i686  1800    0   4096/4096           0/0       0/48/48 jobmanager-lcgpbs  iceage-ce-01.ct.infn.it       
8   1    ScientificSLSL  i686  2400    0     513/513           0/0         0/4/4 jobmanager-lcgpbs  dgt01.ui.savba.sk             
9   1    ScientificSLSL  i686  3000    0   1024/1024           0/0         0/3/3 jobmanager-lcgpbs  dc01.nesc.ed.ac.uk            
10  1    ScientificSLBer i686  2339    0     513/513           0/0         0/6/6 jobmanager-lcgpbs  gilda-ce.rediris.es           
11  1    ScientificSLBer i686  3000    0   1024/1024           0/0       0/16/16 jobmanager-lcgpbs  gn0.hpcc.sztaki.hu            

Feel free to modify the configuration file /opt/gridway/5.5.0/etc/gwd.conf according to the Configuration Guide. To reload GridWay with your modifications you need to restart the daemon as the GridWay administrator globus. Type as root

root@ec2-79-125-53-164:~# pkill gwd
root@ec2-79-125-53-164:~# su - globus -c "gwd -c -m"

In order to set up new GridWay users

1. As root and add an account to the system

   root@ec2-79-125-53-164:~# adduser --ingroup gwusers YOUR-NEW-USERNAME
   

2. Create a folder $HOME/.globus into the new user account and copy there the certificate files usercert.pem and userkey.pem.

Problem

gwuser@ec2-79-125-53-164:~$ gwsubmit -t job.jt
FAILED: failed could not register user (check proxy)

Solution The instance contains some sudoers instructions which points to /opt/gridway/5.6.beta but the actual directory is /opt/gridway/5.5.0, so please add a link as root:

root@ec2-79-125-53-164:~$ ln -s /opt/gridway/5.5.0 /opt/gridway/5.6.beta