Combining Grid and Cloud for e-sciences applications

From Gridkaschool
Revision as of 10:34, 28 August 2013 by Abdennadher (talk | contribs) (→‎Important information)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigationJump to search

Speaker : Nabil Abdennadher[1]


[XtremWeb-CH (XWCH)][2] is a volunteer computing system inspired from large-scale computing platforms for distributed applications. XWCH is easy to install and maintain, it has been bridged with the ARC Grid middleware. A dedicated API is used to write Client programs that use an XWCH infrastructure. Several [e-science applications][3] have been ported on XWCH and several lessons have been learned from these deployments. The most important are the following:

  • In many cases, the performance requested by end users applications are not available on the XWCH infrastructure,
  • When the XWCH platform is used by several users (applications) at the same time, available resources cannot meet user requests.

We propose an alternative which allows users to extend the XWCH volunteer infrastructure by renting private “compute nodes” deployed on cloud infrastructure. The idea is to rent resources on cloud whenever they can not afforded by the XWCH infrastructure. In this context, clouds could be seen as “useful utility that you can plug into your Grid”.

This tutorial proposes a toolkit used to develop, deploy and execute high performance (e-science) applications on heterogeneous infrastructure composed of volunteer computing and Cloud. This toolkit combines the reliability of cloud infrastructures with the “openness” of the XWCH development environment.

Concretely speaking, we propose a generic API used to develop high performance (e-sciences) applications that could be deployed on Grid and Cloud without any re-writing. The following platforms/middlewares will be used during the practical part:

  • the XtremWeb-CH volunteer computing platform
  • two public cloud (IaaS): Amazon and Azure
  • a private cloud IaaS based on [OpenStack][4]

The tutorial is composed of theoretical and practical parts. The theoretical part will deal with the following aspects:

  • Grid and Volunteer computing vs. Cloud computing,
  • Overview of XWCH, Amazon, Azure and OpenStack
  • How to develop applications on XWCH

During the practical part, the students will be able to deploy e-sciences applications on an heterogeneous platform based on XWCH, Azure, Amazon and OpenStack. A generic API is used for this purpose.

Important information

  • For this tutorial a personal notebook is recommended (Mac or Ubuntu Linux)
  • Java (at least 1.6) should be installed on your laptop.
  • Please create an "XtremWeb-CH" account on [5] and send an Email to nabil.abdennadher[at] to validate your account.
  • ppt presentation [6]

Part 1: The XtremWeb-CH Volunteer Computing Platform

  • To install XWCH worker on your machine click here [7]
  • This tutorial uses a Ray Tracing application called PovRay. To download an XWCH version of PovRay, click here [8]. The XWCH Java API is in the same .zip file
  • PovRay uses a ppm extensions. .ppm files can be visualized by one of these tools:
  • XWCH Java API documentation is here [9]

Part 2: Combining XtremWeb-CH with Amazon, OpenStack and Azure

  • Deploying PovRay application on Amazon :
    • You need to have your own Amazon account. In order to deploy XWCH workers on Amazon you should retrieve your AWS access credential (Access key Id and Security access key). For further details, consult Section To view your AWS access credential in this URL[10],
    • Deploying PovRay on XWCH (This program deploys XWCH workers on Amazon)[11]
    • List of AWS AMIs [12]
  • Deploying PovRay application on OpenStack
    • In order to deploy XWCH workers on OpenStack you need to have an OpenStack account. As a participant in this tutorial, you already have one on hepiaCloud portal[13]
    • Deploying PovRay on XWCH (This program deploys XWCH workers on hepiaCloud)[14]
  • Deploying PovRay application on Azure: There will be only a demo.