Using the HPC Cloud/betaevaluation

From SURFsara Grid pages
Jump to: navigation, search

High performance compute cloud

Sara is developing a High performance compute cloud for the scientific community, funded by BiG Grid. Currently this HPC Cloud is offered in beta to a selected group of scientific users for evaluation purposes. We would like to invite members of the Dutch scientific community to apply for evaluation use of the HPC Cloud.

Proposals (250-500 words) should consist of:

  • a short description of your project objectives
  • a project planning in weeks
  • estimate of your desired compute hours
  • estimate of the storage requirements

What can be applied for?

  • applicants will obtain access to the HPC Cloud
  • the ability to submit and start your own virtual machines
  • manage and configure your own submitted virtual machines
    • Use existing templates, upload your own or configure from scratch with free choice of Operating System and applications
    • Access to the virtual machines can be through console, ssh and over public_ip

The SARA support team will actively support the implementation of your planned activities. We are especially interested in applications which are difficult or near impossible to run on the existing HPC platforms (Huygens, Lisa, Grid), but do run in your local environment.

In this beta phase we strive to offer a full production grade environment, we are however continuously improving the technical environment as well as the ways of working. This may have consequences on stability and availability.

The host infrastructure

The cloud is hosted on a cluster with 128 cores with the following characteristics:

  • 16 compute nodes
    • CPU dual quad core 2.2 GHz
    • 24 GB RAM per node
    • 500 GB local Hardisk
    • network 1 Gbs per node
  • backupped storage: 100 TB
    • 20 Gbs aggregated connection from the cluster to storage
  • Host Software
    • OpenNebula
    • Virtual machine software: KVM
    • Multicore/multiprocess is possible, also MPI and OpenMP.
    • Virtual Private Compute Cluster©: starting multiple VMs in their own private network (vlan)

Possible use cases

For your inspiration we have listed some possible applications and interesting aspects.

  • Quick deployment of persistent scalable services (running your own database and/or web server)
  • Using Cloudburst, a gene-assembly application with map-reduce and Hadoop
  • Job-submission with a Topos master pool server in the cloud for effective job orchestration
  • Testing licensing aspects / using closed source and or binary applications
  • Using MPI, shared memory, multi-core (OpenMP)
  • Running G-lite or other grid middlewares
  • Using Amazon Cloud compatible virtual machines, see: Amazon machine images
  • your application that does not run on existing HPC platforms ...

What is expected of successful applicants?

  • an entrepreneurial spirit and open mindedness, because of the beta status of this service
  • a written evaluation report on your experiences (1-2 pages)
  • participation in a planned workshop in september

Procedure

  • your written proposal should be submitted before midnight the 12th of May 2010
  • please mail your application to claudia-support@sara.nl
  • you will receive a response to your applications before the 19th of May 2010
  • This phase will run at least until the end of October 2010 and based on its results a full production service will be set-up
  • This call for evaluation is open to everybody in the Dutch scientific community

Why HPC Cloud Computing ?

E-Science and computational science are very dynamic fields. The needs and demands of researchers are constantly changing, which is reflected by the large variety of computer software that they develop and use. The various High Performance Computing supercomputers and grids that SARA provides are all shared resources and do support the most common needs of the researchers. Specific needs can sometimes be hard or nearly impossible to accommodate using the standardised system environments.

With the newly developed High Performance Computing (HPC) Cloud environment researchers are provided with access to their own Virtual Private HPC Cluster. It is a virtualized HPC Cluster that can be configured to exactly match their needs, without interfering with the needs of other users. It is flexible, offers self service and is dynamically scalable. Users can start from existing templates (images), or built their own cluster from scratch. It is even possible to make a copy from their current software environment (for example their laptop or desktop pc) and turn that into a HPC cluster in the Cloud. In that way, there will hardly be any difference between their development environment and their production environment. There is no need for an (expensive) rewrite of their software, and scientific challenges can be scaled up very easily from desktop scale to High Performance Compute cluster scale.

A HPC cloud lowers barriers to use a HPC system. That will allow researchers to obtain results faster and it provides potential users with specialized needs, a highly flexible HPC infrastructure, that is not available to them now.

In the second half of 2009 we have built a proof-of-concept of the HPC Cloud together with a number of scientific users demonstrating real test cases. These test cases, in the areas of geo-ecology, bioinformatics, text processing and named entity recognition, clearly showed the added value in a large variety of scientific research in terms of flexibility, ease of use and productivity.

Based on the results of this pilot project, SARA has set up a small scale BiG Grid sponsored HPC Cloud service. This service is currently in beta. It is our aim to be able to offer a production grade HPC Cloud service for the benefit of the scientific community by the end of 2010.