Note! The information here is outdated and the pages will be removed. Please refer to our new Grid documentation page at: http://doc.grid.surfsara.nl/

Using the Grid/Access introduction

From SURFsara Grid pages
Jump to: navigation, search

People affiliated with Dutch universities and institutes are eligible for using the NL Grid infrastructure; See https://www.surfsara.nl/access-to-the-national-e-infrastructure for more information.

How it works

As a user you connect to the Grid by using a so-called UI (User Interface). Once you have received the right credentials (i.e. a grid certificate, an account on a user interface machine and a VO membership) you are set to go.

First, you divide your problem into smaller units, called jobs. These jobs are the unit of computation and can be submitted to the Grid. The way to do this is to describe each job in terms of a Job Description Language (JDL). This is not a programming language but consists of attribute value pairs which describe your job. Here you list which program should be executed and what data it should operate on. Your program and data can be send with the job when necessary.

Each job in the form of a JDL file is then submitted to the Workload Management System. This system schedules your jobs and knows which compute clusters in the Grid are ready to accept your job. These clusters each consist of several machines. The Compute Element (CE) is the server which communicates with the WMS and accepts jobs. It then distributes the jobs to other machines in the cluster, called Worker Nodes (WNs). These WNs are the machines which do the actual work. When finished with a job they will report back to the CE, which in turn will inform the user about the status of the job. In addition, Clusters have a storage server, called the Storage Element (SE). These servers can be used to store files on a permanent basis. Data on the SE's can be replicated at other sites and jobs can be told only to land on Worker Nodes which are close to data the jobs will operate upon.

On these wiki pages you can find more information about this whole process.

Intoduction grid.jpg


To use or not to use Grid

Grid is best suited for applications with a data-parallel nature that require many simultaneous independent calculations. This means that the data for the job can be split up relatively easily in multiple, independent parts. You may find it difficult to port software running MPI or multiple intercommunicating threads to the Grid. MPI jobs can be run on our Grid, but will run on a single cluster not across clusters. Of course, SURFsara is willing to help you bring your application to the Grid.

All our clusters run CentOS 6 and every script or program which runs on that can, in be principle, be submitted to the Grid.

At present all grid nodes run a 64 bits Operating System (CentOS 6) and Grid middleware (gLite). A single job can address about 3GB RAM. In addition, each job should only run on a single core, multi-core jobs are experimental.

If your application is not well-suited for Grid you could take a look at the other SURFsara Systems.
Please contact SURFsara to find the best suitable system for your application.

A Grid certificate

Before you can use the Grid, you have to set up your identity on the Grid. To avoid that you have to log in to each resource on the Grid separately, you will use a personal certificate that will identify you on the Grid. You can compare it to a universal key that automatically unlocks every door you are allowed to open. Which resources you can use, is determined by your membership of Virtual Organizations (VOs).

The next few steps in this tutorial will help you with your request for a personal certificate, your membership of one or more VOs and where you have to store your certificate to use it on the Grid. It is a procedure which may seem daunting at first - but it should be a once-in-a-lifetime event anyway. If you need, you can get support in this procedure by contacting grid support.