Cluster 2006
IEEE International Conference on Cluster Computing, Barcelona September 25th-28th, 2006

Main Page
Call for Participation
     Important Dates
     Call for Papers

     Final submission
     Registration
     Student Awards
Program
     Key Notes
     Papers & Posters
Workshops
Tutorials
Panel
Committees
Sponsors&Exhibitors
ClusterSeries

About Barcelona
    Why Barcelona?
     Venue&Hotel Info
     La Mercè 2006
Grid06


 
 

Tutorials

 

Cluster 2006 news

You can still book your accommodation in the Venue hotel (conference rates)

viñeta

Student Awards

viñeta

Registration is opened

viñeta

Session program available

viñeta

Camera-ready papers submission opened

viñeta

Tutorials accepted available

viñeta

Call for Participation (flyer)

viñeta

Panels

viñeta

Call for Exhibitors (pdf)

 

(click to view more photos)

 

 

Parallel I/O in Practice

Rob Latham, Rob Ross, Rajeev Thakur, and William Loewe

Tutorial Description
This tutorial will be a one-day tutorial focusing on parallel I/O for computational scientists. The first half of the day will focus on parallel file systems in theory and practice, including a discussion of the three most relevant parallel file systems in use today: GPFS, Lustre, and PVFS2. The second half of the day will focus on the I/O software stack as a whole, including POSIX I/O,MPI-IO, and high-level I/O libraries. Combined, this material prepares attendees to make informed decisions about how to approach parallel I/O for applications and provides attendees with an understanding of how the I/O stack works to enable high-performance I/O. If it is preferred, we can present a half-day version of the tutorial instead. This was presented at Cluster 2005, with 7 people signing up and 14 showing up for the tutorial. If we were to present in a half-day format, only two of the presenters would attend, and we would focus on the second half of the material.

General Description
This tutorial fundamentally covers how to use the parallel I/O resources available on today’s computational science platforms to obtain high performance for application I/O. To understand how to best use these resources, one must first have an understanding of parallel file systems and how they operate. This provides the base knowledge necessary for understanding how higherlevel I/O components interact with the parallel file system. The first half of our tutorial is dedicated to parallel file systems. We cover common architectures and features of parallel file systems first, then we examine three relevant examples: GPFS, Lustre, and PVFS2. We wrap up our coverage of parallel file systems by discussing the results of benchmarking these systems. We highlight the relationships between benchmarks and application patterns and the strengths and weaknesses of each file system. Effective parallel I/O involves more than just parallel file systems. Applications always interact with the parallel file system through some application programmer interface (API). There are a number of interface options for parallel I/O, and in fact some of these are built on one another (I/O software stacks). The second half of our tutorial covers interfaces for parallel I/O.We cover the four most common APIs: POSIX,MPI-IO, Parallel netCDF, and HDF5, in order of increasing functionality (and complexity). For each interface we first discuss the features of the interface and show a simple example. We follow this up with a more practical and detailed example. We then discuss what happens below the interface, leveraging our discussion from the first half of the day.

Goals
Attendees will come out of this tutorial with an understanding of:
• how parallel file systems and I/O software stacks are constructed,
• the relative merits of some popular parallel file systems,the I/O interfaces available for applications and their features, and
• how in I/O stacks components work together to manage parallel I/O.

This background will allow them to make educated decisions about how to manage I/O in their applications. They will have an opportunity to interact with experts in all of these components during the tutorial and over lunch to discuss how to best approach parallel I/O in the context of their particular applications.


Target Audience, Prerequisites, and Content Level
This tutorial is intended for an audience interested in learning how to best use I/O resources on parallel computers. We assume some proficiency in MPI, but no experience with parallel file systems or any of the I/O interfaces or libraries that we will cover. The content is approximately 30% beginner, 40% intermediate, and 30% advanced.

Topic Relevance
Parallel I/O is always a hot topic in parallel computing. Making best use of available I/O resources is an important issue for applications in order to avoid bottlenecks. New machines such as the IBM BG/L and the Cray Red Storm provide a significant improvement in computational power, making proper use of I/O resources an even more critical issue.

Ensuring Cohesive Content
The three Argonne presenters have worked together for a number of years on projects such as PVFS, PVFS2, Parallel netCDF, and ROMIO. All the presenters have met numerous times for discussions related to parallel I/O and presented this material at SC2005. The second half of the tutorial has been presented a number of times by subsets of the presenters.

Demos and Exercises
It is impractical to provide all attendees access to a system running all the parallel file systems and libraries described in the tutorial itself. Instead we will concentrate on practical examples, benchmarks, and knowledge that could be applied across a wide range of application domains and file systems. We will also provide pointers to benchmarks, file systems, and libraries used in the tutorial so that attendees can experiment on their own.

Outline
1 Introduction
 – I/O software stacks
 – Parallel file systems
2 Parallel file systems in practice
 – Example parallel file systems
 – Benchmarking and parallel file system performance
 -- Lunch--
3 Interfaces for parallel I/O
 – POSIX
 – MPI-IO
 – Parallel netCDF (PnetCDF)
 – HDF5
4 Conclusions
 


Grid Computing and Gridbus

Rajkumar Buyya
 

Audience
This tutorial should be of interest to a large number of participants from academia, government, and commercial organizations as it focuses on both theory and practice of grid computing and applications. They include: (A) students, researchers, and developers interested in creating technologies and applications for Next Generation Grids (B) participants from commercial organizations interested in creating online Grid marketplace and applications, and (C) policy makers of Grid Computing as we will be offering a live demonstration of current Grid technologies and their applications.

Course Description
Grid computing, one of the latest buzzwords in the ICT industry, is emerging as a new paradigm for Internet-based parallel and distributing computing. It enables the sharing, selection, and aggregation of geographically distributed autonomous resources, such as computers (PCs, servers, clusters, supercomputers), databases, and scientific instruments, for solving large-scale problems in science, engineering, and commerce. It leverages existing IT infrastructure to optimize compute resources and manage data and computing workloads. The developers of Grids and Grid applications need to address numerous
challenges: security, heterogeneity, dynamicity, scalability, reliability, service creation and pricing, resource discovery, resource management, application decomposition and service composition, and qualify of services. A number of projects around the world are developing technologies that help address one or more of these challenges. To address some these challenges, the Gridbus Project at the University of Melbourne has developed grid middleware technologies that support rapid creation and deployment of eScience and eBusiness applications on enterprise and global Grids.

The components of Gridbus middleware are: Grid application development environment for rapid creation of distributed applications, Grid service broker and application scheduler, Grid workflow management engine, SLA (service-level agreements) based Scheduler for clusters, Web-services based Grid market directory (GMD), Grid accounting services, Gridscape for creation of dynamic and interactive resource monitoring portals, Portlets for creation of Grid portals that support web-based management of Grid applications execution, and GridSim toolkit for performance evaluation. In addition, Gridbus also includes a widely used .NET-based enterprise Grid technology and Grid web services framework to support the integration of both Windows and Unix-class resources for Grid computing.

The tutorial covers the following topics:
1. Fundamental principles of grid computing and emerging technologies that help in creation of Grid infrastructure and applications.
2. A Review of major international efforts in developing Grid software systems and applications both in academic, research and commercial settings.
3. Service-Oriented Grid Architecture for realising utility computing environment that supports resource sharing in research and commercial environments. Realization of this architecture by leveraging standard computing technologies (such as Web Services) and building new services that are essential for constructing industrial-strength Grid engines.
4. Gridbus middleware and technologies for creating enterprise and global utility Grids.
5. Issues in setting up Grids that can scale from enterprise to global and deploying applications on them.
6. Case studies on the use of Gridbus technologies in creating applications in the area of Drug Discovery, Neuroscience, High Energy Physics, Natural Language Engineering, Environmental Modelling, Medicine, Portfolio and Investment Risk Analysis.
7. Live demonstration of Gridbus technologies and their use in creating and deploying sample applications on the World Wide Grid (WWG).
8. Sociological and industrial implications of this new Internet-based distributed computing paradigm and its impact on the marketplace.

The tutorial places emphasis on concepts of Grid economy, how to design and develop Grid technologies and applications capable of dynamically leasing services of distributed resources at runtime depending on their availability, capability, performance, cost, and users' quality of service requirements.