Creating low-cost "learning clusters" for HPC

Get Schooled

Article from Issue 154/2013
Author(s):

Starting out in the HPC world requires learning to write parallel applications and learning to administer and manage clusters. We take a look at some ways to get started.

On the Beowulf mailing list [1] recently, Jim Lux started a discussion about "learning clusters" [2]  – that is, low-cost units that can be used to learn about clusters. People often wonder how to get started with clusters, usually in one of two ways: "I want to learn parallel programming" or "I want to learn how to build and administer clusters." Sometimes the question is simple curiosity: "What is a Beowulf cluster and how can I build one?"

In the past, you had just a few options, including getting access to a cluster at a nearby university, buying/borrowing/using some old hardware from various sources to build one yourself, or designing your own cluster by shopping for parts. However, with the rise of virtualization, you now have more system options than ever, particularly if your budget is limited.

In this article, I'll take a quick look at various system options for people who want to learn about clusters, focusing on the programming and administration aspects. These options range in price, learning curve, ease of use, complexity, and just plain fun, but the focus is to keep the cost down while you are learning. The one aspect of clusters that I'm not really going to focus on is performance. The goal is to learn, not to find the best price per 109 floating point operations per second ($/GFLOPS) or the fastest performance. Performance can come later, once you have learned the fundamentals.

[...]

Use Express-Checkout link below to read the full article (PDF).

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy Linux Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • StarCluster

    Cloud computing has become a viable option for highperformance computing. In this article, we discuss the use case for cloud-based HPC, introduce the StarCluster toolkit, and show how to build a custom machine image for compute nodes.

  • PelicanHPC

    Crunch big numbers with your very own high-performance computing cluster.

  • OpenSSI

    The OpenSSI framework rearranges processes for easy and transparent clustering.

  • CLUSTERIP

    Iptables gives admins the ability to set up clusters and distribute the load. But what about failover?

  • Hadoop

    Experience the power of supercomputing and the big data revolution with Apache Hadoop.

comments powered by Disqus

Direct Download

Read full article as PDF:

Price $2.95

News