The international scientific research community is one of the leading users of high performance computing (HPC), and being one of the most diverse scientific research organisations in the world, we’ve got a whole heap of researchers looking for that bit of extra grunt to help drive their research to new heights.

Luckily for them, we’ve got them (very well) covered with our range of high performance computers.

Didn’t these things used to take up a whole building?

We have a long and proud history of building and maintaining high performance computers at CSIRO, dating all the way back to 1949 when we built Australia’s very first digital computer, CSIRAC.

But times have been a-changin since then, and what used to take up an entire room (CSIRAC wasn’t big enough to take up a whole building) now fits into racks and pods in custom designed temperature controlled rooms right next to other computing hardware.

Meet Ruby, our newest super (computing) star

We recently welcomed a new machine into the fold. Her name is Ruby, and she’s an SGI UV3000 named after Australia’s pioneering radio astronomer Ruby Payne-Scott.

Ruby is the latest in a line of shared-memory supercomputers we’ve been using from SGI since 2003, when we introduced an SGI Altix 3700 system named Cherax.

She’s packing  640 cores and eight terabytes of memory at her heart, but her real trick is using SGI’s NUMAlink6 technology to connect all those cores together to appear as one ‘giant PC’.

Ruby is based on a NUMA (non uniform memory architecture) design, which means that she can share her memory across multiple processors. This allows all the processing cores access to the memory like it’s their own, and to see what is being stored in the total amount of memory, allowing each process/core to share information, instead of having its own segregated memory in a traditional distributed memory architecture.

This makes her perfect for tackling data intensive tasks requiring large amounts of memory that benefit from the fast interconnection between the memory and processors.

Our users are anything but average

Our Information Management Technology (IMT) team work closely with researchers from an extremely broad range of disciplines to make sure their work is delivering its maximum impact by fully harnessing the capabilities our HPC resources can offer.

By planning things like the type, format and amount of data they’re producing, our IMT team help our researchers determine how to prepare and manage their data and computing needs during and after the life of the project, while making it as easy as possible to work with research collaborators across organisational boundaries.

Researchers at CSIRO are using Ruby for a range of science and engineering such as bioinformatics, genome assembly, climate modelling, nanoscience and imaging and image processing.

It’s like doing a jigsaw puzzle without the picture on the box

Denis Bauer is a scientist in our Health & Biosecurity unit who specialises in biomedical informatics and smart analytics tools.

She uses Ruby (and before that, Ruby’s predecessors Cherax) for analysing next generation sequencing data to look at specific chunks of the human genome.

It’s highly data and memory intensive work, with each genome of up to three billion base pairs being read on average around 100 times.

Her work is helping identify areas in the human genome that cause disease so we can continue to find out more about them. Some areas of focus so far are renal disease, colorectal cancer and Motor Neurone Disease (MND, also known as ALS or Amyotrophic lateral sclerosis).

Denis is using next generation sequencing technologies that have only been available for the last five years, to be able to quickly sequence individual people. The extra computing power that systems like Ruby provide is the backbone to making this new reality cheaper and more accessible than ever before.

Ruby has been especially useful for Denis’s genome sequencing work, enabling her to load an entire genome into memory at once.

“This type of work benefits from having hundreds of cores able to process the work simultaneously, but as well as the processing power, it needs unprecedented amounts of memory to run.” Denis said.

“While the jobs can be run in parallel, they can’t be run in isolation, as they are all interlinked. This means that all the data needs to be loaded into memory so the computer can see all the information to be able to piece the genomic jigsaw puzzle back together. It’s like doing a jigsaw without the picture on the box.”

And that’s not all folks

As well as looking after number crunching for our researchers, Ruby also looks after our Data Migration Facility (DMF).

Our DMF uses Hierarchical Storage Management (HSM) to make a whole range of storage devices of different speeds, sizes and costs act like one big drive, to be able to be used in the most efficient manner possible.

We’ve been operating our DMF on various machines since 1991, which makes it the longest continually operating DMF deployment in the world. Since it was set up it’s expanded by five orders of magnitude to now store more than 14 petabytes of information. To put that in perspective, that’s enough to store every episode of Game of Thrones (in HD) 325,581 times!

With about 1200 registered researchers using Ruby to access and crunch their data, our researchers will continue to push the limits of possibility.

You can read about Ruby’s finer details over at SGI’s blog.