Lead applicant: Prof. Dr. Henk Dijkstra Organisation: Instituut voor Marien en Atmosferisch onderzoek Utrecht (IMAU), Universiteit Utrecht

The research project

The sea level is rising, but how much and where are the subject of debate. The goal of the interdisciplinary eSalsa team is to generate predictions that help us determine the risk of very high rises. This collaborative project is led by Professor Henk Dijkstra: “We see a significantly increasing risk of an extreme sea level rise in the North Sea area.”

The eSalsa project began in July 2012 as a collaborative effort of the Netherlands eScience Center, the VU University Amsterdam and Utrecht University. The project is investigating regional extremes in the sea level, according to Dijkstra: “These arise from currents in the ocean that we call ocean whirlpools. High-resolution models are necessary to determine the structure of the whirlpools. With these ocean models, we can visualise changes in the distribution of whirlpools in the ocean and see how they affect the sea level rise.”

The challenge

Henk Dijkstra wants to process ocean models and climate simulations in high resolution. Instead of simulating the ocean model alone, his team wants to consider the global climate. This requires supercomputers all over the globe to be linked together. The researchers asked themselves the following question: “If we have multiple supercomputers at our disposal, would it be possible – by way of example – to run the atmosphere model on one supercomputer and the ocean model on another?”

he project has already used Cartesius, the national supercomputer of the Netherlands, for its calculations. “Thanks to the graphical processing units (GPUs) in Cartesius, our ocean model is three to four times faster than before”, says Dijkstra. “In our initial simulations it took about one day for one year of ocean simulation, but now we can do four years in one day. We want to compute approximately 100 years, which we can now do in 25 days.”

The solution

The efforts of Utrecht University, SURFnet, SURFsara and the eScience Center have ensured that multiple supercomputers could be used in tandem to perform climate simulations for the first time. The simulations include an ocean model, an atmosphere model, a land model and a sea ice model. The four different models exchange data within their own model and with each other.

Fast 10 Gbit/s SURFnet light paths facilitated the connection between the SURFsara supercomputer in Amsterdam and the supercomputers in Germany and the United States. SURFnet created two light paths: one from Cartesius in Amsterdam to the Stampede supercomputer in the United States and one from Amsterdam to SuperMUC in Germany.”

These supercomputers had to be able to communicate and even exchange data from different simulation models with each other. Jason Maassen, Parallel and Distributed Computing Engineer at the Netherlands eScience Center, had the experience and knowledge of Message Passing Interface (MPI) to make this possible.

In future, Dijkstra and Maassen want to increase the resolution of the models and simulations. The ocean model is now running at a resolution of 10 by 10 kilometres. The aim is to develop this to a resolution of 2 by 2 kilometres or even 1 by 1 kilometre. Maassen: “It is therefore clear that we can process information on several supercomputers that we would not be able to process on one alone.” A higher resolution offers many advantages to the research. An ocean vortex is about 10 to 50 kilometres in size. At a resolution of 1 kilometre, substructures within the vortex can be seen that would not be visible at an accuracy of 10 kilometres. The higher the resolution, the more visible the important details become.