Optimizing Python Code for Climate Research on HPC Systems

From Lsdf

Description

Terabytes of data have to be analyzed in climate research to provide valuable insights for climate modelling and prediction. Even with today's HPC systems processing such amount of data has many challenges. The goal of this project is to optimize existing Python code that runs on different HPC systems to be more computational and memory efficient.

Task

The main task for this semester is to analyze the memory and CPU consumption of the existing Python code. In a first step this should be done for small datasets on single machines (notebook, workstation, server) and in a second step on different HPC systems.

Tasks

  • Analyzing and benchmarking the existing code on different HPC systems
  • Modifying the code to optimize memory and CPU consumption
  • Optimizing the existing HPC run scripts

Requirements

  • Programming experience in Python
  • Programming experience in Bash

Contact

benjamin.ertl@kit.edu