Optimizing Python Code for Climate Research on HPC Systems
From Lsdf
Jump to navigationJump to search
Description
Terabytes of data have to be analyzed in climate research to provide valuable insights for climate modelling and prediction. Even with today's HPC systems processing such amount of data has many challenges. The goal of this project is to optimize existing Python code that runs on different HPC systems to be more computational and memory efficient.
Task
The main task for this semester is to analyze the memory and CPU consumption of the existing Python code. In a first step this should be done for small datasets on single machines (notebook, workstation, server) and in a second step on different HPC systems.
Tasks
- Analyzing and benchmarking the existing code on different HPC systems
- Modifying the code to optimize memory and CPU consumption
- Optimizing the existing HPC run scripts
Requirements
- Programming experience in Python
- Programming experience in Bash
