Containerization and deployment of weather models on HPC infrastructures

In this thesis, we develop and demonstrate a robust and scalable workflow for building, deploying, and testing containerized weather models on the HPC clusters of the EuroHPC Joint Undertaking. The workflow is designed to support different cluster architectures, including x86-based systems (LEONARDO, GALILEO100, MELUXINA) and Cray-based systems (LUMI). Containerization is achieved using the Singularity platform, with container images built via Singularity definition files and optimized through multi-stage builds for lightweight and efficient images. The model-specific software stack is managed and deployed within the container using Spack, which enables the creation of customized environments tailored to the model’s needs. We also highlight the necessary compatibility conditions required for the containerized model to fully leverage the underlying hardware. As a proof of concept, we successfully containerize the RAPS bundle of the IFS global weather model, developed by the European Centre for Medium-Range Weather Forecasts (ECMWF), which utilizes hybrid parallelism (MPI+OpenMP) and GPU acceleration. The containerized model is deployed and benchmarked across multiple clusters, with performance comparisons against native host installations. The results demonstrate that the containerized model achieves performance on par with a host-based execution, with no significant degradation, showcasing the scalability and efficiency of the containerized approach for HPC environments.