Dr Rocco Sedona - lecture on Remote Sensing

HIPPO: Integrating EuroHPC Resources into the Destination Earth Platform with Apache Airflow and UNICORE

‍

Destination Earth (DestinE) aims to build a digital twin of the Earth system, relying on Euro-HPC supercomputers for its most demanding workloads. Making these resources accessible from the DestinE Platform in a programmable, scalable, and operationally sustainable way requires bridging cloud-native services with traditional HPC batch systems. The talk will present HIPPO, a project at the Jülich Supercomputing Centre that prototypes this integration using Apache Airflow 3 deployed on Kubernetes and coupled to HPC via UNICORE, describing the system architecture, the development of custom Airflow-to-UNICORE operators and a UNICORE Executor, and the validation of the approach through MLOps workflows (hyperparameter optimization, foundation model benchmarking) running on JSC supercomputers.