FAIR-by-Design Real-Time Data Pipeline for Hydrogen Digital Twins

Hydrogen energy is a versatile energy carrier, increasingly recognized for its role in a sustainable energy future due to its high energy density and compatibility with existing gas infrastructure. Digital Twin (DT) technology has emerged as a pivotal cyber-physical tool for integrating hydrogen energy into modern energy systems, enabling virtual modelling, predictive analysis, and system optimization across the full production-storage-utilization chain. The H2SmartLab project was conceived to instantiate this vision: a research infrastructure that smartly integrates renewable hydrogen production, storage, and utilization with advanced monitoring and failure prediction, underpinned by a DT platform called Hydor. However, a critical gap remains between the physical laboratory, where raw experimental data is generated, and the DT layer that must consume it in real time. Without a robust, validated data pipeline, the DT cannot operate reliably or reproducibly. This thesis addresses that gap by designing and implementing a real-time, event-driven data pipeline built on Apache Kafka, a three-stage data quality validation layer, and a time-series storage system using QuestDB. A FastAPI synchronization layer provides the interface between the physical laboratory and the Hydor DT platform, offering authenticated, FAIR-aligned data access. The implemented system achieves sub-second ingestion latency, automated anomaly detection via per-record quality validation, and full FAIR4RS software compli ance, demonstrating a reproducible architecture for monitoring hydrogen and broader industrial infrastructure.