High performance data analysis and visualization tools for the MedBFM physical-biogeochemical model

The MedBFM model system is developed and managed by OGS and operationally pro- duces analysis, forecast and reanalysis of biogeochemical 3D fields for the Mediterranean Sea in the framework of the Mediterranean monitoring and forecasting center of the EU Copernicus Marine Environment Monitoring Services (CMEMS). The present post- processing scheme is based on an off-line suite of python scripts (aka \bit.sea") which is used to monitor the model output, to prepare the quality information documents targeted towards the users, and for scientific research. However, the inner complexity of the multivariate 4D data products (i.e., approximately 50 variables organized in 3D gridded fields evolving in time), the increase of the number of products (operational and derived), validation metrics, and users number, all combined with the continuos refining of spatial resolution, pose a series of challenges for the efficient management of the whole data stream analysis workflow and its performance. Indeed, the usual approach to data analysis can easily become too complex for the generic user: the need to exploit a cluster for the analysis of large amount of data poses strong limits on the practical usability of standard analysis routines, as can be seen from skatch in Fig.1.1. The alternative approach proposed in this thesis work aims to develop an efficient and scalable tool that can directly access model's output (thus skipping the postprocessing phase), obtaining on-the-fly and on-demand results, while keeping a flexible and dynamic structure that also provides an intuitive graphical user interface (GUI), granting an easy access to the users. This service may be able to run on a dedicated server for remote visualization, offering the possibility to interactively inquire datasets to a large number of users. The natural environment for this kind of application is Paraview, since it is an open-source software with the capabilities to visualize and analyze large amount of data, both using interactive or batch/scripting methods.