JRC scientists are collaborating with the Observatoire du Sahara et du Sahel (OSS) to support decision-making in the field of sustainable management of natural resources and water through the provision of products and services based on Earth Observation (EO) data analysis. This collaboration is part of the strategic partnership under the umbrella of the Global Monitoring Program for Environment & Security in Africa (GMES&Africa), a joint initiative of the EU and the African Union. One of the main goals of the JRC-OSS collaboration is the monitoring and assessment of land degradation. This is targeted at the design and implementation of optimized processing workflows and web services for further developments of monitoring services.
Services used
JRC scientists are collaborating with the Observatoire du Sahara et du Sahel (OSS) to support decision-making in the field of sustainable management of natural resources and water through the provision of products and services based on Earth Observation (EO) data analysis. This collaboration is part of the strategic partnership under the umbrella of the Global Monitoring Program for Environment & Security in Africa (GMES&Africa), a joint initiative of the EU and the African Union. One of the main goals of the JRC-OSS collaboration is the monitoring and assessment of land degradation. This is targeted at the design and implementation of optimized processing workflows and web services for further developments of monitoring services.
Field of study
The Scientific challenge
In the existing prototype service MISLAND the land degradation monitoring is based on the ‘on-the-fly’ computation of land degradation indicators at different geographic scales. This computation is based on the assessment of the sub-indicators land cover, land productivity, and carbon stock. In the framework of MISLAND, it was decided to first implement a simplified version of the indicators at a reduced scale, and improve them during the execution of the programme. The set-up of collaborative scientific data analysis environments via the Jupyter Notebooks shall facilitate this evolution of the service and allow them to be built on top of the deployment of an initial version to be used at any relevant geographic scale.
The Technical challenge
The current MISLAND infrastructure does not allow the calculation and rendering on-the-fly of land degradation indicators for areas bigger than 10’000 ha. Furthermore, it does not host medium/high-resolution EO data (2000-2020) hence data are currently retrieved and visualized via Google Earth Engine or Amazon AWS services. Bringing user analysis workflows to the data in a collaborative environment and translating workflows to data services adequate for further usage is the main technical challenge. Guaranteeing system security for a system open to external partners via the Internet requires a careful set-up of all components for remote access.
The Business challenge
The data analysis environments and workflows shall follow the FAIR (Findability, Accessibility, Interoperability, and Reuse) principles and be based on publicly made available algorithms and data for reproducibility of results. Following free and open standards in software and service development is a key point in this aspect.
The Solution
The data science environments are collocated with a petabyte-scale storage backend holding all required public EO data (Copernicus, USGS, etc.) for the data analysis. A data catalogue with data discovery API allows identifying the relevant datasets. The development tools offered via the JupyterLab environments are extended with CS3Mesh API’s and extensions to support sharing of notebooks with their processing workflows, including view, edit with locking, and merging of data analysis and visualization workflow definition. This will offer the possibility of distributing and sharing the development of the indicators and data visualization among different experts.
Data workflows developed in the data science environments shall be possible to be exported to web services for further integration into the MISLAND web portal, as well as to interactive standalone dashboards for interactive data visualization. An interface to a batch processing service will allow the export of the data analysis results to downloadable data formats for further flexible usage.
The scientific, societal, economical and policy impact
Most Northern African countries experience recurrent drought spells of varying severity and length. Besides the drought episodes, another significant climate issue is the potential scenarios of climate change over the next decades. In fact, predicted impacts are of significance on natural resources, agriculture productivity, socio-economic implications and processes of desertification. Globally, 80 per cent of land degradation is caused by agriculture. Since 1950, 65 per cent of Africa’s cropland, which millions depend on, has been affected by land degradation by mining, poor farming practices, and illegal logging. Therefore, reliable information on land degradation and trends will support the countries to fight degradation in one or more of the land use categories for better program planning and awareness-raising. The envisaged scientific data analysis environments and related services aim to provide evidence-based proofs on land degradation and its spatiotemporal distribution. This gained knowledge will form the basis for prioritized actions and campaigns for awareness-raising.
Without CS3MESH4EOSC
The data science environments will lack the possibility for data scientists to share processing algorithms directly inside the environments and thus limit the possibility of collaboration between users.
With CS3MESH4EOSC
Data scientists will be able to work together analyzing the same input data, easily sharing data analysis algorithms and workflows, and hence develop data analysis and visualization components that can be included in dissemination services.
WHO BENEFITS & HOW?
End-users and research communities
The User community will benefit from a service to be exploited at various geographic scales, from the local, to national and regional.
Institutional operators and services
The proposed activity is a direct technological and scientific support to the African Institutions implementing the Land Monitoring Service (MISLAND) in the framework of the GMES&Africa project. Through the provision of the indicated information and services, the current limitations of the MISLAND services will be overcome.
Commercial software developers
Commercial software developers and data scientists can use distributed Data Science environments and data analytics patterns to develop a new product based on Earth Observation data (in agriculture, water management, flooding prevention and early warning systems, earth quality management, land cover/land use management, etc. )
Non-commercial software developers
Distributed data science environments will enable developers and data scientists to support research in disciplines based on Earth Observations (climate change, agriculture, water management, air quality, health factors, land cover/use etc.), as well as citizen scientists and Citizen Observatories.
Policy makers & citizens
Public institutions with a mandate of monitoring the land degradation in the region, for prioritizing the interventions and mitigating actions, and thus providing better life quality to citizens.
Standardised tools and distributed Data Science environments will also facilitate the collaboration of citizen scientists and Citizen Observatories who can easier engage in Earth Observation research.
Others
The outcomes of Earth Observation research and Data Science environments can support various educational activities which play a major role in increasing awareness and engagement in the young generation, as well as enhancing the learning process and promoting the collaboration of young environmental science enthusiasts.
Technical implementation
JupyterLab extension integrates with ScienceMesh – provides share and collaboration functionalities (full CS3 API client) directly in JupyterLab environment. Additional share functionalities are available in the file browser: the extension replaces the default JupyterLab file manager, adding new UI elements for additional share functionalities (a shared by/with tab, sharing buttons and entries in the context menus) and adding new modal windows to display file information and sharing status.
Data analysis workflows developed in the Jupyter environments will be possible to be translated to web services and interactive Voilà dashboards. Data export in full resolution will be possible through a backend batch processing cluster.
Contribution to the EOSC
Distributed Data Science environments will support research and development in all disciplines, facilitating the collaboration of distributed science teams.
Future developments
Concurrent collaboration of Notebooks, enhancing user interface for JupyterLab extension.
