DHUS – Sentinel EO product data hub
DHUS is the official Data Hub to acquire Earth Observation (EO) products from the missions of the Sentinels satellites. It is a “web based” system designed to daily provide to users raster images corresponding to raw product received by the spatial station antenna.
The Data Hub Service (DHuS) is designed to provide ESA and GMES users with a distributable service of mirror archives and dissemination means for EO products. DHuS does not duplicate any existing or already planned dissemination services but is more a complement with a versatile, lightweight, easy to setup and distributable functionality. As such, a survey of existing services as the MMFI, EOLI or ngEO from ESA but also from third party organizations such as the USGS EarthExplorer, the USGS Glovis or the SPOT Image EO product distribution systems has been conducted and lead to the design of DHuS. Online Browsing According to the always increasing size of the EO products (Sentinel-1 10 minute SLC products are about 182 Gb), it seems more and more critical to minimize the full product data transfers. On many occasions, end users may not be interested in the overall product. DHuS intends to minimize the data transfers by proposing rich previews including data type oriented metadata, partial downloads in XML and more, an online browser that enables visiting any part of the data without downloading the product. Federation DHuS service is as lightweight and easy to setup as possible and runs on a scalable infrastructure. It can be installed from a single Laptop to a large scale network of clustered servers. The Solr index sharing and replication capability are used to achieve this requirement. The DhuS to DHuS communication are managed via its external API, using security managed by DHuS. Service based The DHuS Server adopts a service-oriented architecture, built within a Spring Framework that is in charge of injecting and wiring DHuS Services in a consistent way from startup to shutdown.
Innovations & Advantages
Open Data Protocol Interface
DHUS exposes the emerging Open Data Protocol (OData) interface that is progressively becoming the standard for accessing “Big Data” and Clouds data made available by governmental and private organizations. This protocol is based on top of the well supported HTTPS/REST transfer protocol that can be handled by a large set of client tools as simple as common Web browsers, download-managers or command line tools such as cURL or wget.
The Odata protocol is extended with additional HTTPS/REST API to extend Odata capabilities.
The Open Data Protocol has more capabilities than standard SSH or FTP protocols by not only providing file listing and transfers but also allowing properties extraction or activation of functions, etc.
DHUS service provides on-demand or automatically processed value-added information issued from transformation of data available in the data store. The transformation module is designed to be scalable in order to easily extend the processing capabilities of the system. Not public at this stage, the transformation API design is based on same design pattern as JAXP transform API.
Current and Potential Domains of Application
1.4. Data Communications
2.7. Computer Software
3.1. Electronic Components
3.7. Analytical and Scientific Instrumentation
5.3. Other Medical/Health Related
7.1. Leisure and Recreational Products and Services
7.4. Consumer Products
7.5. Consumer Services
8.2. Industrial Automation
8.3. Industrial Equipment and Machinery
9.2. Finance, Insurance and Real Estate
9.7. Construction and Building Products
9.8. Utilities and Related Firms
9.9. Other (uncategorised)