1. Introduction
The current page gives a panoramic view of the framework in order to understand the TSDSystem structure and paradigm, hoping to grant a best utilization of the software through the enabled REST API.
The TSDSystem born at INGV Osservatorio Etneo to organize different data sources, such as volcanological, geophysical, and geochemical data collected among the study of volcanic activity.
Given that numerous research organizations and observatories gather extensive amounts of data from sensor networks, it is crucial to establish a systematic approach to organize and share this information for volcano research and monitoring purposes.
TSDSystem is a framework designed to facilitate the acquisition and standardization of time series data from various sources within a relational database, while also offering synchronized data visualization. The architecture of TSDSystem follows a multi-layer paradigm, comprising three main components:
- Loaders
- Database
- Business Logic
The Loaders layer consists of user-developed software applications, scripts, and services specialized in extracting data from custom data sources. These data are then ingested into the database using the TSDSystem REST API. Each data source may correspond to a specific geospatial location and one or more stations, with their information stored in a relational manner into the database.
The Database layer relies on a PostGreSQL RDBMS (Relational Database Management System), which leverages the TimescaleDB and PostGIS extensions for enhanced storage and management of time series and spatial data, respectively.
The Business Logic layer is implemented through the REST API, along with associated procedures and functions stored in the RDBMS.
The system offers a user-friendly web interface for data management and consultation, including the management of data policies.
A REST API is also enabled to perform operations over HTTP (HyperText Transfer Protocol) network protocol.
Additionally, an internally enabled Swagger UI is provided to test and document the REST API.
The TSDSystem REST API simplifies the data acquisition process by enabling sensors/stations equipped with a SIM card or another means of internet connectivity to directly send acquired data to the framework. This data is stored in local memory and transmitted using HTTP. By utilizing the HTTP standard, the framework supports the development and implementation of modules for interfacing with the service using various programming languages (e.g., Python scripts), utilizing the provided credentials.
Authentication and authorization schemes for API usage are based on the utilization of JWT tokens (JSON Web Token).
The source code of the framework is publicly available under the MIT License on GitHub.
You cand find it at the following repository: https://github.com/ingv-oe-dev/tsdsystem.
The software is distributed as a Docker container, and it is possible to install a complete version that includes the Grafana image. Grafana enables advanced and modern data visualization, such as monitoring dashboards, and allows flexible configuration of rules for alert notifications.
The web server used is Apache, and a Nginx instance serves as a reverse proxy for the incoming requests to the server. The deployment of the TSDSystem service adopts a microservices pattern through the orchestration of several Docker containers - carried out with Docker Compose - describing the components listed previously.