Development of an IoT framework for stream based analysis of sensor data. DBPRO, TU Berlin, summer term 2018.
(Original project home: https://gitlab.tubit.tu-berlin.de/anton.rudacov/DBPRO-IoT-Framework)
This application is an example of basic Flink-Kafka-InfluxDB workflow.
The program performs following steps:
- generates a stream of timestamps and attaches data to it (captured before from a sensor).
- sinks stream data to InfluxDB and to a Kafka topic
- consumes data from Kafka topic (in another thread) and sinks it to InfluxDB as well
- splits infinite stream in windows and calculates "Dynamic Time Warping" distances in order to recognise the input
- adds annotations containing recognised character to Grafana via HTTP request
Just run the main() method in jobs/App.java
In order to actually get any observable output you have to run the necessary instances (Kafka, InfluxDB, Grafana) and setup their addresses in corresponding java classes.
- Kafka - distributed streaming platform
- Apache Flink - data processing engine
- InfluxDB - time-series data storage
- Grafana - time series analytics platform
- Maven - Build automation tool and dependency management
- GitLab - Git repository manager, wiki, issue tracking and CI/CD
- Docker - Testing and CI environment (container platform)
- Anton Rudacov - @antonrud
- Janine Schulte - @janineschulte
- Sven Hellweg - @hellweg.sven
This project is licensed under the MIT License - see the LICENSE file for details
Project structure and some implementations are inspired by this awesome demonstration by Jamie Grier.
Dynamic Time Warping algorithm is created by Cheol-Woo Jung (browse source).