The JPO Conflict Monitor is a real time validation system to verify corresponding SPAT, MAP and BSM messages. Validation is done by aggregating input messages, then performing appropriate analysis to establish and detect vehicle events for an intersection. Detected Events are published to Kafka Topics to be stored for further analysis.
The conflict monitor is focused on validating messages (SPaT, MAP and BSM) against one another to develop a picture of what is happening at a given intersection. In particular, the conflict monitor is looking for state information which is not self consistent and is likely the function of incorrect message generation and broadcasting. The conflict monitor is not configurted to validate message structure, as this functionallity is performed by the jpo-geojsonconverter.
All stakeholders are invited to provide input to these documents. To provide feedback, we recommend that you create an "issue" in this repository (https://github.com/usdot-jpo-ode/jpo-conflictmonitor/issues). You will need a GitHub account to create an issue. If you don’t have an account, a dialog will be presented to you to create one at no cost.
- Usage Example
- Configuration
- Installation
- File Manifest
- Development Setup
- Deployment
- Contact Information
- Contributing
The jpo-conflict monitor uses JSON encoded ODE BSM data, along with the processed SPaT and MAP messages from the geojsonconverter to validate intersection performance. Data is partitioned based upon the receiving device (generally the RSU at an intersection) to ensure scalable performance. In order to verify your jpo-conflictmonitor is functioning, you must run the jpo-ode, the jpo-geojsonconverter and the jpo-conflictmonitor and then send the jpo-ode raw ASN1 encoded MAP, SPaT and BSM data through the ODE to validate performance.
Follow the configuration section to properly configure and launch your jpo-ode, jpo-geojsonconverter, and jpo-conflictmonitor.
Recommended machine specs running Docker to run the GeoJsonConverter:
- Minimum RAM: 16 GB
- Minimum storage space: 100 GB
- Supported operating systems:
- Ubuntu 18.04 Linux (Recommended)
- Windows 10 Professional (Professional version required for Docker virtualization)
- OSX 10 Mojave
The ConflictMonitor software can run on most standard Window, Mac, or Linux based computers with Pentium core processors. Performance of the software will be based on the computing power and available RAM in the system. Larger data flows can require much larger space requirements depending on the amount of data being processed by the software. The JPO-ConflictMonitor software application was developed using the open source programming language Java. If running the JPO-ConflictMonitor outside of Docker, the application requires the Java 11 runtime environment.
The JPO-ConflictMonitor is a micro service that runs as an independent application within an existing Apache Kafka System. The JPO-conflictmonitor takes advantages of multiple data models defined in the jpo-geojsonconverter and jpoode. These are included into the Conflict Monitor as library references in maven, and images of these applications are spun up in the provided docker-compose.yml.
- Docker: https://docs.docker.com/engine/installation/
- Docker-Compose: https://docs.docker.com/compose/install/
Read the following guides to familiarize yourself with ConflictMonitors's Docker and the ODE managed Kafka modules. The jpo-utils repository is referenced as a submodule and handles managing the base application that the conflict monitor relies on such as kafka, kafka-connect, and MongoDB.
Installation and Deployment:
- Docker builds may fail if you are on a corporate network due to DNS resolution errors.
- Windows users may find more information on installing and using Docker here.
- Users interested in Kafka may find more guidance and configuration options here.
Configuration:
The JPO-ConflictMonitor configuration is customized through the environment variables provided to Docker when Docker-Compose runs the Docker built JPO-ConflictMonitor image. This provides basic configuration on kafka items such as the kafka broker location. Individual process modules can be configured by modifing the properties files in the resource folder before building the docker image.
Important!
You must rename sample.env to .env for Docker to automatically read the file. A .env file is also required in jpo-utils. Do not push this file to source control.
The following instructions describe the minimal procedure to fetch, build, and run the main ConflictMonitor application.
If running on Windows, please make sure that your global git config is set up to not convert end-of-line characters during checkout.
Disable git core.autocrlf (One Time Only)
git config --global core.autocrlf falseEnable Long Filepaths for Git
git config core.longpaths trueNote: if you intend to run the entire ConflictVisualizer system, you should instead start with step 1 of the ConflictVisualizer Installation Guide before returning to step 2 below.
The jpo-conflictmonitor software system consists of the following modules hosted in separate Github repositories:
| Name | Visibility | Description |
|---|---|---|
| jpo-conflictmonitor | public | Contains the public components of the application code. |
| jpo-geojsonconverter | public | Contains the public classes and libraries of the jpo-geojsonconverter used in the Conflict Monitor. |
| jpo-ode | public | Contains the public classes and libraries of the jpo-ode used in the Conflict Monitor. |
| jpo-utils | public | Contains the shared dependencies for jpo repositories |
You may download the stable, default of the Conflict Monitor and jpo-utils repositories run the following command:
git clone --recurse-submodules https://github.com/usdot-jpo-ode/jpo-conflictmonitor.gitOnce you have these repositories obtained, you are ready to build and deploy the application.
A GitHub token is required to pull artifacts from GitHub repositories. This is required to obtain the jpo-ode and jpo-geojsonconverter jars and must be done before attempting to build this repository.
- Log into GitHub.
- Navigate to Settings -> Developer settings -> Personal access tokens.
- Click "New personal access token (classic)".
- As of now, GitHub does not support
Fine-grained tokensfor obtaining packages.
- As of now, GitHub does not support
- Provide a name and expiration for the token.
- Select the
read:packagesscope. - Click "Generate token" and copy the token.
- Copy the token name and token value into your
.envfile. - Create a copy of settings.xml and save it to
~/.m2/settings.xml - Update the variables in your
~/.m2/settings.xmlwith the token value and target jpo-ode organization. Here is an example filled insettings.xmlfile:
<?xml version="1.0" encoding="UTF-8"?>
<settings>
<activeProfiles>
<activeProfile>default</activeProfile>
</activeProfiles>
<servers>
<server>
<id>github_jpo_ode</id>
<username>jpo_conflictmonitor</username>
<password>ghp_token-string-value</password>
</server>
<server>
<id>github_jpo_geojsonconverter</id>
<username>jpo_conflictmonitor</username>
<password>ghp_token-string-value</password>
</server>
</servers>
<profiles>
<profile>
<id>default</id>
<repositories>
<repository>
<id>github_jpo_ode</id>
<name>GitHub JPO ODE</name>
<url>https://maven.pkg.github.com/usdot-jpo-ode/jpo-ode</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
<repository>
<id>github_jpo_geojsonconverter</id>
<name>GitHub JPO GeojsonConverter</name>
<url>https://maven.pkg.github.com/usdot-jpo-ode/jpo-geojsonconverter</url>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
</profile>
</profiles>
</settings>Notes:
- Docker builds may fail if you are on a corporate network due to DNS resolution errors.
- In order for Docker to automatically read the environment variable file, you must rename it from
sample.envto.env. This file will contain private keys, do not put add it to version control.. A copy of thesample.envfile must be created in jpo-utils in order for the base applications (kafka, mongo, etc.) to start up correctly - The MongoDB keyfile should be randomized for each deployment. To Create a new keyfile run the following:
openssl rand -base64 32and then set theMONGO_DB_KEYFILE_STRINGenvironmental variable to its value. **
Navigate to the root directory of the jpo-conflictmonitor project and run the following command:
docker-compose up --build -d
docker-compose psTo bring down the services and remove the running containers run the following command:
docker-compose downFor a fresh restart, run:
docker-compose down
docker-compose up --build -d
docker-compose psTo completely rebuild from scratch, run:
docker-compose down
docker-compose rm -fvs
docker-compose up --build -d
docker-compose ps- The DOCKER_HOST_IP environment variable is used to communicate with the bootstrap server that the instance of Kafka is running on.
In order to utilize Confluent Cloud:
- DOCKER_HOST_IP must be set to the bootstrap server address (excluding the port)
This section outlines the software technology stacks of the GeoJsonConverter.
Install the IDE of your choice:
- Eclipse: https://eclipse.org/
- STS: https://spring.io/tools/sts/all
- IntelliJ: https://www.jetbrains.com/idea/
Deployment of resources is largely managed through docker compose profiles. The available profiles are described in the Conflict Monitor sample.env as well as in the JPO Utils sample.env
## 7. Contact InformationContact the developers of the GeoJsonConverter application by submitting a Github issue.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either expressed or implied. See the License for the specific language governing permissions and limitations under the License.
Please read our contributing guide to learn about our development process, how to propose pull requests and improvements, and how to build and test your changes to this project.
- Main repository on GitHub (public)
- USDOT GeoJsonConverter on Github (public)
- USDOT ITS JPO ODE on Github (public)