Theta Informatics LLC - OpenAthena for Android CoT and GCP Data Analysis Tools

This repository contains tools for capturing and analyzing Cursor on Target (CoT) calculation messages generated by OpenAthena for Android (debug builds v0.21.0 and later only) from drone images. The provided R script then compares the calculated locations with pre-recorded ground control points (GCP) to validate the accuracy of terrain-raycast technique used by OpenAthena software.

This testing framework is still under active development and is not guaranteed to be stable

Contents

1. CoT_Listener_Athena_Filtered_to_csv_file_with_calculation_info.py: A Python script to listen for multicast UDP messages and dump the relevant data into a CSV file.
2. Analyze-OA-Android-CoT-and-GCP-csv-data.R: An R script to analyze CoT data output from OpenAthena for Android (with extended debug CoT mode enabled) against GCP data to understand the accuracy and performance of the terrain-raycast technique.
3. Analyze-OA-Core-CoT-and_GCP-csv-data.R: An equivalent R script to analyze CoT data output from OpenAthena Core (with extended debug CoT mode enabled)

CoT_Listener_Athena_Filtered_to_csv_file_with_calculation_info.py

Overview

The Python script CoT_Listener_Athena_Filtered_to_csv_file_with_calculation_info.py is designed to capture multicast UDP messages sent by debug builds of OpenAthena for Android or OpenAthena Core, which include various calculation details and coordinates. The script outputs this data into a timestamped CSV file for later analysis.

Requirements

• Python 3

Usage

Run the script on a system configured to receive UDP multicast messages on the network. The script listens to the multicast group 239.2.3.1 at port 6969 and outputs data to a CSV file named in the format OA-CoT-Capture-{timestamp}.csv.

NOTE: you will need to specify either android or core for the listening mode as a command line argument for the script. The script can only listen in one mode at a time.

E.g. for listening for CoT messages from OpenAthena for Android

python3 CoT_Listener_Athena_Filtered_to_csv_file_with_calculation_info.py android

E.g. for listening for CoT messages from OpenAthena Core

python3 CoT_Listener_Athena_Filtered_to_csv_file_with_calculation_info.py core

Data collection methodology in the field

1. Mark the location of outdoor ground control points (GCP) using SW Maps: Ideally, use an enhanced accuracy GPS such as the Ardusimple RTK Calibrated Surveyor Kit to mark points with greater accuracy than regular GPS. Choose an outdoor area with significant open space for testing. Prepare markers for GCPs which may be easily-visible from a distance by drone, such as traffic cones or these Soccer Disc Cones. In the SW Maps app, create a new project and a new layer. Save the location of each ground control point using SW maps' Record Feature function, ensuring all GCP features are recorded within the only layer of your new Project. For best results, make sure each ground control point is at least 30 meters from its nearest neighboring point.
2. Export the list of outdoor ground control points as a Zipped comma separated values (CSV) file. Share/Export your project in SW maps containing the list of control points as a Zipped CSV file
3. Calibrate your drone's magnetometer (compass) sensor: Consult your drone's operation manual for this procedure. Calibration for the local magnetic environment is critical for obtaining accurate heading information from the drone camera. Lack of calibration can cause heading inaccuracy of up to 10° or more. It is important to perform this procedure away from any ferrous (magnetic) metal such as steel.
4. Take drone photos of your ground control points from multiple slant angles, focal length/zoom, orientations, and drone models (if available). Take as many photos as possible from your drone(s) of your outdoor ground control points. For experimentation, it is desirable to vary the slant angle (downward pitch) of the camera to several values.

Analysis of collected data

The following steps may be performed after experimental data from the field is captured and stored.

4. Start CoT listener for capturing calculations from OpenAthena Android: Set up a Wifi router (such as a phone mobile hotspot, or an offline LAN router will work too) and connect both your computer for running the listener script and your device running OpenAthena for Android. Ensure your connected Wifi router is not configured to drop UDP multicast messages such as Cursor on Target and that your device running OpenAthena for Android is the sole source of CoT messages for devices connected on the network. On your computer, run the script python3 CoT_Listener_to_csv_file.py. The script is now listening for CoT messages the script with the correct paths to these files.
5. Use OpenAthena for Android to obtain target calculations for each ground control point of each image: Using the OpenAthena for Android app, for each grond control point of each image mark the pixel of the ground control point within the image and send it as a CoT message using the "✉️" button. The R script for post-processing analysis will automatically match your calculation output to the nearest ground control point.
6. Run Analysis: Execute the script in an R environment (such as R-studio) to perform statistical analysis. The script will output plots and regression analysis results to understand the impact of various factors on measurement accuracy.

Analyze-OA-Android-CoT-and-GCP-csv-data.R

Overview

The R script Analyze-OA-Android-CoT-and-GCP-csv-data.R loads the captured CoT data and ground control point data to conduct various statistical analyses. It calculates horizontal and vertical errors and examines the dependency of these errors on various factors like slant angle, camera make, model, and focal length.

Another R script Analyze-OA-Core-CoT-and-GCP-csv-data.R is available providing similar functionality for CoT data recorded originating from OpenAthena Core

Requirements

• R
• dplyr package
• ggplot2 package

Usage

# In RStudio or R Console
source('Analyze-OA-CoT-and-GCP-csv-data.R')

Obtaining target location error model parameters for inclusion in droneModels.json

All versions of OpenAthena now use a one factor linear model which predicts target location error (TLE) using slant range from drone to target.

Each drone has a subtly-different magnitude of increase in TLE per unit of range.

The TLE estimation linear model for each drone model is defined by two parameters in droneModels.json:

• tle_model_y_intercept representing the starting value for target location error, e.g. if slant range is 0. This is usually around 5.25 meters, which may be attributed to the (in)accuracy of common GPS units.
• tle_model_slant_range_coeff representing how much (in meters) each additional meter of distance adds to target location error for this drone. Slant range has routinely been observed as the primary factor influencing target location error. Values between 0.02 and 0.035 meters error per meter distance are typical.

The end of this R script calculates these two parameters from your test data with a given drone model. It works by filtering out outliers by Cook’s Distance, then fiting a one factor linear model to the remaining data points.

For more information, read the code at the end of this R script.

A reference is also available in the README.md for DroneModels:

https://github.com/Theta-Limited/DroneModels?tab=readme-ov-file#target-location-error-tle-estimation-model-parameters

Getting Started

To get started, clone this repository and ensure you have the necessary dependencies installed for both Python and R. Perform the experimental data collection, Update the scripts with paths to your specific data files, and run them according to the usage instructions provided above.