Skip to content

Commit

Permalink
Feature/mac os app (#17)
Browse files Browse the repository at this point in the history
* Add base version of the preprocessor

* Get rid of DS_Stores

* Add new demonstration code

* Add new manager for processing of complete dataset

* Add new metadata to the resulting CSV files, add documentation

* Add method for video size calculation, add documentation

* Add video size parameter

* Update the project to new, non-deprecated values

* Fixes the bug causing an error on demonstration run and removes redundant observation configurtion

* Init macOS version

* Add swiftlint and remove static methods

* Clean and format code

* Set architecture

* Move logic into viewmodel

* Remove UI test target

* setup base for annotation video and dataset

* Introduce settings, add titles

* Add settings binding

* Change to utilities category

* Add new name

* Change background color

* Remove duplicated project

* Introduce settings, add titles

* Add settings binding

* Change to utilities category

* Add new name

* Implement the values from Preferences into the actual analyses settings

* Create skeleton of README.md

* Update README.md

* Add Usage

* Update README.md

* Update README.md

* Add icon proposed (temporary) and fix certain dataset preprocessing bugs

* Fix bug leading to crashes once certain body parts are visible only for periods of the recordings

* Fix bug causing crashes when certain parts of body are not visible for parts of recording

* Update macOS/DataAnnotation/Managers/VisionAnalysisManager.swift

Make the for loop with condition more efficient

Co-authored-by: Đỗ Long Thành <thanh.dolong@gmail.com>

* Fix the missing bracket

* Add additional keys to the analysis

* Update README.md

* Update README.md

* Update README.md

* Fix data filling mechanism for datasets

* Introduce new icons and settings

* Annotate video input asynchronously

* Add links to icons file; add face annotations

* Implement the FPS configuration from settings

* Fix filling up of empty landmarks, fix processing of both hands, remove redundant logs, and more

* Introduce body parts analysis configuration in the app's setting

* Fix AppStorage types

* Memory optimization

* Fix

* Merge design

* Fix setup userdefault

* remove unused code

* Fix appending visual data

Co-authored-by: Matyáš Boháček <matyas.bohacek@matsworld.io>
Co-authored-by: Rastislav Cervenak <cervenak.rastislav@gmail.com>
  • Loading branch information
3 people authored May 6, 2021
1 parent 38a322e commit 8fd99bb
Show file tree
Hide file tree
Showing 70 changed files with 3,785 additions and 9 deletions.
15 changes: 7 additions & 8 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,15 @@
# gitignore contributors: remember to update Global/Xcode.gitignore, Objective-C.gitignore & Swift.gitignore

## User settings
xcuserdata/
macOS/xcuserdata/

## compatibility with Xcode 8 and earlier (ignoring not required starting Xcode 9)
*.xcscmblueprint
*.xccheckout

## compatibility with Xcode 3 and earlier (ignoring not required starting Xcode 4)
build/
DerivedData/
macOS/build/
macOS/DerivedData/
*.moved-aside
*.pbxuser
!default.pbxuser
Expand All @@ -31,7 +31,7 @@ DerivedData/
*.dSYM

## Playgrounds
timeline.xctimeline
preprocessor/timeline.xctimeline
playground.xcworkspace

# Swift Package Manager
Expand All @@ -54,10 +54,8 @@ playground.xcworkspace
# you should judge for yourself, the pros and cons are mentioned at:
# https://guides.cocoapods.org/using/using-cocoapods.html#should-i-check-the-pods-directory-into-source-control
#
# Pods/
#
# Add this line if you want to avoid checking in source code from the Xcode workspace
# *.xcworkspace
macOS/Pods/
macOS/*.xcworkspace

# Carthage
#
Expand Down Expand Up @@ -88,3 +86,4 @@ fastlane/test_output
# https://github.com/johnno1962/injectionforxcode

iOSInjectionProject/
.DS_Store
55 changes: 54 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,54 @@
# sign-language-recognition-trainer
# Pose Data Annotator

The **Pose Data Annotator** app is a handy desktop app for seamless Vision analysis of body, hand and face landmarks for applied tasks such as sign language recognition, action classification, and more. The app makes it easy to annotate both single videos and large datasets, and output the data in a standardized CSV file for any further work.

![App screenshot 1: Annotate video](http://data.matsworld.io/body-pose-annotator/screenshot_1.png)
![App screenshot 2: Annotate dataset](http://data.matsworld.io/body-pose-annotator/screenshot_2.png)

### Backbone

The backbone of the body, hand, and face pose analysis is built on top of the [Vision](https://developer.apple.com/documentation/vision) framework by Apple. We have found it to be working significantly better and more efficiently than the other publicly available frameworks.

### Data format

The data is saved in a custom data format into a `.csv` file. More information regarding the formatting system can be found [here](data_format.md). The app also supports the following data annotations formats:

- ...

## Installation

### Download the app

The latest (as well as all the previous stable builds) can be downloaded as a `.dmg` here.

#### Requirements

- macOS 11.0+ (Big Sur and above)

### Build from source

You can build Latest directly on your machine. To do that, you have to download the source code by cloning the repository: `git clone https://github.com/thanhdolong/sign-language-recognition-trainer.git`.

Then you can open the BodyPoseAnnotator.xcodeproj and hit **Build and Run**. Make sure that the BodyPoseAnnotator scheme is selected.

#### Requirements

- macOS 11.0+ (Big Sur and above)
- Xcode 12.0+
- Swift 5

## Usage

The app has two main ways for annotation – either from single videos, or from full datasets. Either way, please ensure that all of your videos are in the `.mp4` format.

### Single video

To annotate a single video, simply select the **Annotate video** section in the left navigation bar. Then either drag-and-drop your file, or select it using the **Load Video** button. Then start the analysis using the **Start processing** button. Once the analysis is finished, you will be prompted with an alert view to save the resulting `.csv` file.

### Full dataset

To annotate a full dataset, ensure that you have a dataset folder with structured folders of the individual labels and the videos inside them. There should also be no other files irrelevant to the analysis. Then select the **Annotate dataset** section in the left navigation bar and either drag-and-drop your folder, or select it using the **Load Dataset** button. You can start the analysis using the **Start processing** button. Once the analysis is finished, you will be prompted with an alert view to save the resulting `.csv` file.

## Contribution

Any contribution is highly encouraged and welcome! You can take a look at the [Issues](https://github.com/thanhdolong/sign-language-recognition-trainer/issues) section to see what you can do. If you spot a bug, please file a Bug Report, or if you have some idea of your own, please submit a Feature Request. Use the according templates for both please and provide as much information or context as possible.
Loading

0 comments on commit 8fd99bb

Please sign in to comment.