Skip to content

Commit

Permalink
Add Initial version
Browse files Browse the repository at this point in the history
  • Loading branch information
thomas694 committed May 30, 2021
1 parent f68127b commit ca4abd8
Show file tree
Hide file tree
Showing 32 changed files with 1,614 additions and 1 deletion.
17 changes: 17 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Auto detect text files and perform LF normalization
* text=auto

# Custom for Visual Studio
*.cs diff=csharp

# Standard to msysgit
*.doc diff=astextplain
*.DOC diff=astextplain
*.docx diff=astextplain
*.DOCX diff=astextplain
*.dot diff=astextplain
*.DOT diff=astextplain
*.pdf diff=astextplain
*.PDF diff=astextplain
*.rtf diff=astextplain
*.RTF diff=astextplain
38 changes: 38 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ mono_crash.*
[Rr]eleases/
x64/
x86/
[Ww][Ii][Nn]32/
[Aa][Rr][Mm]/
[Aa][Rr][Mm]64/
bld/
Expand Down Expand Up @@ -61,6 +62,9 @@ project.lock.json
project.fragment.lock.json
artifacts/

# ASP.NET Scaffolding
ScaffoldingReadMe.txt

# StyleCop
StyleCopReport.xml

Expand All @@ -86,6 +90,7 @@ StyleCopReport.xml
*.tmp_proj
*_wpftmp.csproj
*.log
*.tlog
*.vspscc
*.vssscc
.builds
Expand Down Expand Up @@ -137,6 +142,11 @@ _TeamCity*
.axoCover/*
!.axoCover/settings.json

# Coverlet is a free, cross platform Code Coverage Tool
coverage*.json
coverage*.xml
coverage*.info

# Visual Studio code coverage results
*.coverage
*.coveragexml
Expand Down Expand Up @@ -196,6 +206,9 @@ PublishScripts/
*.nuget.props
*.nuget.targets

# Nuget personal access tokens and Credentials
nuget.config

# Microsoft Azure Build Output
csx/
*.build.csdef
Expand Down Expand Up @@ -348,3 +361,28 @@ MigrationBackup/

# Ionide (cross platform F# VS Code tools) working folder
.ionide/

# Fody - auto-generated XML schema
FodyWeavers.xsd

# VS Code files for those working on multiple tools
.vscode/*
!.vscode/settings.json
!.vscode/tasks.json
!.vscode/launch.json
!.vscode/extensions.json
*.code-workspace

# Local History for Visual Studio Code
.history/

# Windows Installer files from build outputs
*.cab
*.msi
*.msix
*.msm
*.msp

# JetBrains Rider
.idea/
*.sln.iml
1 change: 1 addition & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
Expand Down
14 changes: 14 additions & 0 deletions NOTICE
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
Copyright 2021 thomas694 (@GH)

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this project and its files except in compliance
with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
108 changes: 107 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,107 @@
# Classificationbox_Toolkit
# Classificationbox Toolkit

You can use this toolkit to build your own image classification solution.<br/>
The toolkit consists of two projects:
- Classificationbox.Net - a .Net wrapper for Machine Box's Classificationbox service (.NET Standard 2.0 library)
- imgclass.net - a command line program using the Classificationbox.Net library to train your model and sort your images

The `imgclass.net` tool is a real-world command line program that shows how the `Classificationbox.Net` library can be
used to train a model and later classify folders of images. It's a solution where you don't have to upload your images
to an online service or use a tool which extracts "features" which are send to the service. The solution can be run
disconnected from the internet. Also it is not only a example implementation which lacks features for a real life usage.
It's used in the field and will be improved and extended with new features as the necessity arrises.

## Prerequisites

For using this toolkit you need the following (free) components:
- [Docker](https://www.docker.com/products/docker-desktop)<br/>
You should run your docker daemon with at least 2 CPUs and 4GB RAM. The more CPUs you assign the faster is your training and classification process.
- [Machine Box's Classificationbox docker image](https://machinebox.io/docs/classificationbox)

## Usage

1. Prepare training images
1. Run Classificationbox
1. Train and test your model
1. Classify (sort) a folder of images

### Prepare training images

Create a directory structure that organizes the images into classes, with each folder as the class name:

```
/training-images
/class1
class1example1.jpg
class1example2.jpg
...
/class2
class2example1.jpg
class2example2.jpg
...
/class3
class3example1.jpg
class3example2.jpg
...
```

You should put roughly the same amount of images in every class folder.

### Run Classificationbox

In a terminal do:

```
docker run -p 8080:8080 -e "MB_KEY=$MB_KEY" machinebox/classificationbox
```

* Get yourself an `MB_KEY` from https://machinebox.io/account

### Train and test your model

Use the `imgclass.net` tool to train the model:

```
imgclass.net -cb http://localhost:8080 -model Abc -src ./training-images -trainratio 0.8 -passes 3
```

The tool will post a random 80% (`-trainratio 0.8`) of the images to Classificationbox for training, and the
remaining images will be used to test the model. The process is repeated three times (`-passes 3`) to train
the model better and make later predictions more accurate. You should use roughly the same amount of images
for every class, so that the model doesn't get biased towards any class.

The tool logs information to the console and a log file in the src folder.
At the end the trained model and the state of the training files are automatically saved to the source folder.
On the next run they will be loaded again.

### Classify (sort) a folder of images

After you have trained your model you can use `imgclass.net` tool to sort a folder of images

```
imgclass.net -cb http://localhost:8080 -model Abc -src ./training-images -classify C:\ImageFolder -threshold 0.95
```

The tool requests an image class prediction for each image and moves the file to a subfolder according to the
predicted class and its accuracy. Related to the `training-images` example the images will be sorted into subfolders:
```
/ImageFolder
/class1
/class1_check
/class2
/class2_check
/class3
/class3_check
```

Images moved to `class1` have a accuracy of 95% (`-threshold 0.95`) or higher. Those moved to `class1_check` have
a lower accuracy and need to be checked.

## Download

Latest versions can be found [here](https://github.com/thomas694/Classification_Toolkit/releases).

## Contributing to Classificationbox Toolkit

PRs and contributions of any kind are welcome!<br/>
Please open an issue and introduce your planned changes first, before starting major work.
106 changes: 106 additions & 0 deletions appveyor.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
#skip_branch_with_pr: true
image: Visual Studio 2019
install:
# Set "build version number" to current version or when tagged to "tag name"
- ps: >-
if ($env:APPVEYOR_REPO_TAG -eq "true")
{
$x = $env:APPVEYOR_REPO_TAG_NAME
if($x.startswith("v","CurrentCultureIgnoreCase")) { $x = $x.substring(1) }
$env:MY_BUILD_VERSION="$x"
$x = $x + ".$env:APPVEYOR_BUILD_NUMBER"
Update-AppveyorBuild -Version "$x"
}
else
{
Set-AppveyorBuildVariable -Name TT_VER -Value (& "$env:APPVEYOR_BUILD_FOLDER\scripts\get-version.ps1" -versionInfoFile "$env:APPVEYOR_BUILD_FOLDER\src\imgclass.net\Properties\AssemblyInfo.cs")
$s = $env:TT_VER
[VERSION]$vs = $s -replace '^.+((\d+\.){3}\d+).+', '$1'
$s = '{0}.{1}.{2}' -f $vs.Major,$vs.Minor,$vs.Build
Update-AppveyorBuild -Version "$s.$env:APPVEYOR_BUILD_NUMBER"
}
environment:
matrix:
- job_name: build_skip
job_group: build
- job_name: build_Any_m
job_group: build
- job_name: build_Any
job_group: build
- job_name: deployZip
job_depends_on: build
matrix:
fast_finish: true
assembly_info:
patch: true
file: '**\AssemblyInfo.cs'
assembly_version: '{version}'
assembly_file_version: '{version}'
assembly_informational_version: '{version}'
configuration: Release
branches:
except:
- dummy_branch
for:
-
matrix:
only:
- job_name: build_skip
skip_commits:
files:
- src/
skip_tags: true
init:
- appveyor exit
environment:
PLATFORM: Any CPU
-
matrix:
only:
- job_name: build_Any_m
only_commits:
files:
- src/
skip_tags: true
environment:
PLATFORM: Any CPU
-
matrix:
only:
- job_name: build_Any
skip_non_tags: true
environment:
PLATFORM: Any CPU
-
matrix:
only:
- job_name: deployZip
skip_non_tags: true
build: off
environment:
DeployNow: true
before_build:
- ps: scripts/appveyor-prebuild.ps1
nuget restore "src\Classificationbox_Toolkit.sln"
build:
project: src\Classificationbox_Toolkit.sln
verbosity: normal
after_build:
- ps: scripts/appveyor-postbuild.ps1
artifacts:
- path: artifacts/*.zip
name: Classificationbox_Toolkit
before_deploy:
- ps: scripts/appveyor-predeploy.ps1
deploy:
release: Classificationbox_Toolkit-v$(MY_BUILD_VERSION)
description: 'Release description'
provider: GitHub
auth_token:
secure: PWzd/WicmutMxrc66FtBXAQMgCUKsX8yjeqv+2SHZyflLOTYZz5393rynVUtCltZ
artifact: /.*\.zip/ # upload all zips to release assets
draft: true
prerelease: false
on:
#branch: master # release from master branch only
DeployNow: true # deploy on tag push only
20 changes: 20 additions & 0 deletions scripts/appveyor-postbuild.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Harvest Data
$harvestPath = "$env:APPVEYOR_BUILD_FOLDER\src\imgclass.net\bin\Release"
$harvestPath2 = "$env:APPVEYOR_BUILD_FOLDER\src\Classificationbox.Net\bin\Release\netstandard2.0"
$fileVersion = (Get-Item "$harvestPath\ImgClass.Net.exe").VersionInfo.ProductVersion
[VERSION]$vs = $fileVersion -replace '^.+((\d+\.){3}\d+).+', '$1'
$version = '{0}.{1}.{2}' -f $vs.Major,$vs.Minor,$vs.Build

# Artifacts Paths
$artifactsPath = "$env:APPVEYOR_BUILD_FOLDER\artifacts"
$applicationArtifactsPath = "$artifactsPath\Classificationbox_Toolkit"

New-Item -ItemType Directory -Force -Path $applicationArtifactsPath

# Copy in Application Artifacts
Get-ChildItem -Path "$harvestPath\*" -Include *.exe,*.dll,*.config | Copy-Item -Destination $applicationArtifactsPath
Get-ChildItem -Path "$harvestPath2\*" -Include *.dll,*.pdb,*.json | Copy-Item -Destination $applicationArtifactsPath

# Zip Application
$applicationZipPath = "$artifactsPath\Classificationbox_Toolkit-v$version.zip"
Compress-Archive -Path "$artifactsPath\Classificationbox_Toolkit\" -DestinationPath "$applicationZipPath"
Empty file added scripts/appveyor-prebuild.ps1
Empty file.
Empty file added scripts/appveyor-predeploy.ps1
Empty file.
11 changes: 11 additions & 0 deletions scripts/get-version.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
param ([Parameter(Mandatory=$true)][string]$versionInfoFile)

$RegularExpression = [regex] 'AssemblyFileVersion\(\"(.*)\"\)'
$fileContent = Get-Content $versionInfoFile
foreach($content in $fileContent)
{
$match = [System.Text.RegularExpressions.Regex]::Match($content, $RegularExpression)
if($match.Success) {
$match.groups[1].value
}
}
18 changes: 18 additions & 0 deletions src/Classificationbox.Net/Classificationbox.Net.csproj
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>netstandard2.0</TargetFramework>
<GenerateAssemblyInfo>false</GenerateAssemblyInfo>
<Authors>thomas694</Authors>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Microsoft.CSharp" Version="4.7.0" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
</ItemGroup>

<ItemGroup>
<None Include="Properties\AssemblyInfo.cs" />
</ItemGroup>

</Project>
6 changes: 6 additions & 0 deletions src/Classificationbox.Net/Models/CreateModelRequest.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
namespace Classificationbox.Net.Models
{
public class CreateModelRequest
{
}
}
Loading

0 comments on commit ca4abd8

Please sign in to comment.