Skip to content

Commit

Permalink
Merging README.MD
Browse files Browse the repository at this point in the history
  • Loading branch information
hugozanini committed Jun 21, 2020
2 parents 3ee0b9a + 1c7be90 commit 57caf01
Show file tree
Hide file tree
Showing 3 changed files with 49 additions and 5 deletions.
54 changes: 49 additions & 5 deletions README.MD
Original file line number Diff line number Diff line change
@@ -1,15 +1,59 @@

# Realtime instance segmentation in the browser


# Realtime Semantic Segmentation in the browser

This project uses TensorFlow.js to perform realtime instance segmentation in the browser using [RefineNet](https://arxiv.org/pdf/1611.06612.pdf). Many thanks to [Vladimir Nekrasov](https://drsleep.github.io/) that provided the *.js weights* of the [Light-Weight RefineNet](http://bmvc2018.org/contents/papers/0494.pdf) - a more compact implementation of RefineNet, suitable for tasks requiring real-time performance.

This project uses TensorFlow.js to perform realtime instance segmentation in the browser using [RefineNet](https://arxiv.org/pdf/1611.06612.pdf). Many thanks to [Bernardo de Oliveira](https://github.com/bernieOllie), that help me to fix some bugs in the code, and [Vladimir Nekrasov](https://drsleep.github.io/) that provided the *.js weights* of the [Light-Weight RefineNet](http://bmvc2018.org/contents/papers/0494.pdf) - a more compact implementation of RefineNet, suitable for tasks requiring real-time performance.

![enter image description here](./git_media/demo.gif)


![enter image description here](./git_media/objects_demo.gif)



### Getting started
Tensorflow.js models have to be served through an URL to be loaded in javascript. The Refine Net model is based on a main JSON file and shared weights files in a binary format. The idea is to create an HTTP server that will make the model available in a URL allowing requests and be thread as REST API. When loading the model, the TensorFlow.js will do the following requests:

GET /model.json
GET /group1-shard1of4
GET /group1-shard2of4
GET /group1-shard3of4
GET /group1-shard4of4

The [http-server](https://www.npmjs.com/package/http-server) can be installed using the following command:

`npm-install http-server -g`

Once you have the program installed, go to the weights folder, run the command below and the model you be available at http://127.0.0.1:8080.

`http-server -c1 --cors .`

To install the required packages, it can be used `npm` or `yarn`.

If you are using `yarn`:

yarn
yarn start

If you are using `npm`:

npm install
npm start

To test the project just go to [http://localhost:3000/] and the application will be running there.

### References
[1] Romera-Paredes, Bernardino, and Philip Hilaire Sean Torr. “Recurrent instance segmentation.” _European conference on computer vision_. Springer, Cham, 2016.

[2] Lin, Guosheng, et al. “Refinenet: Multi-path refinement networks for high-resolution semantic segmentation.” _Proceedings of the IEEE conference on computer vision and pattern recognition_. 2017.

[3] Nekrasov, Vladimir, Chunhua Shen, and Ian Reid. “Light-weight refinenet for real-time semantic segmentation.” _arXiv preprint arXiv:1810.03272_ (2018).

[4] Medium. 2020. _Introducing Tensorflow.Js: Machine Learning In Javascript_. [online] Available at: <https://medium.com/tensorflow/introducing-tensorflow-js-machine-learning-in-javascript-bf3eab376db> [Accessed 3 June 2020].

[5] [https://www.tensorflow.org/js/tutorials/conversion/import_keras](https://www.tensorflow.org/js/tutorials/conversion/import_keras)

TODO
[6] [https://github.com/DrSleep/light-weight-refinenet](https://github.com/DrSleep/light-weight-refinenet)

[7] [https://www.tensorflow.org/js/guide/conversion](https://www.tensorflow.org/js/guide/conversion)
Binary file removed git_media/demo.gif
Binary file not shown.
Binary file added git_media/objects_demo.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 57caf01

Please sign in to comment.