Add ability for the TSNE script to use the tfjs-node library #13
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When running the TSNE script, the
@tensorflow/tfjs
library prints a message to console suggesting that the@tensorflow/tfjs-node
library is a much faster alternative. This library is already in thepackage.json
, but is not called in the code. I observed that it does not work on all systems (e.g. on Alpine Linux due to differences between musl and glibc that are not fixable with the compatibility layer), so perhaps this is why?Testing a straight swap of libraries showed a speed improvement of over 30x - based on 43 input images and using
time node tsne.js -i /data/vikus/images/input/
for measurement, it went from 2m32s to 5s.This PR adds the ability to switch to the
@tensorflow/tfjs-node
library by passing a flag to the script-t
. I left the original@tensorflow/tfjs
library as the default to ensure that this would not break existing deployments.