Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability for the TSNE script to use the tfjs-node library #13

Merged
merged 3 commits into from
Jun 19, 2022

Conversation

digitaldogsbody
Copy link
Contributor

When running the TSNE script, the @tensorflow/tfjs library prints a message to console suggesting that the @tensorflow/tfjs-node library is a much faster alternative. This library is already in the package.json, but is not called in the code. I observed that it does not work on all systems (e.g. on Alpine Linux due to differences between musl and glibc that are not fixable with the compatibility layer), so perhaps this is why?

Testing a straight swap of libraries showed a speed improvement of over 30x - based on 43 input images and using time node tsne.js -i /data/vikus/images/input/ for measurement, it went from 2m32s to 5s.

This PR adds the ability to switch to the @tensorflow/tfjs-node library by passing a flag to the script -t. I left the original @tensorflow/tfjs library as the default to ensure that this would not break existing deployments.

@cpietsch
Copy link
Owner

Hey @digitaldogsbody, thanks for looking into that. Indeed the tfjs-node or even tfjs-node with gpu is much faster. I left it in there because locally I always used the tfjs-node module, but there were quite some compatibility problems back then.
Adding the selection of tfjs via a flag is great!

@cpietsch cpietsch merged commit c1bb9da into cpietsch:master Jun 19, 2022
@digitaldogsbody
Copy link
Contributor Author

Thanks for the super quick response and improving my crappy javascript! Is there any chance you could add a tag / do a release with these and your other changes? I am pulling the .tar.gz down as part of a Docker image build, so it would be really helpful for me.

@cpietsch
Copy link
Owner

Sure thing, here it is https://github.com/cpietsch/vikus-viewer-script/releases/tag/v2.0.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants