Skip to content

Conversation

@do-me
Copy link
Contributor

@do-me do-me commented May 20, 2025

Add compilation steps for more than 10x faster inference. Benchmark on an M3 Max:

Without compilation

time cargo run -- encode "Hello world" minishlab/potion-base-8M

0.46s user 0.07s system 62% cpu 0.846 total

With compilation

time model2vec-rs encode "Hello world" "minishlab/potion-base-8M"

0.03s user 0.01s system 90% cpu 0.054 total`

Add compilation steps for more than 10x faster inference. Benchmark on an M3 Max: 

### Without compilation

`cargo run -- encode "Hello world" minishlab/potion-base-8M`

0.46s user 0.07s system 62% cpu 0.846 total

### With compilation

`time model2vec-rs encode "Hello world" "minishlab/potion-base-8M"`

0.03s user 0.01s system 90% cpu 0.054 total`
@Pringled
Copy link
Member

Hey @do-me, thanks for making this PR, good catch! Since the crate is published on crates.io, I think a nicer way to update the readme would be something like this:

cargo install model2vec-rs

model2vec-rs encode "Hello world" minishlab/potion-base-8M

This is a bit nicer than cloning the package and compiling it manually, let me know what you think.

@do-me
Copy link
Contributor Author

do-me commented May 21, 2025

I thought I'd keep it similar to the previous command to have sort of comparable results for the benchmark.
But yes great idea, these lines are useful if you just want to test it out really quickly. I can push some changes to the this PR later but feel free to modify of course!

Also, can you reproduce the 10x performance boost on your device?

@Pringled
Copy link
Member

Nice! Sure thing, feel free to push the changes whenever you have a chance. I do see the same performance. The benchmarks I ran were already compiled though so the performance numbers do stay the same. Perhaps a nice way to structure the readme is to do an "either" in the quickstart of either adding the crate (to use it in a package), or installing the crate, to use it with the CLI. And then in the CLI section it can just be model2vec-rs encode "Hello world" minishlab/potion-base-8M etc instead of cargo run.

- added badges
- added headline "Fast State-of-the-Art Static Embeddings in Rust"
- added more section references 
- added citation 
- fully restructured quickstart section
@do-me
Copy link
Contributor Author

do-me commented May 21, 2025

Completely restructured the readme similar to the Python repo. Also fixed some minor errors (quickstart reference was not clickable; some shell code snippets had Rust linting), aded a few things like citations, badges etc.
Hope this looks appropriate to you!

@Pringled
Copy link
Member

@do-me nice, this looks great, thanks a lot for the updates 😄 ! Good to go, merging!

@Pringled Pringled merged commit e97c015 into MinishLab:main May 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants