Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to reuse server for pytorch models #245

Merged
merged 5 commits into from
Nov 14, 2024
Merged

Allow to reuse server for pytorch models #245

merged 5 commits into from
Nov 14, 2024

Conversation

jinjingforever
Copy link
Collaborator

@jinjingforever jinjingforever commented Nov 14, 2024

Also added tests to cover several pytorch related use cases.

The new API can be used as follows. I will update wiki after release next week:

# Assume the model explorer server has started at port 8080.

import model_explorer

# Prepare a pytorch model.
model = torchvision.models.mobilenet_v2().eval()
inputs = (torch.rand([1, 3, 224, 224]),)
ep = torch.export.export(model, inputs)

# Load it into the existing server.
model_explorer.visualize_pytorch('test pytorch', ep, reuse_server=True) # 👈

or:

import model_explorer

# Prepare a pytorch model.
model = torchvision.models.mobilenet_v2().eval()
inputs = (torch.rand([1, 3, 224, 224]),)
ep = torch.export.export(model, inputs)

# Load it into the existing server.
config.add_model_from_pytorch('torch test', ep).set_reuse_server() # 👈
model_explorer.visualize_from_config(config)

Fixes #231

@yijie-yang
Copy link
Collaborator

Thanks for adding more coverages in our CI!

@jinjingforever
Copy link
Collaborator Author

Thanks for adding more coverages in our CI!

Thanks for setting this up! Really nice to use and debug.

@jinjingforever jinjingforever merged commit 1e5e178 into main Nov 14, 2024
3 checks passed
@jinjingforever jinjingforever deleted the reuseserver branch November 14, 2024 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

reuse_server() does not work with pytorch ExportedProgram
2 participants