Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Neo] Add Neo Neuron entrypoint script #1752

Merged
merged 2 commits into from
Apr 9, 2024

Conversation

a-ys
Copy link
Contributor

@a-ys a-ys commented Apr 9, 2024

Description

This PR adds an entrypoint script that enables the LMI Neuron (pytorch-inf2) container to be used by SageMaker Neo to AOT compile models. This script does not modify default djl-serving behavior.

Changes to djl-serving code

  • A check is added to model_loader.py to check if the partitioning output directory is mounted before calling rmtree() on it, which would cause an error.
  • Added an optional "addl_properties" kwarg field to Partition PropertiesManager to enable passing model properties without parsing through serving.properties.

@a-ys a-ys requested review from zachgk, frankfliu and a team as code owners April 9, 2024 02:54
@tosterberg tosterberg merged commit ef92c2b into deepjavalibrary:master Apr 9, 2024
8 checks passed
lanking520 pushed a commit that referenced this pull request Apr 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants