feat(sagemaker): add trust remote code to entrypoint

[OlivierDehaene]

@philschmid what do you think?
Fix #390

[philschmid]

Would that mean we need to add another env when using it? Is that what you do for the inference API as well? Or do you fix the transformers fork?

[OlivierDehaene]

Would that mean we need to add another env when using it? Is that what you do for the inference API as well?

Yes and yes

For falcon models I already merged #396 to allow them. This is only for future models.

[monuminu]

Hi @OlivierDehaene , I am stil getting error for Falcon model using sagemaker. Do you know how can fix this .

from sagemaker.huggingface import get_huggingface_llm_image_uri

# retrieve the llm image uri
llm_image = get_huggingface_llm_image_uri(
  "huggingface",
  version="0.7.0"
)

# print ecr image uri
print(f"llm image uri: {llm_image}")

[philschmid]

Hi @OlivierDehaene , I am stil getting error for Falcon model using sagemaker. Do you know how can fix this .

from sagemaker.huggingface import get_huggingface_llm_image_uri

# retrieve the llm image uri
llm_image = get_huggingface_llm_image_uri(
  "huggingface",
  version="0.7.0"
)

# print ecr image uri
print(f"llm image uri: {llm_image}")

@monuminu for SageMaker we currenlty only have the 0.6.0 image available you can find the available versions here: https://github.com/aws/deep-learning-containers/blob/master/available_images.md#huggingface-text-generation-inference-containers

I hope we can soon add the new versions

[philschmid]

LGTM! what happens if the variable is not set? Is it the default None?

[OlivierDehaene]

what happens if the variable is not set? Is it the default None?

Yes