Update SageMaker components and sample pipeline #1682

carolynwang · 2019-07-26T00:02:11Z

Summary

@Jeffwan @mbaijal @gautamkmr @kalyc

Added/Updated Dockerfiles for individual components

HPO, train, and batch transform
Note: not actually being used for the components

Updates to HPO component

Make job name optional
Add link to CloudWatch logs in pipeline logs
Add 'training_image' output, which can be used to pipeline with the create model component
Update ReadMe

Updates to train, create model, deploy, and batch transform components

Expose all options to the user
Convert job request creation to use template for all jobs except create endpoint
Update wait for training job to be consistent with other components
Note: SageMaker allows an unlimited number of models per endpoint, but the deploy component only allows a maximum of 3 models. This was done to preserve the ability for users to use the output of the create model component as the input for the deploy component

Add HPO and new parameters to MNIST classification sample pipeline

Show sample usage of all components via an MNIST classification pipeline
Update sample pipeline ReadMe
Note: the default dataset input used in the train component is not actually practical and is just being used to demonstrate that the component works

Testing

Updated components functionality

mnist-classification-pipeline.py compiles and runs without errors using the default values set in the pipeline parameters
Note: HPO and train components have been tested with BYOC and the K-Means built-in algorithm

This change is

…HPO pipeline

k8s-ci-robot · 2019-07-26T00:02:26Z

Hi @carolynwang. Thanks for your PR.

I'm waiting for a kubeflow member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Ark-kun · 2019-07-26T00:51:15Z

/ok-to-test

components/aws/sagemaker/batch_transform/Dockerfile

components/aws/sagemaker/batch_transform/component.yaml

Jeffwan · 2019-07-26T22:15:02Z

components/aws/sagemaker/batch_transform/src/batch_transform.py

+  parser.add_argument('--input_filter', type=str, required=False, help='A JSONPath expression used to select a portion of the input data to pass to the algorithm.', default='')
+  parser.add_argument('--output_filter', type=str, required=False, help='A JSONPath expression used to select a portion of the joined dataset to save in the output file for a batch transform job.', default='')
+  parser.add_argument('--join_source', choices=['None', 'Input', ''], required=False, help='Specifies the source of the data to join with the transformed data.', default='None')
+  parser.add_argument('--instance_type', choices=['ml.m4.xlarge', 'ml.m4.2xlarge', 'ml.m4.4xlarge', 'ml.m4.10xlarge', 'ml.m4.16xlarge', 'ml.m5.large', 'ml.m5.xlarge', 'ml.m5.2xlarge', 'ml.m5.4xlarge',


Minor: We need to consider how to better maintain this list. It could easily out of sync once SM rollout new instances

Is there a way to fetch supported instances using sagemaker api? like the way you get built-in algorithm?

There isnt a quick search through the sagemaker documentation

Jeffwan · 2019-07-26T22:17:52Z

components/aws/sagemaker/common/_utils.py

+        request['AlgorithmSpecification']['TrainingImage'] = args['image']
+        request['AlgorithmSpecification'].pop('AlgorithmName')
+    else:
+        # TODO: determine if users can make custom algorithm resources that have the same name as built-in algorithm names


How is custom algorithm supported here? Ideally, user should use BYOC or first party algorithms. No more options should be exposed

Custom algorithm is referring to algorithms that users create as an algorithm resource in SageMaker. The API has an AlgorithmName param for these custom algos and algos from AWS marketplace

OK. Have you tried to create a custom algorithm using exist first-party algorithm name? I do think their API object is different or same name can not be used.

Just tried it, created an algo with the name 'K-Means' and it worked so I need to adjust this implementation

components/aws/sagemaker/common/_utils.py

components/aws/sagemaker/train/Dockerfile

Jeffwan · 2019-07-26T22:28:30Z

/cc @Jeffwan

Jeffwan · 2019-07-30T00:22:22Z

/retest

Jeffwan · 2019-07-30T18:46:25Z

/lgtm
/approve

k8s-ci-robot · 2019-07-30T18:46:29Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Jeffwan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~components/aws/OWNERS~~ [Jeffwan]
~~samples/aws-samples/OWNERS~~ [Jeffwan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

See the rationale at https://github.com/hadolint/hadolint/wiki/DL3042 In addition to space saving, it also ensures that pip doesn't install from local cache (especially in case of a rebuild) and reduces the chance of creating bugs during development.

carolynwang added 4 commits July 24, 2019 15:16

Update HPO, train, batch transform components, add MNIST kmeans with …

d30283f

…HPO pipeline

Minor bug fixes

4a7b72a

Minor bug fixes, update Dockerfiles

2f9b780

Update docker images and sample pipeline

51161c5

k8s-ci-robot added the size/XL label Jul 26, 2019

k8s-ci-robot requested review from Jeffwan and mameshini July 26, 2019 00:02

k8s-ci-robot added the needs-ok-to-test label Jul 26, 2019

carolynwang changed the title ~~Sagemaker components~~ Update SageMaker HPO, train, and batch transform components Jul 26, 2019

carolynwang changed the title ~~Update SageMaker HPO, train, and batch transform components~~ Update SageMaker HPO, train, and batch transform components and sample pipeline Jul 26, 2019

k8s-ci-robot added ok-to-test and removed needs-ok-to-test labels Jul 26, 2019

Jeffwan suggested changes Jul 26, 2019

View reviewed changes

k8s-ci-robot requested a review from Jeffwan July 26, 2019 22:28

Update all components and sample pipeline

6ea1e5b

k8s-ci-robot added size/XXL and removed size/XL labels Jul 29, 2019

carolynwang changed the title ~~Update SageMaker HPO, train, and batch transform components and sample pipeline~~ Update SageMaker components and sample pipeline Jul 29, 2019

carolynwang and others added 2 commits July 29, 2019 13:31

Delete Dockerfiles for individual component

80556bb

Typo fix in Dockerfile

aa25e22

k8s-ci-robot assigned Jeffwan Jul 30, 2019

k8s-ci-robot added the lgtm label Jul 30, 2019

k8s-ci-robot added the approved label Jul 30, 2019

k8s-ci-robot merged commit 81b0f08 into kubeflow:master Jul 30, 2019

carolynwang deleted the sagemaker-components branch July 30, 2019 23:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update SageMaker components and sample pipeline #1682

Update SageMaker components and sample pipeline #1682

carolynwang commented Jul 26, 2019 •

edited

Loading

k8s-ci-robot commented Jul 26, 2019

Ark-kun commented Jul 26, 2019

Jeffwan Jul 26, 2019

Jeffwan Jul 26, 2019

carolynwang Jul 26, 2019 •

edited

Loading

Jeffwan Jul 26, 2019

carolynwang Jul 26, 2019

Jeffwan Jul 27, 2019

carolynwang Jul 29, 2019

Jeffwan commented Jul 26, 2019

Jeffwan commented Jul 30, 2019

Jeffwan commented Jul 30, 2019

k8s-ci-robot commented Jul 30, 2019

Update SageMaker components and sample pipeline #1682

Update SageMaker components and sample pipeline #1682

Conversation

carolynwang commented Jul 26, 2019 • edited Loading

Summary

Testing

k8s-ci-robot commented Jul 26, 2019

Ark-kun commented Jul 26, 2019

Jeffwan Jul 26, 2019

Choose a reason for hiding this comment

Jeffwan Jul 26, 2019

Choose a reason for hiding this comment

carolynwang Jul 26, 2019 • edited Loading

Choose a reason for hiding this comment

Jeffwan Jul 26, 2019

Choose a reason for hiding this comment

carolynwang Jul 26, 2019

Choose a reason for hiding this comment

Jeffwan Jul 27, 2019

Choose a reason for hiding this comment

carolynwang Jul 29, 2019

Choose a reason for hiding this comment

Jeffwan commented Jul 26, 2019

Jeffwan commented Jul 30, 2019

Jeffwan commented Jul 30, 2019

k8s-ci-robot commented Jul 30, 2019

carolynwang commented Jul 26, 2019 •

edited

Loading

carolynwang Jul 26, 2019 •

edited

Loading