Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated to AI Platform #248

Merged
merged 1 commit into from
May 14, 2019
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -1328,7 +1328,7 @@
"id": "cAfB1M0KzAdP"
},
"source": [
"Let us define some environment variables that CMLE uses. Do not forget to replace all the variables in square brackets (along with the square brackets) with your credentials."
"Let us define some environment variables that AI Platform uses. Do not forget to replace all the variables in square brackets (along with the square brackets) with your credentials."
]
},
{
Expand All @@ -1342,7 +1342,6 @@
"outputs": [],
"source": [
"%env GCS_BUCKET=<GCS_BUCKET>\n",
"%env PROJECT_ID=<PROJECT_ID> \n",
"%env REGION=us-central1\n",
"%env LOCAL_DIRECTORY=./trainer/data\n",
"%env TRAINER_PACKAGE_PATH=./trainer"
Expand All @@ -1355,7 +1354,7 @@
"id": "QWQ8kPKc9rYk"
},
"source": [
"ML Engine needs a Python package with our code to train models. We need to create a directory and move our code there. We also need to create an `__init__.py` file, this is a unique feature of python. You can read the [docs](https://docs.python.org/3/tutorial/modules.html#packages) to understand more about this file."
"AI Platform needs a Python package with our code to train models. We need to create a directory and move our code there. We also need to create an `__init__.py` file, this is a unique feature of python. You can read the [docs](https://docs.python.org/3/tutorial/modules.html#packages) to understand more about this file."
]
},
{
Expand Down Expand Up @@ -1590,7 +1589,7 @@
"id": "yStJtBfqFlGF"
},
"source": [
"To actully run the `train.py` file we need some parameters so that CMLE knows how to set up the environment to run sucessfully."
"To actully run the `train.py` file we need some parameters so that AI Platform knows how to set up the environment to run sucessfully."
]
},
{
Expand All @@ -1609,7 +1608,7 @@
"\n",
"echo $JOBNAME \n",
"\n",
"gcloud ml-engine jobs submit training model_training_$JOBNAME \\\n",
"gcloud ai-platform jobs submit training model_training_$JOBNAME \\\n",
" --job-dir $GCS_BUCKET/$JOBNAME/output \\\n",
" --package-path trainer \\\n",
" --module-name trainer.task \\\n",
Expand All @@ -1626,7 +1625,7 @@
"id": "EdG8UV2RL5tV"
},
"source": [
"The above cell submits a job to CMLE which you can view by going to the Google Cloud Console's sidebar and select `ML Engine > Jobs` or search `ML Engine` in the search bar. ONLY run the cells below after your job has completed sucessfully. (It should take approximately 8 minutes to run).\n",
"The above cell submits a job to AI Platform which you can view by going to the Google Cloud Console's sidebar and select `AI Platform > Jobs` or search `AI Platform` in the search bar. ONLY run the cells below after your job has completed sucessfully. (It should take approximately 8 minutes to run).\n",
"\n"
]
},
Expand Down Expand Up @@ -1672,18 +1671,40 @@
"test = test_prepared.as_matrix().tolist()"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "oWGm6c6KIs2O"
},
"source": [
"Next, find the model directory in your GCS bucket that contains the model created in the previous steps."
]
},
{
"cell_type": "code",
"execution_count": 88,
"metadata": {
"colab": {},
"colab_type": "code",
"id": "tJdbJg2jNd05"
},
"source": [
"!gsutil ls $GCS_BUCKET"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "8AlU8Bo2KJS8"
},
"source": [
"Just like training in CMLE, we set some environment variables when we run our command line commands. Note that `<MODEL_DIRECTORY>` can be found in your bucket, it will be of the form `model_YYYYMMDD_HHMMSS` (e.g. `model_190114_134228`)."
"Just like training in AI Platform, we set some environment variables when we run our command line commands. Note that `<GCS_BUCKET>` is your the name of your GCS bucket set earlier. The `MODEL_DIRECTORY` will be inside the GCS bucket and of the form `model_YYYYMMDD_HHMMSS` (e.g. `model_190114_134228`)."
]
},
{
"cell_type": "code",
"cell_type": "code",
"execution_count": null,
"metadata": {
"colab": {},
Expand All @@ -1695,7 +1716,7 @@
"%env VERSION_NAME=v1\n",
"%env MODEL_NAME=cmle_model\n",
"%env JSON_INSTANCE=input.json\n",
"%env MODEL_DIR=gs://<MODEL_DIRECTORY>/model_YYYYMMDD_HHMMSS\n",
"%env MODEL_DIR=gs://<GCS_BUCKET>/MODEL_DIRECTORY\n",
"%env FRAMEWORK=SCIKIT_LEARN"
]
},
Expand All @@ -1719,7 +1740,7 @@
},
"outputs": [],
"source": [
"! gcloud ml-engine models create $MODEL_NAME --regions=us-central1"
"! gcloud ai-platform models create $MODEL_NAME --regions=us-central1"
]
},
{
Expand All @@ -1732,7 +1753,7 @@
},
"outputs": [],
"source": [
"! gcloud ml-engine versions create $VERSION_NAME \\\n",
"! gcloud ai-platform versions create $VERSION_NAME \\\n",
" --model $MODEL_NAME --origin $MODEL_DIR \\\n",
" --runtime-version 1.9 --framework $FRAMEWORK \\\n",
" --python-version 3.5"
Expand Down Expand Up @@ -1796,7 +1817,7 @@
},
"outputs": [],
"source": [
"! gcloud ml-engine predict --model $MODEL_NAME \\\n",
"! gcloud ai-platform predict --model $MODEL_NAME \\\n",
" --version $VERSION_NAME \\\n",
" --json-instances $JSON_INSTANCE"
]
Expand Down Expand Up @@ -1825,11 +1846,11 @@
"import os\n",
"import pandas as pd\n",
"\n",
"PROJECT_ID = os.environ['PROJECT_ID']\n",
"PROJECT_ID = os.environ['GOOGLE_CLOUD_PROJECT']\n",
"VERSION_NAME = os.environ['VERSION_NAME']\n",
"MODEL_NAME = os.environ['MODEL_NAME']\n",
"\n",
"# Create our CMLE service\n",
"# Create our AI Platform service\n",
"service = googleapiclient.discovery.build('ml', 'v1')\n",
"name = 'projects/{}/models/{}'.format(PROJECT_ID, MODEL_NAME)\n",
"name += '/versions/{}'.format(VERSION_NAME)\n",
Expand Down