chore: move samples from python-docs-sample to here (#29)

* Data Labeling Beta samples [(#2096)](#2096) * add files * upate create_annotation_spec_set and test * add requirements.txt * update create_instruction and test * update import data and test * add label image and test * add label_text test * add label_video_test * add manage dataset and tests * flake * fix * add README * Adds updates including compute [(#2436)](#2436) * Adds updates including compute * Python 2 compat pytest * Fixing weird \r\n issue from GH merge * Put asset tests back in * Re-add pod operator test * Hack parameter for k8s pod operator * Update datalabeling samples to hit test endpoint. [(#2641)](#2641) * Auto-update dependencies. [(#2005)](#2005) * Auto-update dependencies. * Revert update of appengine/flexible/datastore. * revert update of appengine/flexible/scipy * revert update of bigquery/bqml * revert update of bigquery/cloud-client * revert update of bigquery/datalab-migration * revert update of bigtable/quickstart * revert update of compute/api * revert update of container_registry/container_analysis * revert update of dataflow/run_template * revert update of datastore/cloud-ndb * revert update of dialogflow/cloud-client * revert update of dlp * revert update of functions/imagemagick * revert update of functions/ocr/app * revert update of healthcare/api-client/fhir * revert update of iam/api-client * revert update of iot/api-client/gcs_file_to_device * revert update of iot/api-client/mqtt_example * revert update of language/automl * revert update of run/image-processing * revert update of vision/automl * revert update testing/requirements.txt * revert update of vision/cloud-client/detect * revert update of vision/cloud-client/product_search * revert update of jobs/v2/api_client * revert update of jobs/v3/api_client * revert update of opencensus * revert update of translate/cloud-client * revert update to speech/cloud-client Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com> Co-authored-by: Doug Mahugh <dmahugh@gmail.com> * Update datalabeling to match lint. [(#2642)](#2642) * datalabeling: ensure all tests use test endpoint [(#2918)](#2918) * datalabeling: ensure all tests use test endpoint * requires an input csv for text input, slight print statement cleanup Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com> * chore(deps): update dependency google-cloud-datalabeling to v0.4.0 [(#3081)](#3081) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [google-cloud-datalabeling](https://togithub.com/googleapis/python-datalabeling) | minor | `==0.3.0` -> `==0.4.0` | --- ### Release Notes <details> <summary>googleapis/python-datalabeling</summary> ### [`v0.4.0`](https://togithub.com/googleapis/python-datalabeling/blob/master/CHANGELOG.md#040-httpswwwgithubcomgoogleapispython-datalabelingcomparev030v040-2020-01-31) [Compare Source](https://togithub.com/googleapis/python-datalabeling/compare/v0.3.0...v0.4.0) ##### Features - **datalabeling:** undeprecate resource name helper methods (via synth) ([#10039](https://www.github.com/googleapis/python-datalabeling/issues/10039)) ([88f8090](https://www.github.com/googleapis/python-datalabeling/commit/88f809008ee6a709c02c78b1d93af779fab19adb)) ##### Bug Fixes - **datalabeling:** deprecate resource name helper methods (via synth) ([#9832](https://www.github.com/googleapis/python-datalabeling/issues/9832)) ([e5f9021](https://www.github.com/googleapis/python-datalabeling/commit/e5f902154ebe7fcb139aa405babfe9993fd51319)) </details> --- ### Renovate configuration :date: **Schedule**: At any time (no schedule defined). :vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied. :recycle: **Rebasing**: Never, or you tick the rebase/retry checkbox. :no_bell: **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples). * Simplify noxfile setup. [(#2806)](#2806) * chore(deps): update dependency requests to v2.23.0 * Simplify noxfile and add version control. * Configure appengine/standard to only test Python 2.7. * Update Kokokro configs to match noxfile. * Add requirements-test to each folder. * Remove Py2 versions from everything execept appengine/standard. * Remove conftest.py. * Remove appengine/standard/conftest.py * Remove 'no-sucess-flaky-report' from pytest.ini. * Add GAE SDK back to appengine/standard tests. * Fix typo. * Roll pytest to python 2 version. * Add a bunch of testing requirements. * Remove typo. * Add appengine lib directory back in. * Add some additional requirements. * Fix issue with flake8 args. * Even more requirements. * Readd appengine conftest.py. * Add a few more requirements. * Even more Appengine requirements. * Add webtest for appengine/standard/mailgun. * Add some additional requirements. * Add workaround for issue with mailjet-rest. * Add responses for appengine/standard/mailjet. Co-authored-by: Renovate Bot <bot@renovateapp.com> * testing: mark some tests as flaky [(#3288)](#3288) fixes #3138 * [datalabeling] testing: wrap rpcs with backoff [(#3443)](#3443) * wrap all the rpcs with backoff * add a shared testing lib * remove flaky * [datalabeling] fix: clean up old datasets before the test [(#3707)](#3707) fixes #3710 fixes #3711 * [datalabeling] testing: retry upon ServerError [(#3762)](#3762) fixes #3760 * Replace GCLOUD_PROJECT with GOOGLE_CLOUD_PROJECT. [(#4022)](#4022) * chore(deps): update dependency pytest to v5.4.3 [(#4279)](#4279) * chore(deps): update dependency pytest to v5.4.3 * specify pytest for python 2 in appengine Co-authored-by: Leah Cole <coleleah@google.com> * Update dependency pytest to v6 [(#4390)](#4390) * chore: update templates * chore: fix docs error * chore: skip unavailable samples * chore: use staging endpoint for labeling tests Co-authored-by: Rebecca Taylor <remilytaylor@gmail.com> Co-authored-by: Gus Class <gguuss@gmail.com> Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com> Co-authored-by: DPEBot <dpebot@google.com> Co-authored-by: Doug Mahugh <dmahugh@gmail.com> Co-authored-by: Noah Negrey <nnegrey@users.noreply.github.com> Co-authored-by: Leah E. Cole <6719667+leahecole@users.noreply.github.com> Co-authored-by: WhiteSource Renovate <bot@renovateapp.com> Co-authored-by: Takashi Matsuo <tmatsuo@google.com> Co-authored-by: Leah Cole <coleleah@google.com> Co-authored-by: Bu Sun Kim <busunkim@google.com>
GoogleCloudPlatform · Nov 4, 2022 · 7858a28 · 7858a28
1 parent 912ccc3
commit 7858a28
Show file tree

Hide file tree

Showing 24 changed files with 1,835 additions and 0 deletions.
diff --git a/datalabeling/AUTHORING_GUIDE.md b/datalabeling/AUTHORING_GUIDE.md
@@ -0,0 +1 @@
+See https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/AUTHORING_GUIDE.md
diff --git a/datalabeling/CONTRIBUTING.md b/datalabeling/CONTRIBUTING.md
@@ -0,0 +1 @@
+See https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/CONTRIBUTING.md
diff --git a/datalabeling/snippets/README.rst b/datalabeling/snippets/README.rst
@@ -0,0 +1,78 @@
+.. This file is automatically generated. Do not edit this file directly.
+
+Google Cloud Data Labeling Service Python Samples
+===============================================================================
+
+.. image:: https://gstatic.com/cloudssh/images/open-btn.png
+   :target: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=datalabeling/README.rst
+
+
+This directory contains samples for Google Cloud Data Labeling Service. `Google Cloud Data Labeling Service`_ allows developers to request having human labelers label a collection of data that you plan to use to train a custom machine learning model.
+
+
+
+
+.. _Google Cloud Data Labeling Service: https://cloud.google.com/data-labeling/docs/
+
+Setup
+-------------------------------------------------------------------------------
+
+
+Authentication
+++++++++++++++
+
+This sample requires you to have authentication setup. Refer to the
+`Authentication Getting Started Guide`_ for instructions on setting up
+credentials for applications.
+
+.. _Authentication Getting Started Guide:
+    https://cloud.google.com/docs/authentication/getting-started
+
+Install Dependencies
+++++++++++++++++++++
+
+#. Clone python-docs-samples and change directory to the sample directory you want to use.
+
+    .. code-block:: bash
+
+        $ git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
+
+#. Install `pip`_ and `virtualenv`_ if you do not already have them. You may want to refer to the `Python Development Environment Setup Guide`_ for Google Cloud Platform for instructions.
+
+   .. _Python Development Environment Setup Guide:
+       https://cloud.google.com/python/setup
+
+#. Create a virtualenv. Samples are compatible with Python 2.7 and 3.4+.
+
+    .. code-block:: bash
+
+        $ virtualenv env
+        $ source env/bin/activate
+
+#. Install the dependencies needed to run the samples.
+
+    .. code-block:: bash
+
+        $ pip install -r requirements.txt
+
+.. _pip: https://pip.pypa.io/
+.. _virtualenv: https://virtualenv.pypa.io/
+
+
+
+The client library
+-------------------------------------------------------------------------------
+
+This sample uses the `Google Cloud Client Library for Python`_.
+You can read the documentation for more details on API usage and use GitHub
+to `browse the source`_ and  `report issues`_.
+
+.. _Google Cloud Client Library for Python:
+    https://googlecloudplatform.github.io/google-cloud-python/
+.. _browse the source:
+    https://github.com/GoogleCloudPlatform/google-cloud-python
+.. _report issues:
+    https://github.com/GoogleCloudPlatform/google-cloud-python/issues
+
+
+.. _Google Cloud SDK: https://cloud.google.com/sdk/
diff --git a/datalabeling/snippets/README.rst.in b/datalabeling/snippets/README.rst.in
@@ -0,0 +1,18 @@
+# This file is used to generate README.rst
+
+product:
+  name: Google Cloud Data Labeling Service
+  short_name: Cloud Data Labeling
+  url: https://cloud.google.com/data-labeling/docs/
+  description: >
+    `Google Cloud Data Labeling Service`_ allows developers to request having
+    human labelers label a collection of data that you plan to use to train a
+    custom machine learning model.
+
+setup:
+- auth
+- install_deps
+
+cloud_client_library: true
+
+folder: datalabeling
diff --git a/datalabeling/snippets/create_annotation_spec_set.py b/datalabeling/snippets/create_annotation_spec_set.py
@@ -0,0 +1,87 @@
+#!/usr/bin/env python
+
+# Copyright 2019 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import os
+
+from google.api_core.client_options import ClientOptions
+
+
+# [START datalabeling_create_annotation_spec_set_beta]
+def create_annotation_spec_set(project_id):
+    """Creates a data labeling annotation spec set for the given
+    Google Cloud project.
+    """
+    from google.cloud import datalabeling_v1beta1 as datalabeling
+    client = datalabeling.DataLabelingServiceClient()
+    # [END datalabeling_create_annotation_spec_set_beta]
+    # If provided, use a provided test endpoint - this will prevent tests on
+    # this snippet from triggering any action by a real human
+    if 'DATALABELING_ENDPOINT' in os.environ:
+        opts = ClientOptions(api_endpoint=os.getenv('DATALABELING_ENDPOINT'))
+        client = datalabeling.DataLabelingServiceClient(client_options=opts)
+    # [START datalabeling_create_annotation_spec_set_beta]
+
+    project_path = client.project_path(project_id)
+
+    annotation_spec_1 = datalabeling.types.AnnotationSpec(
+        display_name='label_1',
+        description='label_description_1'
+    )
+
+    annotation_spec_2 = datalabeling.types.AnnotationSpec(
+        display_name='label_2',
+        description='label_description_2'
+    )
+
+    annotation_spec_set = datalabeling.types.AnnotationSpecSet(
+        display_name='YOUR_ANNOTATION_SPEC_SET_DISPLAY_NAME',
+        description='YOUR_DESCRIPTION',
+        annotation_specs=[annotation_spec_1, annotation_spec_2]
+    )
+
+    response = client.create_annotation_spec_set(
+        project_path, annotation_spec_set)
+
+    # The format of the resource name:
+    # project_id/{project_id}/annotationSpecSets/{annotationSpecSets_id}
+    print('The annotation_spec_set resource name: {}'.format(response.name))
+    print('Display name: {}'.format(response.display_name))
+    print('Description: {}'.format(response.description))
+    print('Annotation specs:')
+    for annotation_spec in response.annotation_specs:
+        print('\tDisplay name: {}'.format(annotation_spec.display_name))
+        print('\tDescription: {}\n'.format(annotation_spec.description))
+
+    return response
+# [END datalabeling_create_annotation_spec_set_beta]
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(
+        description=__doc__,
+        formatter_class=argparse.RawDescriptionHelpFormatter
+    )
+
+    parser.add_argument(
+        '--project-id',
+        help='Project ID. Required.',
+        required=True
+    )
+
+    args = parser.parse_args()
+
+    create_annotation_spec_set(args.project_id)
diff --git a/datalabeling/snippets/create_annotation_spec_set_test.py b/datalabeling/snippets/create_annotation_spec_set_test.py
@@ -0,0 +1,53 @@
+#!/usr/bin/env python
+
+# Copyright 2019 Google, Inc
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import os
+
+import backoff
+from google.api_core.exceptions import ServerError
+import pytest
+
+import create_annotation_spec_set
+import testing_lib
+
+
+PROJECT_ID = os.getenv('GOOGLE_CLOUD_PROJECT')
+
+
+@pytest.fixture(scope='module')
+def cleaner():
+    resource_names = []
+
+    yield resource_names
+
+    for resource_name in resource_names:
+        testing_lib.delete_annotation_spec_set(resource_name)
+
+
+def test_create_annotation_spec_set(cleaner, capsys):
+
+    @backoff.on_exception(
+        backoff.expo, ServerError, max_time=testing_lib.RETRY_DEADLINE)
+    def run_sample():
+        return create_annotation_spec_set.create_annotation_spec_set(PROJECT_ID)
+
+    response = run_sample()
+
+    # For cleanup
+    cleaner.append(response.name)
+
+    out, _ = capsys.readouterr()
+    assert 'The annotation_spec_set resource name:' in out
diff --git a/datalabeling/snippets/create_instruction.py b/datalabeling/snippets/create_instruction.py
@@ -0,0 +1,103 @@
+#!/usr/bin/env python
+
+# Copyright 2019 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import argparse
+import os
+
+from google.api_core.client_options import ClientOptions
+
+
+# [START datalabeling_create_instruction_beta]
+def create_instruction(project_id, data_type, instruction_gcs_uri):
+    """ Creates a data labeling PDF instruction for the given Google Cloud
+    project. The PDF file should be uploaded to the project in
+    Google Cloud Storage.
+    """
+    from google.cloud import datalabeling_v1beta1 as datalabeling
+    client = datalabeling.DataLabelingServiceClient()
+    # [END datalabeling_create_instruction_beta]
+    # If provided, use a provided test endpoint - this will prevent tests on
+    # this snippet from triggering any action by a real human
+    if 'DATALABELING_ENDPOINT' in os.environ:
+        opts = ClientOptions(api_endpoint=os.getenv('DATALABELING_ENDPOINT'))
+        client = datalabeling.DataLabelingServiceClient(client_options=opts)
+    # [START datalabeling_create_instruction_beta]
+
+    project_path = client.project_path(project_id)
+
+    pdf_instruction = datalabeling.types.PdfInstruction(
+        gcs_file_uri=instruction_gcs_uri)
+
+    instruction = datalabeling.types.Instruction(
+        display_name='YOUR_INSTRUCTION_DISPLAY_NAME',
+        description='YOUR_DESCRIPTION',
+        data_type=data_type,
+        pdf_instruction=pdf_instruction
+    )
+
+    operation = client.create_instruction(project_path, instruction)
+
+    result = operation.result()
+
+    # The format of the resource name:
+    # project_id/{project_id}/instruction/{instruction_id}
+    print('The instruction resource name: {}'.format(result.name))
+    print('Display name: {}'.format(result.display_name))
+    print('Description: {}'.format(result.description))
+    print('Create time:')
+    print('\tseconds: {}'.format(result.create_time.seconds))
+    print('\tnanos: {}'.format(result.create_time.nanos))
+    print('Data type: {}'.format(
+        datalabeling.enums.DataType(result.data_type).name))
+    print('Pdf instruction:')
+    print('\tGcs file uri: {}\n'.format(
+        result.pdf_instruction.gcs_file_uri))
+
+    return result
+# [END datalabeling_create_instruction_beta]
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(
+        description=__doc__,
+        formatter_class=argparse.RawDescriptionHelpFormatter
+    )
+
+    parser.add_argument(
+        '--project-id',
+        help='Project ID. Required.',
+        required=True
+    )
+
+    parser.add_argument(
+        '--data-type',
+        help='Data type. Only support IMAGE, VIDEO, TEXT and AUDIO. Required.',
+        required=True
+    )
+
+    parser.add_argument(
+        '--instruction-gcs-uri',
+        help='The URI of Google Cloud Storage of the instruction. Required.',
+        required=True
+    )
+
+    args = parser.parse_args()
+
+    create_instruction(
+        args.project_id,
+        args.data_type,
+        args.instruction_gcs_uri
+    )