chore: move samples from python-docs-sample (#18)

* Add Error Reporting Code * Updating doc links [(#324)](#324) * Commenting noxfile, renaming a test file. Change-Id: Ice25caa7c035c24a585a66575dda69e170862df8 * Update requirements. [(#436)](#436) * Fix import order lint errors Change-Id: Ieaf7237fc6f925daec46a07d2e81a452b841198a * bump Change-Id: I02e7767d13ba267ee9fc72c5b68a57013bb8b8d3 * Auto-update dependencies. [(#537)](#537) * Update samples to support latest Google Cloud Python [(#656)](#656) * Switch Error Reporting to Google Cloud Python [(#663)](#663) * Switch Error Reporting to Google Cloud Python * Switch Error Reporting to Google Cloud Python * Auto-update dependencies. [(#715)](#715) * Auto-update dependencies. [(#914)](#914) * Auto-update dependencies. * xfail the error reporting test * Fix lint * Re-generate all readmes * Auto-update dependencies. [(#922)](#922) * Auto-update dependencies. * Fix pubsub iam samples * Fix README rst links [(#962)](#962) * Fix README rst links * Update all READMEs * Auto-update dependencies. [(#1004)](#1004) * Auto-update dependencies. * Fix natural language samples * Fix pubsub iam samples * Fix language samples * Fix bigquery samples * Auto-update dependencies. [(#1011)](#1011) * Auto-update dependencies. [(#1055)](#1055) * Auto-update dependencies. * Explicitly use latest bigtable client Change-Id: Id71e9e768f020730e4ca9514a0d7ebaa794e7d9e * Revert language update for now Change-Id: I8867f154e9a5aae00d0047c9caf880e5e8f50c53 * Remove pdb. smh Change-Id: I5ff905fadc026eebbcd45512d4e76e003e3b2b43 * Auto-update dependencies. [(#1093)](#1093) * Auto-update dependencies. * Fix storage notification poll sample Change-Id: I6afbc79d15e050531555e4c8e51066996717a0f3 * Fix spanner samples Change-Id: I40069222c60d57e8f3d3878167591af9130895cb * Drop coverage because it's not useful Change-Id: Iae399a7083d7866c3c7b9162d0de244fbff8b522 * Try again to fix flaky logging test Change-Id: I6225c074701970c17c426677ef1935bb6d7e36b4 * Update all generated readme auth instructions [(#1121)](#1121) Change-Id: I03b5eaef8b17ac3dc3c0339fd2c7447bd3e11bd2 * Added Link to Python Setup Guide [(#1158)](#1158) * Update Readme.rst to add Python setup guide As requested in b/64770713. This sample is linked in documentation https://cloud.google.com/bigtable/docs/scaling, and it would make more sense to update the guide here than in the documentation. * Update README.rst * Update README.rst * Update README.rst * Update README.rst * Update README.rst * Update install_deps.tmpl.rst * Updated readmegen scripts and re-generated related README files * Fixed the lint error * Auto-update dependencies. [(#1186)](#1186) * Add error reporting sample for manual reporting * Readd fluentd samples * rename dir * Auto-update dependencies. [(#1205)](#1205) * Auto-update dependencies. [(#1215)](#1215) * Auto-update dependencies. [(#1245)](#1245) * Auto-update dependencies. [(#1248)](#1248) * Added "Open in Cloud Shell" buttons to README files [(#1254)](#1254) * Auto-update dependencies. [(#1276)](#1276) * Auto-update dependencies. [(#1282)](#1282) * Auto-update dependencies. * Fix storage acl sample Change-Id: I413bea899fdde4c4859e4070a9da25845b81f7cf * Auto-update dependencies. [(#1317)](#1317) * Auto-update dependencies. [(#1320)](#1320) * Auto-update dependencies. [(#1359)](#1359) * Auto-update dependencies. [(#1377)](#1377) * Auto-update dependencies. * Update requirements.txt * Regenerate the README files and fix the Open in Cloud Shell link for some samples [(#1441)](#1441) * Update READMEs to fix numbering and add git clone [(#1464)](#1464) * Auto-update dependencies. [(#1658)](#1658) * Auto-update dependencies. * Rollback appengine/standard/bigquery/. * Rollback appengine/standard/iap/. * Rollback bigtable/metricscaler. * Rolledback appengine/flexible/datastore. * Rollback dataproc/ * Rollback jobs/api_client * Rollback vision/cloud-client. * Rollback functions/ocr/app. * Rollback iot/api-client/end_to_end_example. * Rollback storage/cloud-client. * Rollback kms/api-client. * Rollback dlp/ * Rollback bigquery/cloud-client. * Rollback iot/api-client/manager. * Rollback appengine/flexible/cloudsql_postgresql. * Put in new region code error_reporting_quickstart [(#1842)](#1842) * New region tags [(#1843)](#1843) * Auto-update dependencies. [(#1980)](#1980) * Auto-update dependencies. * Update requirements.txt * Update requirements.txt * Adds updates including compute [(#2436)](#2436) * Adds updates including compute * Python 2 compat pytest * Fixing weird \r\n issue from GH merge * Put asset tests back in * Re-add pod operator test * Hack parameter for k8s pod operator * Auto-update dependencies. [(#2005)](#2005) * Auto-update dependencies. * Revert update of appengine/flexible/datastore. * revert update of appengine/flexible/scipy * revert update of bigquery/bqml * revert update of bigquery/cloud-client * revert update of bigquery/datalab-migration * revert update of bigtable/quickstart * revert update of compute/api * revert update of container_registry/container_analysis * revert update of dataflow/run_template * revert update of datastore/cloud-ndb * revert update of dialogflow/cloud-client * revert update of dlp * revert update of functions/imagemagick * revert update of functions/ocr/app * revert update of healthcare/api-client/fhir * revert update of iam/api-client * revert update of iot/api-client/gcs_file_to_device * revert update of iot/api-client/mqtt_example * revert update of language/automl * revert update of run/image-processing * revert update of vision/automl * revert update testing/requirements.txt * revert update of vision/cloud-client/detect * revert update of vision/cloud-client/product_search * revert update of jobs/v2/api_client * revert update of jobs/v3/api_client * revert update of opencensus * revert update of translate/cloud-client * revert update to speech/cloud-client Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com> Co-authored-by: Doug Mahugh <dmahugh@gmail.com> * chore(deps): update dependency fluent-logger to v0.9.5 [(#3004)](#3004) * chore(deps): update dependency fluent-logger to v0.9.6 [(#3080)](#3080) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [fluent-logger](https://togithub.com/fluent/fluent-logger-python) | patch | `==0.9.5` -> `==0.9.6` | --- ### Release Notes <details> <summary>fluent/fluent-logger-python</summary> ### [`v0.9.6`](https://togithub.com/fluent/fluent-logger-python/compare/v0.9.5...v0.9.6) [Compare Source](https://togithub.com/fluent/fluent-logger-python/compare/v0.9.5...v0.9.6) </details> --- ### Renovate configuration :date: **Schedule**: At any time (no schedule defined). :vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied. :recycle: **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. :no_bell: **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#GoogleCloudPlatform/python-docs-samples). * Simplify noxfile setup. [(#2806)](#2806) * chore(deps): update dependency requests to v2.23.0 * Simplify noxfile and add version control. * Configure appengine/standard to only test Python 2.7. * Update Kokokro configs to match noxfile. * Add requirements-test to each folder. * Remove Py2 versions from everything execept appengine/standard. * Remove conftest.py. * Remove appengine/standard/conftest.py * Remove 'no-sucess-flaky-report' from pytest.ini. * Add GAE SDK back to appengine/standard tests. * Fix typo. * Roll pytest to python 2 version. * Add a bunch of testing requirements. * Remove typo. * Add appengine lib directory back in. * Add some additional requirements. * Fix issue with flake8 args. * Even more requirements. * Readd appengine conftest.py. * Add a few more requirements. * Even more Appengine requirements. * Add webtest for appengine/standard/mailgun. * Add some additional requirements. * Add workaround for issue with mailjet-rest. * Add responses for appengine/standard/mailjet. Co-authored-by: Renovate Bot <bot@renovateapp.com> * chore(deps): update dependency google-cloud-error-reporting to v0.34.0 [(#3782)](#3782) * chore(deps): update dependency pytest to v5.4.3 [(#4279)](#4279) * chore(deps): update dependency pytest to v5.4.3 * specify pytest for python 2 in appengine Co-authored-by: Leah Cole <coleleah@google.com> * chore(deps): update dependency mock to v4 [(#4287)](#4287) * chore(deps): update dependency mock to v4 * specify mock version for appengine python 2 Co-authored-by: Leah Cole <coleleah@google.com> * Update dependency pytest to v6 [(#4390)](#4390) * chore: update templates Co-authored-by: Bill Prin <waprin@google.com> Co-authored-by: Jon Wayne Parrott <jonwayne@google.com> Co-authored-by: DPE bot <dpebot@google.com> Co-authored-by: Bill Prin <waprin@gmail.com> Co-authored-by: michaelawyu <chenyumic@google.com> Co-authored-by: Andrew Gorcester <gorcester@google.com> Co-authored-by: Andrew Gorcester <andrew.gorcester@gmail.com> Co-authored-by: Frank Natividad <frankyn@users.noreply.github.com> Co-authored-by: Charles Engelke <github@engelke.com> Co-authored-by: Gus Class <gguuss@gmail.com> Co-authored-by: Kurtis Van Gent <31518063+kurtisvg@users.noreply.github.com> Co-authored-by: Doug Mahugh <dmahugh@gmail.com> Co-authored-by: WhiteSource Renovate <bot@renovateapp.com> Co-authored-by: Leah Cole <coleleah@google.com>
GoogleCloudPlatform · Aug 13, 2020 · 5169bd4 · 5169bd4
commit 5169bd4
Show file tree

Hide file tree

Showing 7 changed files with 362 additions and 0 deletions.
diff --git a/error_reporting/fluent_on_compute/README.md b/error_reporting/fluent_on_compute/README.md
@@ -0,0 +1,35 @@
+# Google Error Reorting Samples Samples
+
+[![Open in Cloud Shell][shell_img]][shell_link]
+
+[shell_img]: http://gstatic.com/cloudssh/images/open-btn.png
+[shell_link]: https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/GoogleCloudPlatform/python-docs-samples&page=editor&open_in_editor=error_reporting/fluent_on_compute/README.md
+
+This section contains samples for [Google Cloud Error Reporting](https://cloud.google.com/error-reporting).
+
+A startup script has been provided to demonstrated how to properly provision a GCE 
+instance with fluentd configured. Note the intallation of fluentd, the addition of the config file,
+ and the restarting of the fluetnd service. You can start an instance  using
+it like this:
+
+    gcloud compute instances create example-instance --metadata-from-file startup-script=startup_script.sh
+
+or simply use it as reference when creating your own instance.
+
+After fluentd is configured, main.py could be used to simulate an error:
+
+    gcloud compute copy-files main.py example-instance:~/main.py
+
+Then, 
+
+    gcloud compute ssh example-instance
+    python ~/main.py
+
+And you will see the message in the Errors Console.
+
+<!-- auto-doc-link -->
+These samples are used on the following documentation page:
+
+> https://cloud.google.com/error-reporting/docs/setting-up-on-compute-engine
+
+<!-- end-auto-doc-link -->
diff --git a/error_reporting/fluent_on_compute/main.py b/error_reporting/fluent_on_compute/main.py
@@ -0,0 +1,42 @@
+# Copyright 2016 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# [START error_reporting]
+import traceback
+
+import fluent.event
+import fluent.sender
+
+
+def simulate_error():
+    fluent.sender.setup('myapp', host='localhost', port=24224)
+
+    def report(ex):
+        data = {}
+        data['message'] = '{0}'.format(ex)
+        data['serviceContext'] = {'service': 'myapp'}
+        # ... add more metadata
+        fluent.event.Event('errors', data)
+
+    # report exception data using:
+    try:
+        # simulate calling a method that's not defined
+        raise NameError
+    except Exception:
+        report(traceback.format_exc())
+# [END error_reporting]
+
+
+if __name__ == '__main__':
+    simulate_error()
diff --git a/error_reporting/fluent_on_compute/main_test.py b/error_reporting/fluent_on_compute/main_test.py
@@ -0,0 +1,23 @@
+# Copyright 2016 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import mock
+
+import main
+
+
+@mock.patch("fluent.event")
+def test_error_sends(event_mock):
+    main.simulate_error()
+    event_mock.Event.assert_called_once_with(mock.ANY, mock.ANY)
diff --git a/error_reporting/fluent_on_compute/noxfile.py b/error_reporting/fluent_on_compute/noxfile.py
@@ -0,0 +1,224 @@
+# Copyright 2019 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from __future__ import print_function
+
+import os
+from pathlib import Path
+import sys
+
+import nox
+
+
+# WARNING - WARNING - WARNING - WARNING - WARNING
+# WARNING - WARNING - WARNING - WARNING - WARNING
+#           DO NOT EDIT THIS FILE EVER!
+# WARNING - WARNING - WARNING - WARNING - WARNING
+# WARNING - WARNING - WARNING - WARNING - WARNING
+
+# Copy `noxfile_config.py` to your directory and modify it instead.
+
+
+# `TEST_CONFIG` dict is a configuration hook that allows users to
+# modify the test configurations. The values here should be in sync
+# with `noxfile_config.py`. Users will copy `noxfile_config.py` into
+# their directory and modify it.
+
+TEST_CONFIG = {
+    # You can opt out from the test for specific Python versions.
+    'ignored_versions': ["2.7"],
+
+    # An envvar key for determining the project id to use. Change it
+    # to 'BUILD_SPECIFIC_GCLOUD_PROJECT' if you want to opt in using a
+    # build specific Cloud project. You can also use your own string
+    # to use your own Cloud project.
+    'gcloud_project_env': 'GOOGLE_CLOUD_PROJECT',
+    # 'gcloud_project_env': 'BUILD_SPECIFIC_GCLOUD_PROJECT',
+
+    # A dictionary you want to inject into your test. Don't put any
+    # secrets here. These values will override predefined values.
+    'envs': {},
+}
+
+
+try:
+    # Ensure we can import noxfile_config in the project's directory.
+    sys.path.append('.')
+    from noxfile_config import TEST_CONFIG_OVERRIDE
+except ImportError as e:
+    print("No user noxfile_config found: detail: {}".format(e))
+    TEST_CONFIG_OVERRIDE = {}
+
+# Update the TEST_CONFIG with the user supplied values.
+TEST_CONFIG.update(TEST_CONFIG_OVERRIDE)
+
+
+def get_pytest_env_vars():
+    """Returns a dict for pytest invocation."""
+    ret = {}
+
+    # Override the GCLOUD_PROJECT and the alias.
+    env_key = TEST_CONFIG['gcloud_project_env']
+    # This should error out if not set.
+    ret['GOOGLE_CLOUD_PROJECT'] = os.environ[env_key]
+
+    # Apply user supplied envs.
+    ret.update(TEST_CONFIG['envs'])
+    return ret
+
+
+# DO NOT EDIT - automatically generated.
+# All versions used to tested samples.
+ALL_VERSIONS = ["2.7", "3.6", "3.7", "3.8"]
+
+# Any default versions that should be ignored.
+IGNORED_VERSIONS = TEST_CONFIG['ignored_versions']
+
+TESTED_VERSIONS = sorted([v for v in ALL_VERSIONS if v not in IGNORED_VERSIONS])
+
+INSTALL_LIBRARY_FROM_SOURCE = bool(os.environ.get("INSTALL_LIBRARY_FROM_SOURCE", False))
+#
+# Style Checks
+#
+
+
+def _determine_local_import_names(start_dir):
+    """Determines all import names that should be considered "local".
+
+    This is used when running the linter to insure that import order is
+    properly checked.
+    """
+    file_ext_pairs = [os.path.splitext(path) for path in os.listdir(start_dir)]
+    return [
+        basename
+        for basename, extension in file_ext_pairs
+        if extension == ".py"
+        or os.path.isdir(os.path.join(start_dir, basename))
+        and basename not in ("__pycache__")
+    ]
+
+
+# Linting with flake8.
+#
+# We ignore the following rules:
+#   E203: whitespace before ‘:’
+#   E266: too many leading ‘#’ for block comment
+#   E501: line too long
+#   I202: Additional newline in a section of imports
+#
+# We also need to specify the rules which are ignored by default:
+# ['E226', 'W504', 'E126', 'E123', 'W503', 'E24', 'E704', 'E121']
+FLAKE8_COMMON_ARGS = [
+    "--show-source",
+    "--builtin=gettext",
+    "--max-complexity=20",
+    "--import-order-style=google",
+    "--exclude=.nox,.cache,env,lib,generated_pb2,*_pb2.py,*_pb2_grpc.py",
+    "--ignore=E121,E123,E126,E203,E226,E24,E266,E501,E704,W503,W504,I202",
+    "--max-line-length=88",
+]
+
+
+@nox.session
+def lint(session):
+    session.install("flake8", "flake8-import-order")
+
+    local_names = _determine_local_import_names(".")
+    args = FLAKE8_COMMON_ARGS + [
+        "--application-import-names",
+        ",".join(local_names),
+        "."
+    ]
+    session.run("flake8", *args)
+
+
+#
+# Sample Tests
+#
+
+
+PYTEST_COMMON_ARGS = ["--junitxml=sponge_log.xml"]
+
+
+def _session_tests(session, post_install=None):
+    """Runs py.test for a particular project."""
+    if os.path.exists("requirements.txt"):
+        session.install("-r", "requirements.txt")
+
+    if os.path.exists("requirements-test.txt"):
+        session.install("-r", "requirements-test.txt")
+
+    if INSTALL_LIBRARY_FROM_SOURCE:
+        session.install("-e", _get_repo_root())
+
+    if post_install:
+        post_install(session)
+
+    session.run(
+        "pytest",
+        *(PYTEST_COMMON_ARGS + session.posargs),
+        # Pytest will return 5 when no tests are collected. This can happen
+        # on travis where slow and flaky tests are excluded.
+        # See http://doc.pytest.org/en/latest/_modules/_pytest/main.html
+        success_codes=[0, 5],
+        env=get_pytest_env_vars()
+    )
+
+
+@nox.session(python=ALL_VERSIONS)
+def py(session):
+    """Runs py.test for a sample using the specified version of Python."""
+    if session.python in TESTED_VERSIONS:
+        _session_tests(session)
+    else:
+        session.skip("SKIPPED: {} tests are disabled for this sample.".format(
+            session.python
+        ))
+
+
+#
+# Readmegen
+#
+
+
+def _get_repo_root():
+    """ Returns the root folder of the project. """
+    # Get root of this repository. Assume we don't have directories nested deeper than 10 items.
+    p = Path(os.getcwd())
+    for i in range(10):
+        if p is None:
+            break
+        if Path(p / ".git").exists():
+            return str(p)
+        p = p.parent
+    raise Exception("Unable to detect repository root.")
+
+
+GENERATED_READMES = sorted([x for x in Path(".").rglob("*.rst.in")])
+
+
+@nox.session
+@nox.parametrize("path", GENERATED_READMES)
+def readmegen(session, path):
+    """(Re-)generates the readme for a sample."""
+    session.install("jinja2", "pyyaml")
+    dir_ = os.path.dirname(path)
+
+    if os.path.exists(os.path.join(dir_, "requirements.txt")):
+        session.install("-r", os.path.join(dir_, "requirements.txt"))
+
+    in_file = os.path.join(dir_, "README.rst.in")
+    session.run(
+        "python", _get_repo_root() + "/scripts/readme-gen/readme_gen.py", in_file
+    )
diff --git a/error_reporting/fluent_on_compute/requirements-test.txt b/error_reporting/fluent_on_compute/requirements-test.txt
@@ -0,0 +1,2 @@
+pytest==6.0.1
+mock==4.0.2
diff --git a/error_reporting/fluent_on_compute/requirements.txt b/error_reporting/fluent_on_compute/requirements.txt
@@ -0,0 +1 @@
+fluent-logger==0.9.6
diff --git a/error_reporting/fluent_on_compute/startup_script.sh b/error_reporting/fluent_on_compute/startup_script.sh
@@ -0,0 +1,35 @@
+#!/usr/bin/env bash
+# Copyright 2016 Google Inc. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -v
+
+curl -sSO "https://dl.google.com/cloudagents/install-logging-agent.sh"
+chmod +x install-logging-agent.sh
+./install-logging-agent.sh
+mkdir -p /etc/google-fluentd/config.d/
+cat <<EOF > /etc/google-fluentd/config.d/forward.conf
+<source>
+  type forward
+  port 24224
+</source>
+EOF
+service google-fluentd restart
+
+apt-get update
+apt-get install -yq \
+        git build-essential supervisor python python-dev python-pip libffi-dev \
+            libssl-dev
+pip install fluent-logger
+