Test all links and code examples in docs for GA/Beta.

[daspecster]

Test all doc links and examples for...

• BigQuery (results)
  • Links/references (Update redirects and references for BigQuery docs. #3196)
  • Examples/Snippets
• Datastore 1 anchor reference error, (results)
  • Links/references (Update datastore batch API doc reference. #3186)
  • Examples/Snippets
• Storage
  • Links/references (Update 301 redirects for storage documentation. #3192) (results)
  • Examples/Snippets (generate_upload_policy snippet appears to be broken) (results) (PR: Fix storage encryption example, remove policy document example. #3221)
• Logging (results)
  • Links/references (Logging docs, update references that were 301 redirecting. #3187)
  • Examples/Snippets (Issue(s): Logging seems to not support type.googleapis.com/google.cloud.audit.AuditLog. #3218) (PR: Fix untested logging snippet examples. #3219), (results)
• Natural Language
  • Links/references (Update NL documentation references. #3197)
  • Examples/Snippets (results) (PR: Fix broken language examples. #3220)
• Speech
  • Links/references (Fix broken speech link. #3198)
  • Examples/Snippets (PR: Update speech usage examples. #3206) (results)
• Translation
  • Links/references (Fix translate links. #3199)
  • Examples/Snippets (results)
• Vision
  • Links/references (Fix vision documentation redirects and references. #3202)
  • Examples/Snippets (results) PR: Small updates to Vision examples. #3227

/cc @lukesneeringer @omaray

[dhermes]

This is in some sense the grandchild of #212. I vote for the approach used in #2738.

[daspecster]

Moving discussion from #3206 (comment) to this thread.

@lukesneeringer so here's the current status of snippets for GA.

• Datastore snippet tests work/pass and are the most current architecture.
• BigQuery snippet tests work/pass but they are the old architecture and need to be updated.
• Storage snippet tests do not work and are the old architecture.
• Logging snippet tests do not pass and are the old architecture.

The Right Thing™ to do is probably update the BigQuery and Storage snippets to the same methodology that Datastore is using. I think that can be done in reasonable time.
Logging however, is pretty huge and may take a fair bit of work.

I thought this was already a known, and that's why manual testing was the preferred route for now.

[dhermes]

@daspecster FYI: https://github.com/GoogleCloudPlatform/google-cloud-python/blob/1bcff68a53435301e1f309d634420703ec8c2464/docs/logging_snippets.py

[dhermes]

that's why manual testing was the preferred route for now.

There is an inflection point where doing things "by hand" starts to become longer than doing it the right way

[daspecster]

@dhermes logging_snippets.py looks fairly outdated.

╰─$ python docs/logging_snippets.py
instantiate_client       : Instantiate client.
  ERROR: ValueError('This library only supports credentials from google-auth-library-python. See https://google-cloud-python.readthedocs.io/en/latest/google-cloud-auth.html for help on authentication with this library.',)
client_list_entries      : List entries via client.
  ERROR: TypeError('Can not find message descriptor by type_url: type.googleapis.com/google.cloud.audit.AuditLog.',)
logger_usage             : Logger usage.
metric_crud              : Metric CRUD.
sink_storage             : Sink log entries to storage.
sink_bigquery            : Sink log entries to bigquery.
sink_pubsub              : Sink log entries to pubsub.
logging_handler          : None
  ERROR: TypeError('logging_handler() takes exactly 1 argument (2 given)',)
setup_logging            : None
  ERROR: TypeError('setup_logging() takes exactly 1 argument (2 given)',)

[dhermes]

I am unaware of how up to date it is, just posted as a response to "Logging snippet tests do not exists."

[daspecster]

@dhermes yeah thanks! I updated the status in the list.

[daspecster]

@dhermes the Storage snippet for generate_upload_policy seems to be broken. I looked at the docs but I don't see where the breakdown is.

Here's the output when trying to upload with the generated HTML form.

<Error>
<Code>InvalidArgument</Code>
<Message>Invalid argument.</Message>
<Details>Cannot create buckets using a POST.</Details>
</Error>

Any ideas?

[dhermes]

@jonparrott Wrote the snippet.

[theacodes]

It's probably fine, it's just that the bucket has to exist.

[daspecster]

@jonparrott in my test, I was/am using a bucket that exists.

[theacodes]

Weird. That error message indicates otherwise!

[daspecster]

@jonparrott haha I know! 😕

[theacodes]

Can you try manually verifying it?

[daspecster]

@jonparrott yeah that's how I'm getting the error. I'm not just running the snippets. The storage snippets need some updating to work.

bucket = client.bucket('my-bucket24356uytrew')
conditions = [
    ['starts-with', '$key', ''],
    {'acl': 'public-read'}]

policy = bucket.generate_upload_policy(conditions)

# Generate an upload form using the form fields.
policy_fields = ''.join(
    '<input type="hidden" name="{key}" value="{value}">'.format(
        key=key, value=value)
    for key, value in policy.items()
)

upload_form = (
    '<form action="http://{bucket_name}.storage.googleapis.com"'
    '   method="post" enctype="multipart/form-data">'
    '<input type="text" name="key" value="">'
    '<input type="hidden" name="bucket" value="{bucket_name}">'
    '<input type="hidden" name="acl" value="public-read">'
    '<input name="file" type="file">'
    '<input type="submit" value="Upload">'
    '{policy_fields}'
    '<form>').format(bucket_name=bucket.name, policy_fields=policy_fields)

print(upload_form)

[theacodes]

Hmm. I'm at a a bit of a loss.

[daspecster]

I was able to get the logging snippets running somewhat.

instantiate_client       : Instantiate client.
  ERROR: ValueError('This library only supports credentials from google-auth-library-python. See https://google-cloud-python.readthedocs.io/en/latest/google-cloud-auth.html for help on authentication with this library.',)
client_list_entries      : List entries via client.
  ERROR: TypeError('Can not find message descriptor by type_url: type.googleapis.com/google.cloud.audit.AuditLog.',)
logger_usage             : Logger usage.
metric_crud              : Metric CRUD.
sink_storage             : Sink log entries to storage.
sink_bigquery            : Sink log entries to bigquery.
sink_pubsub              : Sink log entries to pubsub.
logging_handler          : None
setup_logging            : None

[theacodes]

@daspecster can you file a separate bug about the storage policy document stuff and assign it to me? Thanks!

[daspecster]

@jonparrott sure thing!

#3222

[daspecster]

Ok, just a few PR's to merge and I think we're pretty much there.

• Fix untested logging snippet examples. #3219
• Fix broken language examples. #3220
• Update speech usage examples. #3206
• Small updates to Vision examples. #3227

[daspecster]

Ok all the PRs are merged.

I'm not saying the docs are perfect but they should be a lot closer.

I'm going to close this issue since all the outlined tasks are completed.

[lukesneeringer]

🎉, thanks.