Skip to content

Commit

Permalink
Documentation Update (GoogleCloudPlatform#222)
Browse files Browse the repository at this point in the history
* Convert CAI compute properties, support json column, complete asset Schema.

Three new features.

- Some compute engine assets have fields in the CAI export that don't match the
  fields in the API resource object. This is due to differences in the API
  backend which Google hopes to correct in the future. Until that happens this
  change maps those fields to the same name they have in the API document.

- Because BigQuery sanitation might truncate some values, keep a column with the
  json description of the resource in each asset type table. There are also some
  use cases were querying across all resource types is useful, so also includes
  a table called 'resources' which includes every asset type in the export as
  json. Also add the 'NONE' group_by type to support users who don't want the
  asset type tables generated at all and only use the json exports

- Generate a complete asset schema from API discovery documents. Previously the
  asset schema was incrementally build from the exported json. The asset table
  would only have a column if the export contained a value for that property.
  This made it difficult to write queries that were usable across
  projects/organizations. We now generate a complete asset schema from the API
  documentation and will augment it with the values in the export if they every
  include fields not included in the API document.

* Convert CAI compute properties, support json column, complete asset Schema.

Three new features.

- Some compute engine assets have fields in the CAI export that don't match the
  fields in the API resource object. This is due to differences in the API
  backend which Google hopes to correct in the future. Until that happens this
  change maps those fields to the same name they have in the API document.

- Because BigQuery sanitation might truncate some values, keep a column with the
  json description of the resource in each asset type table. There are also some
  use cases were querying across all resource types is useful, so also includes
  a table called 'resources' which includes every asset type in the export as
  json. Also add the 'NONE' group_by type to support users who don't want the
  asset type tables generated at all and only use the json exports

- Generate a complete asset schema from API discovery documents. Previously the
  asset schema was incrementally build from the exported json. The asset table
  would only have a column if the export contained a value for that property.
  This made it difficult to write queries that were usable across
  projects/organizations. We now generate a complete asset schema from the API
  documentation and will augment it with the values in the export if they every
  include fields not included in the API document.

* Documentation updates

* add tempLocation value to runtime environment

* clarify permissions

* revert to default service

* Move Dataflow configuration instructions to README.md

These were in comments in config.yaml, now they are more visible in the
README.md

Also removed a print statement in a unit test.
  • Loading branch information
bmenasha authored and Jacob Ferriero committed Apr 1, 2019
1 parent 4ef559d commit 54c8d0b
Show file tree
Hide file tree
Showing 3 changed files with 168 additions and 117 deletions.
Loading

0 comments on commit 54c8d0b

Please sign in to comment.