refactor: kind aware codegen tools

[acpana]

I've reworked the 3 cmds (generate-types, generate-mapper and add) to be kind aware. The new structure for apis:

bigqueryanalyticshub
├─v1alpha1
└─v1beta1
   ├─ dataexchangelisting _types.go // has Spec and Status
   ├─ dataexchangelisting.types.generated.go // has generated go structs
   ├─ dataexchange_types.go
   ├─ shared_types.go // not generated today, but will be in the future
   │--doc.go
   └─group_version.go

and for the controller:

bigqueryanalyticshub
├─dataexchange
|─dataexchangelisting
|   ├─dataexchangelisting_controller.go
|   ├─dataexchangelisting_externalresource.go
|   └─dataexchangelisting_mapper.generated.go
└ mapper.generated.go

NOTES:

• the mapper.generated.go is a catch all but ideally all relevant mappings should be in each file; I will follow on about this but for now the structure is more important
• the shared_types.go is more to share the direction that we are going in; the tooling doesn't support this yet but I/ we can explore that out of band.

Reviewers:

• PR is best viewed by commits

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from acpana. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[acpana]

fyi/ cc y'all

@jingyih / @jasonvigil / @justinsb

[jingyih]

I was under the impression we'd have a single 'types.generated.go' shared across multiple resources. Did I get that wrong?

[acpana]

types.generated.go will get clobbered by the most recent type IIUC. The intent here is to keep these separate and the cmd user can select how they deal with this file

[jasonvigil]

types.generated.go will get clobbered by the most recent type IIUC

To fix this issue, can we modify the flags to pass in a list of protos? That way, we can generate all the types for a set of protos in the same API group, instead of just one at a time.

[acpana]

That way, we can generate all the types for a set of protos in the same API group

Or maybe we can pass in just the API group. The issue w a list of protos is what happens, say, 6 months after we added resources A and B. Someone wants to add resource C, they then have to recall to add both resources that came before to their generation cmd which is a bit of a paper cut imo.

[jasonvigil]

I thought that was the purpose of https://github.com/GoogleCloudPlatform/k8s-config-connector/blob/master/dev/tools/controllerbuilder/generate.sh ? To record what had been run before. And, eventually, we would run it regularly to catch field updates / changes.

[jingyih]

I would prefer that we avoid prepending the resource kind to the auto-generated types and mappings. In my view, it isn't necessary, and it could also prevent the command from being rerun on existing SciFi resources.

I think ideally the tool should generate the following:

• Option 1

apis/bigqueryanalyticshub
├─v1alpha1
└─v1beta1
   ├─dataexchangelisting_types.go // spec and status of dataexchangelisting
   ├─dataexchange_types.go // spec and status of dataexchange
   ├─types.generated.go // generated go structs for dataexchange and dataexchangelisting (they may share some go structs)
   ├─doc.go
   └─group_version.go

pkg/controller/direct/bigqueryanalyticshub
|─dataexchange
|   ├─dataexchange_controller.go
|   ├─dataexchange_externalresource.go
|   └─dataexchange_mapping.go // this file is not generated, it includes manually modified mapper functions for dataexchange
|─dataexchangelisting
|   ├─dataexchangelisting_controller.go
|   ├─dataexchangelisting_externalresource.go
|   └─dataexchangelisting_mapping.go // this file is not generated, it includes manually modified mapper functions for dataexchangelisting
└ mapper.generated.go // generated mapper functions of dataexchange and dataexchangelisting (they may share some mapper functions)

• Option 2

apis/bigqueryanalyticshub
├─v1alpha1
└─v1beta1
   ├─dataexchangelisting_types.go // spec and status of dataexchangelisting
   ├─dataexchange_types.go // spec and status of dataexchange
   ├─types.generated.go // has generated go structs for dataexchange and dataexchangelisting (they may share some go structs)
   ├─doc.go
   └─group_version.go

pkg/controller/direct/bigqueryanalyticshub
├─dataexchangelisting_controller.go
├─dataexchangelisting_externalresource.go
├─dataexchangelisting_mapping.go // this file is not generated, it includes manually modified mapper functions for dataexchangelisting
├─dataexchange_controller.go
├─dataexchange_externalresource.go
├─dataexchange_mapping.go // this file is not generated, it includes manually modified mapper functions for dataexchange
└─mapper.generated.go // generated mapper functions of dataexchange and dataexchangelisting (they may share some mapper functions)