Promoting Managed Kafka Cluster and Topic resources to GA

[jessdejong]

https://b.corp.google.com/issues/371049718

Release Note Template for Downstream PRs (will be copied)

See Write release notes for guidance.

`google_managed_kafka_cluster` and `google_managed_kafka_topic` (ga)

[github-actions]

Hello! I am a robot. Tests will require approval from a repository maintainer to run.

@SarahFrench, a repository maintainer, has been assigned to review your changes. If you have not received review feedback within 2 business days, please leave a comment on this PR asking them to take a look.

You can help make sure that review is quick by doing a self-review and by running impacted tests locally.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 20 files changed, 2264 insertions(+), 27 deletions(-))
google-beta provider: Diff ( 6 files changed, 5 insertions(+), 63 deletions(-))
Open in Cloud Shell: Diff ( 12 files changed, 399 insertions(+))

[modular-magician]

Tests analytics

Total tests: 5
Passed tests: 5
Skipped tests: 0
Affected tests: 0

Click here to see the affected service packages

• managedkafka

#### Non-exercised tests

🔴 Tests were added that are GA-only additions and require manual runs:

• TestAccManagedKafkaCluster_managedkafkaClusterBasicExample
• TestAccManagedKafkaCluster_managedkafkaClusterCmekExample
• TestAccManagedKafkaCluster_update
• TestAccManagedKafkaTopic_managedkafkaTopicBasicExample
• TestAccManagedKafkaTopic_update
  🟢 All tests passed!

View the build log

[jessdejong]

Hi! Our GA is technically November 12th. Is it possible to submit this before that date? Our beta and GA endpoints are the same.

[SarahFrench]

Review note: GA testing here

---

Hi! Our GA is technically November 12th. Is it possible to submit this before that date? Our beta and GA endpoints are the same.

Hi @jessdejong - we can get this merged today, assuming the tests pass etc, but it's too late to be included in the release scheduled for Monday 11th Nov. The only changes that we would cherry-pick into a prepared release are bug fixes. I'm afraid this PR's changes would need to go out in the release on November 18th.

[SarahFrench]

I'm seeing a problem running the test TestAccManagedKafkaCluster_managedkafkaClusterCmekExample with the GA provider:

------- Stdout: -------
=== RUN   TestAccManagedKafkaCluster_managedkafkaClusterCmekExample
=== PAUSE TestAccManagedKafkaCluster_managedkafkaClusterCmekExample
=== CONT  TestAccManagedKafkaCluster_managedkafkaClusterCmekExample
    resource_managed_kafka_cluster_generated_test.go:94: Step 1/2 error: Error running pre-apply refresh: exit status 1
        Error: Invalid resource type
          on terraform_plugin_test.tf line 20, in resource "google_project_service_identity" "kafka_service_identity":
          20: resource "google_project_service_identity" "kafka_service_identity" {
        The provider hashicorp/google does not support resource type
        "google_project_service_identity".
--- FAIL: TestAccManagedKafkaCluster_managedkafkaClusterCmekExample (1.68s)
FAIL

The google_project_service_identity resource is only in the Beta provider, so that test will need to use google-beta still. I believe that to solve this you just need to re-add min_version: 'beta' to the example named managedkafka_cluster_cmek in mmv1/products/managedkafka/Cluster.yaml.

[jessdejong]

Hi @SarahFrench, thanks for letting me know about the releases. Given that we can not make it into the Monday release, I would like to still get this in ASAP so that we can be a part of the following release. Thanks!

Also -- I'm curious where you're seeing that the CMEK test is failing. Based on this issue comment, it looks like all of the GA tests are passing.

But anyway, I took your advice and added the beta min_version to the Cluster.yaml file and I'll wait to see if the tests pass. Would I also have to add the provider = google-beta line back into the mmv1/templates/terraform/examples/managedkafka_cluster_cmek.tf.tmpl file?

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 20 files changed, 2188 insertions(+), 27 deletions(-))
google-beta provider: Diff ( 6 files changed, 4 insertions(+), 62 deletions(-))
Open in Cloud Shell: Diff ( 8 files changed, 255 insertions(+))

[modular-magician]

Tests analytics

Total tests: 5
Passed tests: 4
Skipped tests: 0
Affected tests: 1

Click here to see the affected service packages

• managedkafka

#### Non-exercised tests

🔴 Tests were added that are GA-only additions and require manual runs:

• TestAccManagedKafkaCluster_managedkafkaClusterBasicExample
• TestAccManagedKafkaCluster_update
• TestAccManagedKafkaTopic_managedkafkaTopicBasicExample
• TestAccManagedKafkaTopic_update

Action taken

Found 1 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

• TestAccManagedKafkaCluster_managedkafkaClusterCmekExample

Get to know how VCR tests work

[modular-magician]

🔴 Tests failed during RECORDING mode:
TestAccManagedKafkaCluster_managedkafkaClusterCmekExample [Error message] [Debug log]

🔴 Errors occurred during RECORDING mode. Please fix them to complete your PR.

View the build log or the debug log for each test

[jessdejong]

Ok, looks like we have a new error on the CMEK test:

Error: Inconsistent dependency lock file
  | 
  | The following dependency selections recorded in the lock file are
  | inconsistent with the current configuration:
  |   - provider registry.terraform.io/hashicorp/google: required by this configuration but no version is selected

I'm going to try and add provider = google-beta back into the cmek test example to try and address this issue.

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

google provider: Diff ( 20 files changed, 2188 insertions(+), 16 deletions(-))
google-beta provider: Diff ( 6 files changed, 4 insertions(+), 40 deletions(-))
Open in Cloud Shell: Diff ( 8 files changed, 255 insertions(+))

[modular-magician]

Tests analytics

Total tests: 5
Passed tests: 5
Skipped tests: 0
Affected tests: 0

Click here to see the affected service packages

• managedkafka

#### Non-exercised tests

🔴 Tests were added that are GA-only additions and require manual runs:

• TestAccManagedKafkaCluster_managedkafkaClusterBasicExample
• TestAccManagedKafkaCluster_update
• TestAccManagedKafkaTopic_managedkafkaTopicBasicExample
• TestAccManagedKafkaTopic_update
  🟢 All tests passed!

View the build log

[SarahFrench]

Hi @SarahFrench, thanks for letting me know about the releases. Given that we can not make it into the Monday release, I would like to still get this in ASAP so that we can be a part of the following release. Thanks!

Also -- I'm curious where you're seeing that the CMEK test is failing. Based on #12264 (comment), it looks like all of the GA tests are passing.

But anyway, I took your advice and added the beta min_version to the Cluster.yaml file and I'll wait to see if the tests pass. Would I also have to add the provider = google-beta line back into the mmv1/templates/terraform/examples/managedkafka_cluster_cmek.tf.tmpl file?

Hi! On our PRs we run all the automated tests with the Beta version of the provider. I believe that comment is highlighting to the reviewer that those tests have newly been added to the GA provider code but require manual testing as while they pass for the Beta provider the aren't guaranteed to also pass for the GA provider. You won't be able to access this, but I ran the tests here using the GA provider code and saw the error.

I'll double check your latest changes. Edit: new GA test run here.

[jessdejong]

I'm unfortunately not able to access the team city link. Can you let me know when/if the tests complete? @SarahFrench

[jessdejong]

Hi @SarahFrench, were there any failures on the manual tests?

[SarahFrench]

The tests all passed, and the TestAccManagedKafkaCluster_managedkafkaClusterCmekExample test is still Beta-only as needed