Update BootstrapProject to support use across multiple projects/environments

[roaks3]

Only affects tests

With the project split work coming soon to support isolating our test environments, one of the issues we run into is that the projects for all 3 environments are occupying the same space. This is problematic for bootstrapped projects, because each environment will attempt to use the same project for these tests (and potentially do other things like change billing account or permissions).

The fix proposed here is to simply append the "environment test project" id to the end of the bootstrapped project id. This way, each environment will end up creating a unique project that is owned by that environment.

I'm not sure if this will fix hashicorp/terraform-provider-google#13711, but it's possible, because I think we started seeing issues that as soon as the new GA project started running. If not, then I can follow up after this gets merged to double check that there is not some other issue related to the multiple environments.

If this PR is for Terraform, I acknowledge that I have:

• Searched through the issue tracker for an open issue that this either resolves or contributes to, commented on it to claim it, and written "fixes {url}" or "part of {url}" in this PR description. If there were no relevant open issues, I opened one and commented that I would like to work on it (not necessary for very small changes).
• Ensured that all new fields I added that can be set by a user appear in at least one example (for generated resources) or third_party test (for handwritten resources or update tests).
• Generated Terraform providers, and ran make test and make lint in the generated providers to ensure it passes unit and linter tests.
• Ran relevant acceptance tests using my own Google Cloud project and credentials (If the acceptance tests do not yet pass or you are unable to run them, please let your reviewer know).
• Read the Release Notes Guide before writing my release note below.

Release Note Template for Downstream PRs (will be copied)

[roaks3]

Adding @trodge as well to weigh in, since he introduced this function initially

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 7 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 2 files changed, 7 insertions(+), 2 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[modular-magician]

Tests analytics

Total tests: 0
Passed tests 0
Skipped tests: 0
Failed tests: 0

Errors occurred during REPLAYING mode. Please fix them to complete your PR
View the build log

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 7 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 2 files changed, 7 insertions(+), 2 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[modular-magician]

Tests analytics

Total tests: 0
Passed tests 0
Skipped tests: 0
Failed tests: 0

Errors occurred during REPLAYING mode. Please fix them to complete your PR
View the build log

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 7 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 2 files changed, 7 insertions(+), 2 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2480
Passed tests 2223
Skipped tests: 255
Failed tests: 2

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccApigeeAddonsConfig_apigeeAddonsTestExample|TestAccWorkstationsWorkstationConfig_workstationConfigEncryptionKeyExample

[modular-magician]

Tests passed during RECORDING mode:
TestAccApigeeAddonsConfig_apigeeAddonsTestExample[Debug log]

Tests failed during RECORDING mode:
TestAccWorkstationsWorkstationConfig_workstationConfigEncryptionKeyExample[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

[trodge]

Overall LGTM, just a few nits.

[trodge]

Is a prefix better than a suffix for distinguishing bootstrapped projects?

[roaks3]

I honestly couldn't come up with a good reason for one over the other. The prefix, as I have it written, will make the projects a little more clearly "tf-" test resources (ie. they can be deleted/recreated if necessary), but a suffix might be slightly nicer for grouping alphabetically. That said, I would still lean toward the prefix strategy to start.

[trodge]

To stay under 30 characters (max length of a project ID) let's shorten tf-fail-cluster- to tf-cc-f-.

[roaks3]

Yea, the max length is a good callout. We actually may need to tweak the strategy overall, as ci-test-project-nightly-beta is already 29 characters.

I'll think on this a bit more, but we could potentially trim the environment project id, eg. nightly-beta.

[hao-nan-li]

How about something like tf-ci-test-?

[roaks3]

I made an update so that the suffix is shorter, so now we can technically keep the old prefix. Open to suggestions though after that change was made.

With the current implementation, here are the names we'd see:

• VCR/presubmit - tf-fail-cluster-188019
• Beta - tf-fail-cluster-nightly-beta
• GA - tf-fail-cluster-nightly-ga
• Random local test - tf-fail-cluster-ryantest

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 11 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 2 files changed, 11 insertions(+), 2 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[modular-magician]

Tests analytics

Total tests: 0
Passed tests 0
Skipped tests: 0
Failed tests: 0

Errors occurred during REPLAYING mode. Please fix them to complete your PR
View the build log

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 11 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 2 files changed, 11 insertions(+), 2 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2487
Passed tests 2227
Skipped tests: 258
Failed tests: 2

Action taken

Triggering VCR tests in RECORDING mode for the tests that failed during VCR. Click here to see the failed tests

TestAccComposerEnvironment_withWebServerConfig|TestAccWorkstationsWorkstationConfig_workstationConfigEncryptionKeyExample

[modular-magician]

Tests passed during RECORDING mode:
TestAccWorkstationsWorkstationConfig_workstationConfigEncryptionKeyExample[Debug log]

Tests failed during RECORDING mode:
TestAccComposerEnvironment_withWebServerConfig[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 11 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 2 files changed, 11 insertions(+), 2 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[modular-magician]

Tests analytics

Total tests: 0
Passed tests 0
Skipped tests: 0
Affected tests: 0

Errors occurred during REPLAYING mode. Please fix them to complete your PR
View the build log

[modular-magician]

Hi there, I'm the Modular magician. I've detected the following information about your changes:

Diff report

Your PR generated some diffs in downstreams - here they are.

Terraform GA: Diff ( 2 files changed, 11 insertions(+), 2 deletions(-))
Terraform Beta: Diff ( 2 files changed, 11 insertions(+), 2 deletions(-))
TF Validator: Diff ( 2 files changed, 3 insertions(+), 3 deletions(-))

[modular-magician]

Tests analytics

Total tests: 2491
Passed tests 2231
Skipped tests: 258
Affected tests: 2

Action taken

Found 2 affected test(s) by replaying old test recordings. Starting RECORDING based on the most recent commit. Click here to see the affected tests

TestAccFirebaserulesRelease_BasicRelease|TestAccWorkstationsWorkstationConfig_workstationConfigEncryptionKeyExample

Get to know how VCR tests work

[modular-magician]

Tests passed during RECORDING mode:
TestAccFirebaserulesRelease_BasicRelease[Debug log]

Tests failed during RECORDING mode:
TestAccWorkstationsWorkstationConfig_workstationConfigEncryptionKeyExample[Error message] [Debug log]

Please fix these to complete your PR
View the build log or the debug log for each test

[roaks3]

@hao-nan-li @trodge Would either of you be able to give this another look when you have a moment? It looks like it might have stalled out, and it is a blocker for the project split.

[hao-nan-li]

LGTM, just a quick recap:

VCR/presubmit - tf-fail-cluster-188019
Beta - tf-fail-cluster-nightly-beta
GA - tf-fail-cluster-nightly-ga
Random local test - tf-fail-cluster-ryantest