[CPO] help on secrets for new GS bucket

[chrigl]

Hey folks,

May I ask you for help on getting a new secret into a job, because I'm pretty out of ideas.

I added the secret gcp_account_cpo t o zuul.d/secrets.yaml.
See https://github.com/theopenlab/openlab-zuul-jobs/blob/master/zuul.d/secrets.yaml#L561

# Google cloud account info for the new bucket gs://k8s-conform-provider-openstack
# See https://github.com/kubernetes/k8s.io/issues/1311#issuecomment-778507615
- secret:
    name: gcp_account_cpo
    data:
      key_json: !encrypted/pkcs1-oaep
...

And added it to the services alongside the secret I am about to replace gcp_account

• https://github.com/theopenlab/openlab-zuul-jobs/blob/master/zuul.d/jobs.yaml#L537
• https://github.com/theopenlab/openlab-zuul-jobs/blob/master/zuul.d/jobs.yaml#L598

But in the actual test run gcp_account_cpo.key_json is undefined. And thus, the upload fails.
e.g. http://status.openlabtesting.org/build/a7e9831c38204f9d885717c347f494f7

2021-02-19 17:10:51.166834 | TASK [export-gcp-account : DEBUG secret accessible]
2021-02-19 17:10:51.276543 | gcp_account_cpo.key_json is defined: False
...

2021-02-19 17:14:06.715475 | k8s-master | ERROR: (gcloud.auth.activate-service-account) Unable to read file [/tmp/gcp_cpo_key.json]: [Errno 2] No such file or directory: '/tmp/gcp_cpo_key.json'
2021-02-19 17:14:06.802850 | k8s-master | Run: ['gcloud', 'auth', 'list', '--filter=status:ACTIVE', '--format=value(account)']
2021-02-19 17:14:06.803013 | k8s-master | Run: ['gcloud', 'auth', 'activate-service-account', '--key-file=/tmp/gcp_cpo_key.json']
2021-02-19 17:14:06.804120 | k8s-master | Traceback (most recent call last):
2021-02-19 17:14:06.804181 | k8s-master |   File "upload_e2e.py", line 326, in <module>
2021-02-19 17:14:06.804210 | k8s-master |     main(sys.argv[1:])
2021-02-19 17:14:06.804246 | k8s-master |   File "upload_e2e.py", line 292, in main
2021-02-19 17:14:06.804266 | k8s-master |     activate_service_account(args.key_file, args.dry_run)
2021-02-19 17:14:06.804302 | k8s-master |   File "upload_e2e.py", line 218, in activate_service_account
2021-02-19 17:14:06.804324 | k8s-master |     subprocess.check_call(cmd)
2021-02-19 17:14:06.804343 | k8s-master |   File "/usr/lib/python3.7/subprocess.py", line 363, in check_call
2021-02-19 17:14:06.804361 | k8s-master |     raise CalledProcessError(retcode, cmd)
2021-02-19 17:14:06.804380 | k8s-master | subprocess.CalledProcessError: Command '['gcloud', 'auth', 'activate-service-account', '--key-file=/tmp/gcp_cpo_key.json']' returned non-zero exit status 1.

• https://github.com/theopenlab/openlab-zuul-jobs/blob/master/roles/export-gcp-account/tasks/main.yaml#L25

I am pretty sure, it's me holding it wrong, just not seeing it.

Thanks :)

[bzhaoopenstack]

Oh, I look back the said PR(theopenlab/openlab-zuul-jobs#1113) , looks there is a trouble about this file(https://github.com/theopenlab/openlab-zuul-jobs/blob/master/zuul.d/secrets.yaml#L561) , these encrypted info should be done by openlab maintainers on the machine which openlab zuul server runs.
So sorry for that, then if possible could you please share your content which needs to be encrypted? You can send it through private email. I will configure it once you send out and let you know, then we can test again.

My email is bzhaojyathousandy@gmail.com

[bzhaoopenstack]

https://logs.openlabtesting.org/logs/90/1390/b97392ed8777ce62a6cc05021fef880923fdf6a1/cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.20/cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.20/ef5833a/job-output.txt.gz

2021-02-22 09:31:29.064141 | TASK [export-gcp-account : DEBUG secret accessible]
2021-02-22 09:31:29.199494 | gcp_account_cpo.key_json is defined: True

...

2021-02-22 09:34:03.012875 | k8s-master | Activated service account credentials for: [openstack-conformance-logs@k8s-federated-conformance.iam.gserviceaccount.com]
2021-02-22 09:34:13.697796 | k8s-master | Run: ['gcloud', 'auth', 'list', '--filter=status:ACTIVE', '--format=value(account)']
2021-02-22 09:34:13.697906 | k8s-master | Run: ['gcloud', 'auth', 'activate-service-account', '--key-file=/tmp/gcp_key.json']
2021-02-22 09:34:13.697934 | k8s-master | Run: ['gsutil', '-q', '-h', 'Content-Type:text/plain', 'cp', '-', 'gs://k8s-conformance-openstack/pr-logs/ci-cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.20/1613978827/started.json'] stdin={"timestamp": 1613978827}
2021-02-22 09:34:13.698006 | k8s-master | Run: ['gsutil', '-q', '-h', 'Content-Type:text/plain', 'cp', '-', 'gs://k8s-conformance-openstack/pr-logs/ci-cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.20/1613978827/finished.json'] stdin={"timestamp": 1613986441, "result": "SUCCESS"}
2021-02-22 09:34:13.698061 | k8s-master | Run: ['gsutil', '-q', '-h', 'Content-Type:text/plain', 'cp', '/home/zuul/workspace/logs/kubernetes/e2e.log', 'gs://k8s-conformance-openstack/pr-logs/ci-cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.20/1613978827/build-log.txt']
2021-02-22 09:34:13.698091 | k8s-master | Run: ['gsutil', '-q', '-h', 'Content-Type:text/plain', 'cp', '/home/zuul/workspace/logs/kubernetes/junit_01.xml', 'gs://k8s-conformance-openstack/pr-logs/ci-cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.20/1613978827/artifacts/junit_01.xml']
2021-02-22 09:34:13.698111 | k8s-master | Run: ['gsutil', '-q', '-h', 'Content-Type:text/plain', 'cp', '/home/zuul/workspace/logs/kubernetes/junit_runner.xml', 'gs://k8s-conformance-openstack/pr-logs/ci-cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.20/1613978827/artifacts/junit_runner.xml']
2021-02-22 09:34:13.698145 | k8s-master | Uploading entry to: gs://k8s-conformance-openstack/pr-logs/ci-cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.20/1613978827
2021-02-22 09:34:13.698196 | k8s-master | Done.
2021-02-22 09:34:14.078971 | k8s-master | WARNING: [openstack-conformance-logs@k8s-federated-conformance.iam.gserviceaccount.com] appears to be a service account. Service account tokens cannot be revoked, but they will expire automatically. To prevent use of the service account token earlier than the expiration, delete or disable the parent service account.
2021-02-22 09:34:14.079814 | k8s-master | Revoked credentials:
2021-02-22 09:34:14.079859 | k8s-master |  - openstack-conformance-logs@k8s-federated-conformance.iam.gserviceaccount.com
2021-02-22 09:34:14.132284 | k8s-master | Run: ['gcloud', 'auth', 'revoke']
2021-02-22 09:34:14.145994 | k8s-master | + python3.7 upload_e2e.py '--junit=/home/zuul/workspace/logs/kubernetes/junit*.xml' --log=/home/zuul/workspace/logs/kubernetes/e2e.log --bucket=gs://k8s-conform-provider-openstack/pr-logs/ci-cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.20 --key-file=/tmp/gcp_cpo_key.json
2021-02-22 09:34:14.550174 | k8s-master | WARNING: The following filter keys were not present in any resource : status
2021-02-22 09:34:14.961177 | k8s-master | ERROR: (gcloud.auth.activate-service-account) Could not read json file /tmp/gcp_cpo_key.json: Expecting value: line 1 column 1 (char 0)
2021-02-22 09:34:15.013469 | k8s-master | Run: ['gcloud', 'auth', 'list', '--filter=status:ACTIVE', '--format=value(account)']
2021-02-22 09:34:15.013577 | k8s-master | Run: ['gcloud', 'auth', 'activate-service-account', '--key-file=/tmp/gcp_cpo_key.json']
2021-02-22 09:34:15.015112 | k8s-master | Traceback (most recent call last):
2021-02-22 09:34:15.015188 | k8s-master |   File "upload_e2e.py", line 326, in <module>
2021-02-22 09:34:15.015215 | k8s-master |     main(sys.argv[1:])
2021-02-22 09:34:15.015237 | k8s-master |   File "upload_e2e.py", line 292, in main
2021-02-22 09:34:15.015258 | k8s-master |     activate_service_account(args.key_file, args.dry_run)
2021-02-22 09:34:15.015296 | k8s-master |   File "upload_e2e.py", line 218, in activate_service_account
2021-02-22 09:34:15.015317 | k8s-master |     subprocess.check_call(cmd)
2021-02-22 09:34:15.015382 | k8s-master |   File "/usr/lib/python3.7/subprocess.py", line 363, in check_call
2021-02-22 09:34:15.015409 | k8s-master |     raise CalledProcessError(retcode, cmd)
2021-02-22 09:34:15.015430 | k8s-master | subprocess.CalledProcessError: Command '['gcloud', 'auth', 'activate-service-account', '--key-file=/tmp/gcp_cpo_key.json']' returned non-zero exit status 1.
2021-02-22 09:34:15.025901 | k8s-master | + true

Could you please give me the non-encrypted info? Please feel free for the account safety.

[bzhaoopenstack]

@chrigl Hi, the new CPO GS bucket had been updated. Sorry for this.

I will restart the zuul server to enable the new configuration.

[chrigl]

Works. Thank you soooo much @bzhaoopenstack ❤️