Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Carvel Package POC for local development #1701

Merged
merged 15 commits into from
Nov 8, 2023

Conversation

benjaminapetersen
Copy link
Member

@benjaminapetersen benjaminapetersen commented Oct 3, 2023

Carvel Package POC integrated into local development

# optional to clean up the docker network
# the current script does add some additional networking
# to allow kind to talk to a local registry
docker system prune --all

# opt out of the ytt installs of components
# opt in to the alternate script 
# the PINNIPED_USE_LOCAL_KIND_REGISTRY will cause the prepare-for-integration-tests.sh script to 
# configure kind with a local registry rather than side-load the images directly into the cluster
PINNIPED_USE_LOCAL_KIND_REGISTRY=1 ./hack/prepare-for-integration-tests.sh --clean --pre-install ./hack/lib/carvel_packages/build.sh --alternate-deploy ./hack/lib/carvel_packages/deploy.sh

# run prep supervisor as usual
hack/prepare-supervisor-on-kind.sh --oidc

# run e2e test to verify 
ulimit -n 512 && source /tmp/integration-test-env && go test -v -race -count 1 -timeout 0 ./test/integration -run /TestSupervisorLogin_Browser
ulimit -n 512 && source /tmp/integration-test-env && go test -v -race -count 1 -timeout 0 ./test/integration -run  /TestE2EFullIntegration_Browser

Image repository is a local repository, requiring an /etc/hosts edit with the following lines:

# /etc/hosts
127.0.0.1 kind-registry.local

The images created follow this kind of naming convention:

The default `pinniped` image does not contain `/test/build` in the URL per the `values.yaml` file:
    - projects.registry.vmware.com/pinniped/pinniped-server@sha256:8bfe6fe313bf915da228579e48a7f2575aaea0fd9c27385735cb807d701d0131
        - projects.registry.vmware.com/pinniped/pinniped-server:latest
But the `install secret` referened by the PackageInstall does, indicating overrides passed in from the scripts:
- args:
   - image_repo: kind-registry.local:5000/test/build
   - image_tag: 0.0.0-1E7D4396-415A-4EA9-B44C-30F38A3578D8
- kind-registry.local:5000/test/build:0.0.0-1E7D4396-415A-4EA9-B44C-30F38A3578D8
  - note that since this doesn't get passed to kbld it doesn't get a rewrite to a sha256 like the other images
Then the `package_repository .imagpkg images` file:
- kind-registry.local:5000/test/build/package-concierge@sha256:c4e45af92c33d85eb1a988018b6bbb34a80c218f0ae4ef7853e28b0b9160e6d7
- kind-registry.local:5000/test/build/package-local-user-authenticator@sha256:276383f97c1e86b91aa38ebe340083c21a8b93a3a18b6269cd2c3049110cd713
- kind-registry.local:5000/test/build/package-supervisor@sha256:4dcbc9372fab1ebf78396a3a5203d58b3798c93c2d7775152dd17a2d3def5d6e

@codecov
Copy link

codecov bot commented Oct 10, 2023

Codecov Report

Merging #1701 (d5aa682) into main (e3d9eb7) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1701   +/-   ##
=======================================
  Coverage   78.96%   78.96%           
=======================================
  Files         166      166           
  Lines       15933    15933           
=======================================
  Hits        12582    12582           
- Misses       3035     3036    +1     
+ Partials      316      315    -1     

see 1 file with indirect coverage changes

@benjaminapetersen benjaminapetersen force-pushed the ben/carvel-package/local-registry branch 4 times, most recently from 47f6298 to 4bbd631 Compare October 12, 2023 15:24
hack/kind-up.sh Outdated Show resolved Hide resolved
hack/kind-up.sh Outdated Show resolved Hide resolved
@joshuatcasey
Copy link
Member

I've added an approval. The deploy files seem to have been translated correctly, and the hack files are easily changeable.

@benjaminapetersen
Copy link
Member Author

Curious about check-image-version failing, but waiting for the rest of the builds/tests to finish before I tamper.

benjaminapetersen and others added 11 commits November 8, 2023 13:10
- update kind config to include local registry
- configure kind cluster to talk to local registry
- docker build & push pinniped dev code to local registry
- deploy dev code of the following via the local registry:
  - concierge
  - supervisor
  - local-user-authenticator
- Update values.yaml for supervisor,concierge to schema files
- Update values.yaml for local-user-authenticator to schema file
- Add ytt openapi-v3 generation to build carvel package script
- Add supervisor carvel package files
- Add concierge carvel package files
- Add local-user-authenticator carvel package files
- Add hack script to build openapi-v3 files
- add --post-install to hack/prepare-for-integration-tests.sh
- cleanup local registry in kind-down.sh
- webhook_ca_bundle moved in hack script
- adjust were to call post-install script
- deploy/{}/values.yml image_pull_dockerconfigjson type change to base64 string
- Add PINNIPED_USE_LOCAL_KIND_REGISTRY env var
 - ensures regular use of hack/prepare-for-integration-tests.sh
  - PINNIPED_USE_LOCAL_KIND_REGISTRY=1  ./hack/prepare-for-integration-tests.sh --clean --alternate-deploy ./hack/noop.sh --post-install ./hack/build-carvel-packages.sh
  - ./hack/prepare-for-integration-tests.sh --clean
- if PINNIPED_USE_LOCAL_KIND_REGISTRY for kind-down.sh in hack/prepare-for-integration-tests.sh
- Split carvel build & deploy scripts, add --pre-install flag
  - add pre-install flag to hack/prepare-for-integration-tests.sh
  - split /hack/build-carvel-packages.sh and
    /hack/deploy-carvel-packages.sh
- Remove --alternate-deploy-* flags from hack script
- Move scripts to hack/lib/carvel_packages
- Split build.sh deploy.sh
- Separate template files from install artifacts
- Generate all install artifacts in $root/deploy_carvel
  - remove $root/deploy_carvel from git
- Extract ytt values to file in hack/prepare-for-integration-tests.sh
- pass registry/repo to carvel build scripts
Co-authored-by: Ryan Richard <richardry@vmware.com>
@cfryanr cfryanr force-pushed the ben/carvel-package/local-registry branch from 6b2ea5d to 88fbaad Compare November 8, 2023 21:10
cfryanr and others added 2 commits November 8, 2023 13:37
Co-authored-by: Benjamin A. Petersen <ben@benjaminapetersen.me>
@cfryanr cfryanr force-pushed the ben/carvel-package/local-registry branch from 88fbaad to d4e2622 Compare November 8, 2023 21:38
cfryanr and others added 2 commits November 8, 2023 13:45
Co-authored-by: Benjamin A. Petersen <ben@benjaminapetersen.me>
Co-authored-by: Benjamin A. Petersen <ben@benjaminapetersen.me>
@cfryanr cfryanr merged commit 80d5f42 into main Nov 8, 2023
39 checks passed
@cfryanr cfryanr deleted the ben/carvel-package/local-registry branch November 8, 2023 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants