-
Notifications
You must be signed in to change notification settings - Fork 347
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CRDs upgrade support #4001
Comments
While trying to split the chart into 2 charts(app chart and CRDs chart) internally CRDs chart installation failed because helm release secret exceeded max 1MB size limit. In order to resolve that issue I split the CRDs chart into 2 more charts - eg CRDS and gateway api CRDs. |
thinking out loud, we could add two additional templates (w/o deleting the This solves 2 use cases
the naming for the helm knobs can be improved, of course |
@arkodg installing both EG and gateway-api CRDs from the same chart won't work since helm release secret will exceed 1MB limit as I mentioned here. What about keeping the crds folder and create 2 additional CRDs charts? |
ah thanks for trying it out @shahar-h . Looks like we have a decision to make, neither being ideal
Common issues
|
argo also puts |
@shahar-h can you help investigate who is the consuming most of the space ? |
Sure, will do. |
thanks ! I did a little digging, one way to reduce size could be to rm the API descriptions, but that would break |
I computed eg CRDS & gwapi CRDs size by creating separate charts and installing each chart. Then I computed helm secret release size for each one of them with the following command: kubectl -n envoy-gateway-system get secret <helm-release-secret-name> -o jsonpath='{.data.release}' | base64 -d | wc -c Results:
Total: 1.17MB > 1MB size limit Note that the helm release secret size is lower that real CRDs size since helm uses gzip compression. BTW there is an open PR to split helm releases into multiple k8s secrets. |
thanks for analyzing this ! |
Description:
Currently Gateway API and EG CRDs are located in EG
gateway-helm
chart crds folder. This means that CRDs are not upgraded on chart upgrade, and need to be manually upgraded beforehand.Possible solutions:
Real world example: cert-manager chart has CRDs as part of templates folder and exposes
crds.enabled
flag.It also allows to decide if we want to keep the CRDs on chart uninstallation with
crds.keep
flag, by leveraging"helm.sh/resource-policy": keep
helm annotation.See: https://cert-manager.io/v1.13-docs/installation/helm/#crd-considerations.
However, this is a breaking change for consumers that use EG chart as a sub chart and have custom resources as part of the main chart.
Real world example: Istio provides a base chart that installs CRDs: https://istio.io/latest/docs/setup/install/helm/#installation-steps.
Both 1 and 2 can also solve #3094 by providing a separate flags(1)/charts(2) for Gateway API and EG CRDs.
Also Related:
WDYT?
The text was updated successfully, but these errors were encountered: