Can not convert custom resource to the new storage api version

[mariapoliss]

I created my own Custom Resource Definition. Let`s call it logging.customresource.com. It has v1 and I created custom resource logging1.
I need to update this CRD and add v2 as storage version. Some of the properties changed its` names in v2 and the old names removed from this new version.
I deployed conversion webhook and configured it in logging.customresource.com configuration. But conversion webhook does not change and save the actual custom resource logging1 to the storage version. It allows only get the resource to my or other requests to this resource, as I understood.

I would like to make it clear: if I want to mark v1 as deprecated and stop to support it soon, I need to make upgrade of the logging1.

According to documentation

If the storage version changes, existing objects are never converted automatically.

What are the ways to convert resource? I mean saving resource to etcd with the last storage version.

But when I try to modify it, apiVersion=v2 was saved (I checked it in etcd), but the new properties did not apply and the old values are lost. And even if logging1 has apiVersion=v2, I can not add new properties. It is just ignored without errors.

In the section Upgrade existing objects to a new stored version in option 2 said:

Write an upgrade procedure to list all existing objects and write them with the same content. This forces the backend to write objects in the current storage version

What does upgrade procedure mean?
Do I need to delete logging1 of version v1 and then recreate it with v2 or how to make work the existing custom resource but with the last api version? And maybe it is better to watch CRD logging.customresource.com and when storage version changed, I need to recreate custom resource (if I would like to support only v2)?

Also there was a comment kubernetes/website#27534 (comment) and good question about describing what happens in different cases with or without conversion webhook, but it is not described in documentation yet unfortunately.

Documentation link: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/#upgrade-existing-objects-to-a-new-stored-version

[k8s-ci-robot]

This issue is currently awaiting triage.

SIG Docs takes a lead on issue triage for this website, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Ritikaa96]

/sig api-machinery
/wg api-expression

[sftim]

/language en

[sftim]

This is mostly:
/kind support

although we could improve the phrasing to help readers understand the meaning. Here's a bit of text to consider rewording:

Write an upgrade procedure to list all existing objects and write them with the same content. This forces the backend to write objects in the current storage version

By “upgrade procedure” I think the document means that you either:

• write a document aimed at human readers. One part of that document must explain how to list all existing objects and then write back each object, unchanged. When a person follows this procedure, the write that the trigger prompts the storage version upgrade.
• write a computer program. Part of the functionality of that progam will be to list all existing objects and then write back each object, unchanged. That scripted write triggers the storage version upgrade.

You get to choose which approach you prefer.

[mariapoliss]

Thanks!
I found the cause why the parameters added in the v2 were lost and faced the next problem:

I have created go reconciler that can change status subresource of my Custom Resource using client.Status().Patch() from sigs.k8s.io/controller-runtime/pkg/client.
I created the struct Logging with the fields of my custom resource in the apis/customresource/v1. So when I do client.Get() I get the object of type Logging with apiVersion v1.
Then I upgraded the custom resource logging1 to v2 (added new fields too) manually.
Then I`m going to update the reconciler image to start work with the new version of struct in apis/customresource/v2, but did not make it yet, reconcile of previous version of reconciler application has started.
And the client.Status.Patch() method receives object of v1 (because client.Get() returned us the version v1, as reconciler requested) and executes patch, but it removes the new fields of v2 that we added recently. So we have lost the new values.

I would like to know, is it expected behaviour of client.Status.Patch()? If it is expected what is the way to deal with the case I described?

[sftim]

Given that this is a request for help, it's the wrong place, but:
/transfer kubernetes

[fedebongio]

/help
/cc @apelisse @alexzielenski @jiahuif @seans3 in case you can help
/triage accepted

[k8s-ci-robot]

@fedebongio:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help
/cc @apelisse @alexzielenski @jiahuif @seans3 in case you can help
/triage accepted

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[apelisse]

is the conversion from v1 to v2 round-trippable? If you take an object in v2 with all its fields set, convert it to v1 and back to v2, will it drop some fields?

[mariapoliss]

No. v2 has fields that have not analogue in v1.
Yes, fields will be removed after v2 to v1 converting.

[apelisse]

Then that's not a valid conversion. Conversions are supposed to be round-trippable.

[k8s-triage-robot]

This issue has not been updated in over 1 year, and should be re-triaged.

You can:

• Confirm that this issue is still relevant with /triage accepted (org members only)
• Close this issue with /close

For more details on the triage process, see https://www.kubernetes.dev/docs/guide/issue-triage/

/remove-triage accepted

[cici37]

/cc @muff1nman Thanks for jumping in :)

/triage accepted