VolumeSource: OCI Artifact and/or Image

[sallyom]

Enhancement Description

• One-line enhancement description (can be used as a release note): VolumeSource: OCI Artifact or OCI Image

• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/4639-oci-volume-source

• Discussion Link:

• Primary contact (assignee): sgrunert@redhat.com, somalley@redhat.com

• Responsible SIGs: sig-node, sig-storage

• Enhancement target (which target equals to which milestone):

  • Alpha release target (x.y): 1.31
  • Beta release target (x.y):
  • Stable release target (x.y):

• Alpha

  • KEP (k/enhancements) update PR(s):
    • KEP-4639: adding OCI VolumeSource #4642
    • KEP-4639: update to most recent implementation state #4897
  • Code (k/k) update PR(s):
    • [KEP-4639] Add OCI VolumeSource CRI API kubernetes#125659
      • [KEP-4639] Update CRI API and workflow #4751
    • [KEP-4639] Add ImageVolumeSource API kubernetes#125660
    • [KEP-4639] Add ImageVolumeSource implementation kubernetes#125663
    • Nice to have:
      • [KEP-4639] Mention that fsGroupChangePolicy has no effect kubernetes#126281
      • Fix runtime panic in imagevolume CanSupport method kubernetes#126323
      • [KEP-4639] Add ImageVolumeSource node e2e tests kubernetes#126220
  • Out of tree update PR(s):
    • Add OCI Volume Source support to crictl [create|run] --with-pull kubernetes-sigs/cri-tools#1464
    • Add OCI/Image Volume Source support cri-o/cri-o#8317
    • Make ImageVolume garbage collection work cri-o/cri-o#8408
    • CRI-O: add pull-kubernetes-node-crio-cgrpv2-imagevolume-e2e test job test-infra#33071
    • Fix imagevolume test focus test-infra#33088
  • Docs (k/website) update PR(s):
    • Document OCI volume sources / KEP-4639 website#46946
    • Blog post about OCI volume sources / KEP-4639 website#46960

• Beta

  • KEP (k/enhancements) update PR(s):
  • Code (k/k) update PR(s):
  • Docs (k/website) update(s):

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[sallyom]

I've had informal discussions about this - there's enough interest IMO to open a KEP & I will present this issue at upcoming sig-node, sig-storage mtgs with a KEP draft

/sig node
/sig storage

[xing-yang]

Can you use the Volume Populator? It allows you to create a PVC from an extenal data source. https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1495-volume-populators

[saschagrunert]

Happy to support here from a SIG node perspective.

cc @kubernetes/sig-node-proposals

[mikebrow]

+1 happy to help from the SIG-Node/CRI and OCI image/distribution spec perspectives..

[SergeyKanzhelev]

/label lead-opted-in
/milestone v1.31

as discussed at SIG Node meeting this week, we will try and see if this can make it to 1.31

[SergeyKanzhelev]

/stage alpha

[rhuss]

For reference, in KServe a workaround for directly accessing files within an OCI image is currently implemented and available via a sidecar approach ("modelcar") by leveraging root FS system access via the /proc filesystem when shareProcessNamespace: true is set on the Pod. You can find details in the KServe documentation and in the Design Document. It actually implements the desired behavior with current means, but of course is more or less just a workaround for the OCI volume type (as discussed here and raised already a long time ago in kubernetes/kubernetes#831).

So KServe would be more than happy to leverage such a volume type, and we are happy to support any efforts in this direction.

[sreeram-venkitesh]

/stage alpha

[sreeram-venkitesh]

Hello @sallyom 👋, v1.31 Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 14th June 2024 / 19:00 PDT Thursday 13th June 2024.

This enhancement is targeting for stage alpha for v1.31 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: v1.31. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here). If your production readiness review is not completed yet, please make sure to fill the production readiness questionnaire in your KEP by the PRR Freeze deadline (6th June) so that the PRR team has enough time to review your KEP before the enhancements freeze.

For this KEP, most of the above items are taken care of in #4642. We'd need to do the following:

• Update the status from provisional to implementable in the kep.yaml file here
• Create a prod-readiness yaml file as shown here.
• Update the graduation criteria in the KEP readme file.
• Make sure that the PRR questionnaire is filled.

The status of this enhancement is marked as At risk for enhancements freeze. Once the above tasks are done, I can mark it as tracked.

If you anticipate missing enhancements freeze, you can file an exception request in advance. Let me know if you have any questions! Thank you!

[sreeram-venkitesh]

@sallyom Pinging once again as a slight reminder that we're approaching the enhancements freeze deadline on 14th June, this Friday!

[dipesh-rawat]

Hi @sallyom @SergeyKanzhelev 👋, 1.31 Enhancements team here,

Just a quick friendly reminder as we approach the enhancements freeze in few hours, at 02:00 UTC Friday 14th June 2024 / 19:00 PDT Thursday 13th June 2024.

The current status of this enhancement is marked as at risk for enhancement freeze. There are a few requirements mentioned in the comment #4639 (comment) that are addressed as part of PR #4642 which still needs to be merged.

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

[dipesh-rawat]

Hello @sallyom @SergeyKanzhelev 👋, 1.31 Enhancements team here.

Unfortunately, this enhancement did not meet requirements for enhancements freeze.

If you still wish to progress this enhancement in v1.31, please file an exception request as soon as possible, within three days. If you have any questions, you can reach out in the #release-enhancements channel on Slack and we'll be happy to help. Thanks!

[sreeram-venkitesh]

/milestone clear

[bitoku]

/assign

[sreeram-venkitesh]

Marked as tracked for enhancements freeze! Thanks everyone!

[Princesso]

Hello @bitoku, @sallyom 👋, 1.31 Docs Lead here.
Does this enhancement work planned for 1.31 require any new docs or modifications to existing docs?
If so, please follow the steps here to open a PR against dev-1.31 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday June 27, 2024 18:00 PDT.
Also, take a look at Documenting for a release to get yourself familiarised with the docs requirement for the release.
Thank you!

[pacoxu]

kubernetes/kubernetes#108405 (comment)

• Should oci volume pulling block other pod image pulls if the kubelet uses the default serialize-image-pulls? Currently, we shared the same image puller. A big model image pulling may block other new pod image pulling(startup) in the worst case.
• And should we add this to the beta or GA criteria to change the image pull behavior or not count OCI volume as part of pod image pulling? Or is this expected behavior?

[saschagrunert]

• Should oci volume pulling block other pod image pulls if the kubelet uses the default serialize-image-pulls? Currently, we shared the same image puller. A big model image pulling may block other new pod image pulling(startup) in the worst case.
• And should we add this to the beta or GA criteria to change the image pull behavior or not count OCI volume as part of pod image pulling? Or is this expected behavior?

Thank you for the input here! I personally think we should make as less exceptions in behavior as possible for OCI image volumes in comparison to regular images.

[tjons]

Hi, enhancements lead here - I inadvertently added this to the 1.32 tracking board 😀. Please readd it if you wish to progress this enhancement in 1.32.

/remove-label lead-opted-in

[saschagrunert]

@tjons there is no graduation planned for this feature in v1.32.

[haircommander]

/milestone v1.32
/label lead-opted-in

[tjons]

@saschagrunert @haircommander is there still no graduation planned for this feature in 1.32? If so, is the plan to iterate on alpha? and if not, why is this tracked for 1.32 :)?

[jenshu]

Hello @saschagrunert @sallyom 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on 02:00 UTC Friday 11th October 2024 / 19:00 PDT Thursday 10th October 2024.

This enhancement is targeting stage alpha for v1.32 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: v1.32.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here). If your production readiness review is not completed yet, please make sure to fill the production readiness questionnaire in your KEP by the PRR Freeze deadline on Thursday 3rd October 2024 so that the PRR team has enough time to review your KEP.

For this KEP, we would just need to update the following:

• Fill out the Test Plan section of the readme
• Fill out the Graduation Criteria section of the readme
• In your kep.yaml the latest-milestone needs to be updated to v1.32

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well.

If you anticipate missing enhancements freeze, you can file an exception request in advance. Thank you!

[sohankunkerkar]

@saschagrunert @sallyom, I'm part of the SIG-NODE KEPs wrangler team for this release. It looks like the plan for this release is to push this feature to Beta. Could one of you open a PR to move it forward?

[haircommander]

@saschagrunert is out this week, but I'm going to open a PR in his stead to get it on PRR's purview, then he'll probably correct all the incorrect stuff I've written next week 🙂

[haircommander]

update: I've opened #4897

[haircommander]

/stage beta

[haircommander]

/stage alpha
/remove-milestone v1.32
/remove-label lead-opted-in

I believe we won't make stage progress in 1.32. We can still push for subpath support but I don't think we need to track that here.

[jenshu]

1.32 Enhancements team here. Per the above comments, I've updated the 1.32 status to deferred

[tjons]

/milestone clear