[Misc] feature: use kvcache webhook #1187

googs1025 · 2025-06-12T06:16:02Z

Pull Request Description

[Please provide a clear and concise description of your changes here]

Related Issues

Resolves: #[Insert issue number(s)]

Important: Before submitting, please complete the description above and review the checklist below.

Contribution Guidelines (Expand for Details)

We appreciate your contribution to aibrix! To ensure a smooth review process and maintain high code quality, please adhere to the following guidelines:

Pull Request Title Format

Your PR title should start with one of these prefixes to indicate the nature of the change:

[Bug]: Corrections to existing functionality
[CI]: Changes to build process or CI pipeline
[Docs]: Updates or additions to documentation
[API]: Modifications to aibrix's API or interface
[CLI]: Changes or additions to the Command Line Interface
[Misc]: For changes not covered above (use sparingly)

Note: For changes spanning multiple categories, use multiple prefixes in order of importance.

Submission Checklist

PR title includes appropriate prefix(es)
Changes are clearly explained in the PR description
New and existing tests pass successfully
Code adheres to project style and best practices
Documentation updated to reflect changes (if applicable)
Thorough testing completed, no regressions introduced

By submitting this PR, you confirm that you've read these guidelines and your changes align with the project's contribution standards.

gemini-code-assist · 2025-06-12T06:16:05Z

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

googs1025 · 2025-06-12T06:26:03Z

pkg/controller/kvcache/kvcache_controller.go

-			return backend
-		}
-
-		// TODO: Move validation logic to webhook.


Inspired by this todo

please make sure --disable-webhook mode can be supported as well. there're some users want to use individual controllers. they want the minimum setup

googs1025 · 2025-06-12T06:27:16Z

pkg/controller/kvcache/kvcache_controller.go

-	default:
-		return false
-	}
+	return kv.Annotations[constants.KVCacheLabelKeyBackend]


when we have KVCache webhook, then we can just get from Annotations

googs1025 · 2025-06-12T07:00:24Z

~~TODO: add integration test~~

added

pkg/controller/kvcache/kvcache_controller_ginkgo_test.go

Jeffwan · 2025-06-18T23:58:25Z

part of #710

googs1025 · 2025-06-19T04:42:57Z

will add some integration test this week

googs1025 · 2025-06-21T03:17:37Z

pkg/controller/kvcache/kvcache_controller.go

+	if backend == "" {
+		// In some case where webhooks are disabled (e.g., via --disable-webhook),
+		// mutating webhooks cannot inject default values into the spec. Using annotations
+		// ensures that users can still explicitly specify the backend, even when no webhook is active.


https://github.com/vllm-project/aibrix/pull/1187/files#r2155736323

We have made it compatible here and used comment to explain

googs1025 · 2025-06-21T03:32:30Z

test/integration/webhook/kvcache_webhook_test.go

+				cmpopts.IgnoreFields(orchestrationapi.KVCacheSpec{}, "Service"),
+				cmpopts.IgnoreFields(metav1.ObjectMeta{}, "UID", "ResourceVersion", "Generation", "CreationTimestamp", "ManagedFields")))
+		},
+		ginkgo.Entry("apply kvcache with no backend specified", &testDefaultingCase{


add mutating webhook test case

googs1025 · 2025-06-21T03:32:54Z

test/integration/webhook/kvcache_webhook_test.go

+		}),
+	)
+
+	type testValidatingCase struct {


add valid webhook case

googs1025 · 2025-06-29T03:06:13Z

@Jeffwan
I added some test to cover this feature 😄
PTAL when you have some time thanks

Jeffwan · 2025-06-30T09:45:38Z

@googs1025 overall looks good to me. Could you rebase the code and resolve the conflict?

googs1025 · 2025-06-30T10:51:24Z

@googs1025 overall looks good to me. Could you rebase the code and resolve the conflict?

done

Signed-off-by: googs1025 <googs1025@gmail.com>

* feature: use kvcache webhook Signed-off-by: googs1025 <googs1025@gmail.com> * fix integation validating test error Signed-off-by: googs1025 <googs1025@gmail.com> * add mutating webhook integration test for kvcache Signed-off-by: googs1025 <googs1025@gmail.com> --------- Signed-off-by: googs1025 <googs1025@gmail.com>

googs1025 force-pushed the kvcache_webhook branch 2 times, most recently from 50c303c to ed00058 Compare June 12, 2025 06:20

googs1025 commented Jun 12, 2025

View reviewed changes

googs1025 force-pushed the kvcache_webhook branch from ed00058 to 0c24134 Compare June 12, 2025 06:48

googs1025 force-pushed the kvcache_webhook branch from 0c24134 to 49ed023 Compare June 12, 2025 09:45

googs1025 commented Jun 12, 2025

View reviewed changes

pkg/controller/kvcache/kvcache_controller_ginkgo_test.go Show resolved Hide resolved

googs1025 force-pushed the kvcache_webhook branch 3 times, most recently from 61cb366 to a51508a Compare June 13, 2025 12:05

googs1025 force-pushed the kvcache_webhook branch 2 times, most recently from 9d165dd to 2332900 Compare June 21, 2025 03:15

googs1025 commented Jun 21, 2025

View reviewed changes

test/integration/webhook/kvcache_webhook_test.go

}),

)

type testValidatingCase struct {

Copy link

Collaborator Author

googs1025 Jun 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add valid webhook case

googs1025 changed the title ~~WIP: [Misc] feature: use kvcache webhook~~ [Misc] feature: use kvcache webhook Jun 21, 2025

googs1025 force-pushed the kvcache_webhook branch 2 times, most recently from 71b2848 to d97cb90 Compare June 21, 2025 04:04

googs1025 requested a review from Jeffwan June 25, 2025 08:49

googs1025 force-pushed the kvcache_webhook branch from 8a8b0d9 to 7f95457 Compare June 30, 2025 10:11

googs1025 added 3 commits July 3, 2025 16:48

feature: use kvcache webhook

b3dece8

Signed-off-by: googs1025 <googs1025@gmail.com>

fix integation validating test error

9390470

Signed-off-by: googs1025 <googs1025@gmail.com>

add mutating webhook integration test for kvcache

863576d

Signed-off-by: googs1025 <googs1025@gmail.com>

Jeffwan force-pushed the kvcache_webhook branch from 7f95457 to 863576d Compare July 3, 2025 08:48

Jeffwan merged commit 2d8de1d into vllm-project:main Jul 3, 2025
11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Misc] feature: use kvcache webhook #1187

[Misc] feature: use kvcache webhook #1187

Uh oh!

googs1025 commented Jun 12, 2025

Uh oh!

gemini-code-assist bot commented Jun 12, 2025

Uh oh!

googs1025 Jun 12, 2025

Uh oh!

Jeffwan Jun 18, 2025

Uh oh!

googs1025 Jun 12, 2025

Uh oh!

googs1025 commented Jun 12, 2025 •

edited

Loading

Uh oh!

Uh oh!

Jeffwan commented Jun 18, 2025

Uh oh!

googs1025 commented Jun 19, 2025

Uh oh!

googs1025 Jun 21, 2025 •

edited

Loading

Uh oh!

googs1025 Jun 21, 2025

Uh oh!

googs1025 Jun 21, 2025

Uh oh!

googs1025 commented Jun 29, 2025

Uh oh!

Jeffwan commented Jun 30, 2025

Uh oh!

googs1025 commented Jun 30, 2025

Uh oh!

Uh oh!

Uh oh!

[Misc] feature: use kvcache webhook #1187

[Misc] feature: use kvcache webhook #1187

Uh oh!

Conversation

googs1025 commented Jun 12, 2025

Pull Request Description

Related Issues

Pull Request Title Format

Submission Checklist

Uh oh!

gemini-code-assist bot commented Jun 12, 2025

Uh oh!

googs1025 Jun 12, 2025

Choose a reason for hiding this comment

Uh oh!

Jeffwan Jun 18, 2025

Choose a reason for hiding this comment

Uh oh!

googs1025 Jun 12, 2025

Choose a reason for hiding this comment

Uh oh!

googs1025 commented Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Jeffwan commented Jun 18, 2025

Uh oh!

googs1025 commented Jun 19, 2025

Uh oh!

googs1025 Jun 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

googs1025 Jun 21, 2025

Choose a reason for hiding this comment

Uh oh!

googs1025 Jun 21, 2025

Choose a reason for hiding this comment

Uh oh!

googs1025 commented Jun 29, 2025

Uh oh!

Jeffwan commented Jun 30, 2025

Uh oh!

googs1025 commented Jun 30, 2025

Uh oh!

Uh oh!

Uh oh!

googs1025 commented Jun 12, 2025 •

edited

Loading

googs1025 Jun 21, 2025 •

edited

Loading