[Test][HistoryServer] E2E test for live clusters #4406

win5923 · 2026-01-17T07:40:52Z

Why are these changes needed?

Add E2E test to verify history server can proxy API requests to live Ray clusters
Refactor test support code into support package

Live clusters test:

Verifies /clusters/ endpoint returns live clusters with sessionName=live
Tests /enter_cluster/{ns}/{name}/live endpoint sets cluster context correctly
Validates all implemented history server endpoints proxy successfully (status < 500)

Related issue number

Closes #4377

Checks

I've made sure the tests are passing.
Testing Strategy
- Unit tests
- Manual tests
- This PR is not tested :(

win5923 · 2026-01-17T10:11:54Z

historyserver/test/support/historyserver.go

+	sa := &corev1.ServiceAccount{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:      "historyserver",
+			Namespace: namespace.Name,
+		},
+	}
+	clusterRole := &rbacv1.ClusterRole{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: "raycluster-reader",
+		},
+		Rules: []rbacv1.PolicyRule{
+			{
+				APIGroups: []string{"ray.io"},
+				Resources: []string{"rayclusters"},
+				Verbs:     []string{"list", "get"},
+			},
+		},
+	}
+	clusterRoleBinding := &rbacv1.ClusterRoleBinding{
+		ObjectMeta: metav1.ObjectMeta{
+			Name: fmt.Sprintf("historyserver-%s", namespace.Name),
+		},
+		Subjects: []rbacv1.Subject{
+			{
+				Kind:      "ServiceAccount",
+				Name:      "historyserver",
+				Namespace: namespace.Name, // Use the test namespace
+			},
+		},
+		RoleRef: rbacv1.RoleRef{
+			APIGroup: "rbac.authorization.k8s.io",
+			Kind:     "ClusterRole",
+			Name:     "raycluster-reader",
+		},
+	}


The service_account.yaml file has a hardcoded namespace which doesn't work with dynamic test namespaces.

kuberay/historyserver/config/service_account.yaml

Lines 16 to 27 in 0da1dea

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRoleBinding

metadata:

name: historyserver

namespace: default

subjects:

- kind: ServiceAccount

name: historyserver

namespace: default

roleRef:

kind: ClusterRole

name: raycluster-reader

Signed-off-by: win5923 <ken89@kimo.com>

win5923 · 2026-01-17T13:17:26Z

historyserver/Makefile


 .PHONY: localimage-historyserver
-localimage-historyserver: dockerbuilder_instance
+localimage-historyserver:


Fix https://buildkite.com/ray-project/ray-ecosystem-ci-kuberay-ci/builds/12688/steps/canvas?jid=019bcc10-54ca-4443-9d59-1c6d85e5b83a

I think buildkite server don't have docker buildx

Signed-off-by: win5923 <ken89@kimo.com>

win5923 · 2026-01-18T16:56:22Z

historyserver/test/support/historyserver.go

+// Excluded endpoints that are not yet implemented:
+//   - /events
+//   - /api/cluster_status
+//   - /api/grafana_health
+//   - /api/prometheus_health
+//   - /api/data/datasets/{job_id}
+//   - /api/jobs
+//   - /api/serve/applications
+//   - /api/v0/placement_groups
+//   - /api/v0/logs/file


Some APIs work for live clusters but are not implemented for dead clusters.
I have excluded the unimplemented ones for dead clusters for now.

Signed-off-by: win5923 <ken89@kimo.com>

win5923 · 2026-01-21T18:35:12Z

historyserver/test/support/historyserver.go

+//   - /api/v0/placement_groups
+//   - /api/v0/logs/file
+var HistoryServerEndpoints = []string{
+	"/nodes?view=summary",


Endpoint /nodes should not return server error, got 500: {"result": false, "msg": "Unknown view None", "data": {}} Expected <int>: 500 to be < <int>: 500

Just a workaround for live cluster here, need another way to support dead cluster /node endpoint.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

historyserver/test/support/historyserver.go

Signed-off-by: win5923 <ken89@kimo.com>

win5923 force-pushed the historyserver-live-cluster branch 6 times, most recently from 88d8a21 to 1b2f5c4 Compare January 17, 2026 10:02

win5923 marked this pull request as ready for review January 17, 2026 10:05

win5923 force-pushed the historyserver-live-cluster branch 2 times, most recently from 570ddd8 to 947900f Compare January 17, 2026 10:08

win5923 commented Jan 17, 2026

View reviewed changes

This comment was marked as outdated.

Sign in to view

win5923 force-pushed the historyserver-live-cluster branch 3 times, most recently from 3a355b6 to 16d4f08 Compare January 17, 2026 10:46

win5923 marked this pull request as draft January 17, 2026 12:00

win5923 force-pushed the historyserver-live-cluster branch from 16d4f08 to f0d2142 Compare January 17, 2026 12:09

win5923 marked this pull request as ready for review January 17, 2026 12:09

win5923 force-pushed the historyserver-live-cluster branch from f0d2142 to 18e8fb4 Compare January 17, 2026 12:10

This comment was marked as outdated.

Sign in to view

win5923 force-pushed the historyserver-live-cluster branch 4 times, most recently from e06809f to b1ec511 Compare January 17, 2026 13:05

win5923 marked this pull request as draft January 17, 2026 13:15

[Test][HistoryServer] E2E test for live clusters

9c1bd43

Signed-off-by: win5923 <ken89@kimo.com>

win5923 force-pushed the historyserver-live-cluster branch from b1ec511 to 9c1bd43 Compare January 17, 2026 13:17

win5923 marked this pull request as ready for review January 17, 2026 13:17

win5923 commented Jan 17, 2026

View reviewed changes

This comment was marked as outdated.

Sign in to view

Reslove comments

b507364

Signed-off-by: win5923 <ken89@kimo.com>

win5923 commented Jan 18, 2026

View reviewed changes

Merge branch 'upstream-master' into historyserver-live-cluster

786c3ee

This comment was marked as outdated.

Sign in to view

JiangJiaWei1103 added this to My Kuberay & Ray Jan 20, 2026

JiangJiaWei1103 moved this to To Review in My Kuberay & Ray Jan 20, 2026

Future-Outlier mentioned this pull request Jan 21, 2026

[historyserver] implement grafana health for live session #4425

Open

4 tasks

create getClusterFromList helper function

89c4e52

Signed-off-by: win5923 <ken89@kimo.com>

win5923 force-pushed the historyserver-live-cluster branch from 958a8d3 to 5fcfc62 Compare January 21, 2026 18:06

This comment was marked as outdated.

Sign in to view

win5923 force-pushed the historyserver-live-cluster branch from 5fcfc62 to 1601f74 Compare January 21, 2026 18:31

Merge branch 'upstream-master' into historyserver-live-cluster

bcce969

win5923 force-pushed the historyserver-live-cluster branch from 1601f74 to bcce969 Compare January 21, 2026 18:34

win5923 commented Jan 21, 2026

View reviewed changes

cursor bot reviewed Jan 21, 2026

View reviewed changes

historyserver/test/support/historyserver.go Show resolved Hide resolved

Add cleanup for clusterRoleBinding

a2c4e84

Signed-off-by: win5923 <ken89@kimo.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Test][HistoryServer] E2E test for live clusters #4406

[Test][HistoryServer] E2E test for live clusters #4406

win5923 commented Jan 17, 2026 •

edited

Loading

Uh oh!

win5923 Jan 17, 2026

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

win5923 Jan 17, 2026 •

edited

Loading

Uh oh!

This comment was marked as outdated.

Uh oh!

win5923 Jan 18, 2026 •

edited

Loading

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

win5923 Jan 21, 2026

Uh oh!

cursor bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	apiVersion: rbac.authorization.k8s.io/v1
	kind: ClusterRoleBinding
	metadata:
	name: historyserver
	namespace: default
	subjects:
	- kind: ServiceAccount
	name: historyserver
	namespace: default
	roleRef:
	kind: ClusterRole
	name: raycluster-reader

[Test][HistoryServer] E2E test for live clusters #4406

Are you sure you want to change the base?

[Test][HistoryServer] E2E test for live clusters #4406

Conversation

win5923 commented Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why are these changes needed?

Related issue number

Checks

Uh oh!

win5923 Jan 17, 2026

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

win5923 Jan 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

Uh oh!

win5923 Jan 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

win5923 Jan 21, 2026

Choose a reason for hiding this comment

Uh oh!

cursor bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

win5923 commented Jan 17, 2026 •

edited

Loading

win5923 Jan 17, 2026 •

edited

Loading

win5923 Jan 18, 2026 •

edited

Loading