Add gitlab-runner-scaler by fira42073 · Pull Request #6499 · kedacore/keda

fira42073 · 2025-01-25T21:23:14Z

Provide a description of what has been changed

Checklist

When introducing a new scaler, I agree with the scaling governance policy
I have verified that my change is according to the deprecations & breaking changes policy
Tests have been added
Changelog has been updated and is aligned with our changelog requirements
A PR is opened to update our Helm chart (repo) (if applicable, ie. when deployment manifests are modified)
A PR is opened to update the documentation on (repo) (if applicable)
Commits are signed with Developer Certificate of Origin (DCO - learn more)

Signed-off-by: Fira Curie <fira42073@gmail.com>

JorTurFer · 2025-02-03T21:20:13Z

+	meta := &gitlabRunnerMetadata{
+		ProjectID:                 "12345",
+		TargetPipelineQueueLength: 5,
+		TriggerIndex:              0,


It's something quite small but, could we use TriggerIndex: 1 ? 0 is the default value so just using 1 we can also check that the TriggerIndex is correctly reflected in the metric name

JorTurFer · 2025-02-03T21:28:45Z

@@ -0,0 +1,503 @@
+//go:build e2e


Reading the e2e test, it waits a PAT so I guess that we need to create an e2e account to run it. Does it have any special requirement? Does it make sense to use gitlab.com instead of spin up a dummy gitlab instance in the cluster during the e2e test? https://docs.gitlab.com/charts/installation/deployment.html

I actually already tried doing that, and it's a bit too complicated
#5616 (comment)

I think it's best to create a gitlab.com account, since it would only require phone verification to run pipelines, and that's it. Repos will be cleaned up automatically as well as part of e2e tests.

JorTurFer · 2025-02-03T21:31:33Z

+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{.DeploymentName}}
+  namespace: {{.TestNamespace}}
+  labels:
+    app: gitlab-runner
+spec:
+  replicas: 0
+  selector:
+    matchLabels:
+      app: gitlab-runner
+  template:
+    metadata:
+      labels:
+        app: gitlab-runner
+    spec:
+      terminationGracePeriodSeconds: 90
+      containers:
+      - name: gitlab-runner
+        image: busybox
+        command: ["/bin/sh", "-c", "trap 'echo SIGTERM received; exit 0' SIGTERM; while true; do sleep 30; done"]


Does it make sense creating a real runner using their runner image to check that the job is dequeued? https://hub.docker.com/r/gitlab/gitlab-runner/tags

I sat down and tried doing that this evening, but then realized that we actually already kind of had this discussion here.

Gitlab already does have a mechanism of autoscaling (creating runner per job)

don't worry about the token in the image, it's been rotated already.

So I think doesn't make sense to manage that using keda. Again, please correct me if I'm wrong, maybe I simply don't quite understand the usecase here.

JorTurFer · 2025-02-03T21:32:59Z

+	testSONotActivated(t, kc)
+
+	testSOScaleOut(t, kc, projectID)
+	testSOScaleIn(t, kc, projectID)


Let's use the same name as the majority of the tests:

Suggested change

testSONotActivated(t, kc)

testSOScaleOut(t, kc, projectID)

testSOScaleIn(t, kc, projectID)

testActivation(t, kc)

testScaleOut(t, kc, projectID)

testScaleIn(t, kc, projectID)

JorTurFer · 2025-02-03T21:35:08Z

+
+	metric := GenerateMetricInMili(metricName, float64(queueLen))
+
+	return []external_metrics.ExternalMetricValue{metric}, queueLen >= s.metadata.TargetPipelineQueueLength, nil


TargetPipelineQueueLength shouldn't be used here. It is used for the target, for the activation we need to include another parameter ActivationTargetPipelineQueueLength with default value 0 and use it with >

return []external_metrics.ExternalMetricValue{metric}, queueLen > s.metadata.ActivationTargetPipelineQueueLength, nil

JorTurFer · 2025-02-03T21:35:50Z

+      gitlabAPIURL: {{.GitlabAPIURL}}
+      projectID: "{{.ProjectID}}"
+      targetPipelineQueueLength: "{{.TargetPipelineQueueLength}}"
+      labels: "e2eSOtester"


The activation parameter is left here for the activation test

JorTurFer · 2025-02-03T21:36:45Z

+
+func testSONotActivated(t *testing.T, kc *kubernetes.Clientset) {


During the activation test, we have to add an item in the queue and check that the activation value blocks the scaling until the item count in the queue is higher

zroubalik · 2025-02-06T10:47:03Z

 }

+// setConfigValueURL is a function that sets the value of the url.URL field
+func setConfigValueURL(valFromConfig string, field reflect.Value) error {


WDYT @wozniakjan ?

Signed-off-by: Fira Curie <fira42073@gmail.com>

zroubalik · 2025-03-26T19:15:44Z

/run-e2e gitlab
Update: You can check the progress here

fira42073 · 2025-03-26T19:59:30Z

We need gitlab PAT for this to work, have you had a chance to add it?

zroubalik · 2025-03-26T20:30:48Z

We need gitlab PAT for this to work, have you had a chance to add it?

I see, nope. We need to work on that, adding this #6499 (comment) for reference, thanks for letting us know!

stale · 2025-05-25T21:25:27Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

JorTurFer · 2025-05-27T21:06:37Z

Do we need to create a gitlab account for this? I can create an account for KEDA if needed

fira42073 · 2025-05-28T04:04:15Z

Hi, @JorTurFer! We do indeed need a gitlab account. We would also need to verify the phone number to use the shared runners. I guess gitlab asks for verification to prevent spam of its resources.

JorTurFer · 2025-06-01T16:42:25Z

Is it feasible to use Gitlab directly in cluster? we can install it as part of e2e test using the helm chart -> https://docs.gitlab.com/charts/ (or maybe it's not). If the Gitlab account is needed in any case, I'll create it

stale · 2025-07-31T18:00:39Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

zroubalik · 2025-08-19T09:58:02Z

@fira42073 any update here?

fira42073 · 2025-08-19T16:21:51Z

Hi! Not really. I think we have already discussed in the issue the possibility of selfhosting gitlab. While it is possible, I assume it might be too cumbersome to do so. See #5616 (comment)

If you have a better idea, I'm open to suggestions.

stale · 2025-10-18T17:20:07Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

stale · 2025-10-26T01:50:05Z

This issue has been automatically closed due to inactivity.

fira42073 requested a review from a team as a code owner January 25, 2025 21:23

fira42073 force-pushed the gitlab-runner-scaler branch 2 times, most recently from 1abaa5f to 7af19df Compare January 25, 2025 21:25

fira42073 mentioned this pull request Jan 25, 2025

GitLab Runner Scaler #5616

Closed

fira42073 force-pushed the gitlab-runner-scaler branch from 7af19df to b58aaae Compare January 25, 2025 21:42

Add gitlab-runner-scaler

b650bc3

Signed-off-by: Fira Curie <fira42073@gmail.com>

fira42073 force-pushed the gitlab-runner-scaler branch from b58aaae to b650bc3 Compare January 25, 2025 22:02

JorTurFer reviewed Feb 3, 2025

View reviewed changes

zroubalik reviewed Feb 6, 2025

View reviewed changes

fira42073 and others added 2 commits March 4, 2025 21:16

Merge branch 'kedacore:main' into gitlab-runner-scaler

9503900

apply code review suggestions

a2ccae7

Signed-off-by: Fira Curie <fira42073@gmail.com>

fira42073 force-pushed the gitlab-runner-scaler branch from 9f9ad27 to a2ccae7 Compare March 4, 2025 22:03

make use of activation target

f92b18c

Signed-off-by: Fira Curie <fira42073@gmail.com>

fira42073 force-pushed the gitlab-runner-scaler branch from 3fc095e to f92b18c Compare March 4, 2025 22:26

fira42073 requested review from JorTurFer and zroubalik March 4, 2025 22:26

zroubalik mentioned this pull request Mar 26, 2025

Release: 2.17.0 #6502

Closed

40 tasks

stale Bot added the stale All issues that are marked as stale due to inactivity label May 25, 2025

stale Bot removed the stale All issues that are marked as stale due to inactivity label May 27, 2025

stale Bot added the stale All issues that are marked as stale due to inactivity label Jul 31, 2025

rickbrouwer added the waiting-author-response All PR's or Issues where we are waiting for a response from the author label Aug 6, 2025

stale Bot removed the stale All issues that are marked as stale due to inactivity label Aug 6, 2025

stale Bot added the stale All issues that are marked as stale due to inactivity label Oct 18, 2025

stale Bot closed this Oct 26, 2025

This was referenced Feb 27, 2026

🌱 CNCF mission generation 2026-02-27 kubestellar/console-kb#6

Closed

🌱 CNCF mission generation 2026-02-27 kubestellar/console-kb#11

Merged


		metric := GenerateMetricInMili(metricName, float64(queueLen))

		return []external_metrics.ExternalMetricValue{metric}, queueLen >= s.metadata.TargetPipelineQueueLength, nil


		func testSONotActivated(t testing.T, kc kubernetes.Clientset) {

Conversation

fira42073 commented Jan 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zroubalik commented Mar 26, 2025 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fira42073 commented Mar 26, 2025

Uh oh!

zroubalik commented Mar 26, 2025

Uh oh!

stale Bot commented May 25, 2025

Uh oh!

JorTurFer commented May 27, 2025

Uh oh!

fira42073 commented May 28, 2025

Uh oh!

JorTurFer commented Jun 1, 2025

Uh oh!

stale Bot commented Jul 31, 2025

Uh oh!

zroubalik commented Aug 19, 2025

Uh oh!

fira42073 commented Aug 19, 2025

Uh oh!

stale Bot commented Oct 18, 2025

Uh oh!

stale Bot commented Oct 26, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

fira42073 commented Jan 25, 2025 •

edited

Loading

zroubalik commented Mar 26, 2025 •

edited by github-actions Bot

Loading