API: Workflow list displays different workflows when capped at different numbers

[ido-greenfeld]

Pre-requisites

• I have double-checked my configuration
• I have tested with the :latest image tag (i.e. quay.io/argoproj/workflow-controller:latest) and can confirm the issue still exists on :latest. If not, I have explained why, in detail, in my description below.
• I have searched existing issues and could not find a match for this bug
• I'd like to contribute the fix myself (see contributing guide)

What happened? What did you expect to happen?

Hi,

I'm new to Argo Workflows and conducting some tests to evaluate if it fits our needs.
In one test, I ran 100 simple workflows in parallel (each workflow just sleeps for 10 minutes).
I also applied a semaphore limit, allowing only 60 workflows to run concurrently, with the rest staying in a pending state.

The issue I'm encountering is with how the UI displays the workflows when I cap the number of displayed workflows at different values.
The ordering seems inconsistent and changes based on the cap:

• When capped at 10 workflows, I see 3 pending at the top and 7 running below.
• When capped at 50 workflows, I still see only 3 pending at the top (after some delay) and 47 running below.
• When capped at 100 or more workflows, I finally see the expected 40 pending at the top and 60 running below.

I would expect the UI to always display the 40 pending workflows at the top, followed by the running workflows, but this doesn’t seem to happen consistently under different caps.

This issue isn't limited to pending workflows; I've also seen it with failed workflows, and it may affect running workflows too, though it's harder to detect.

Is this behavior expected, or is there something I might be missing? I've attached a video that further explains the issue.

Recording.2024-09-23.170417.1.1.mp4

Version(s)

v3.5.10

Paste a minimal workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

N/A

Logs from the workflow controller

N/A

Logs from in your workflow's wait container

N/A

[agilgur5]

Is this behavior expected, or is there something I might be missing?

It is confusing, but expected, yes. See #3157 (comment) where I linked out to other old issues, including upstream k8s:

Currently, sorting is pretty limited due to the k8s API and etcd -- see #4546 (comment), #3565 (comment), #2926, which all link to kubernetes/kubernetes#80602.

Per the line below that, #3157 is now possible after #12736. That feature request will allow more sorting