[CI][benchmarks] Modify benchmark scripts and CI workflow to enable gdb_mode #20891

luszczewskakasia1 · 2025-12-15T09:00:11Z

Modify benchmark scripts and CI workflow to enable gdb_mode on workflow_dispatch while deugging benchmarks.

lukaszstolarczuk · 2025-12-15T13:09:43Z

.github/workflows/sycl-linux-run-tests.yml

        default: ''
+      gdb_mode:
+        type: string
+        required: False


pls add default = false

lukaszstolarczuk · 2025-12-15T13:16:13Z

.github/workflows/sycl-linux-run-tests.yml

          Extra options to be added to LIT_OPTS.
        type: string
        default: ''
+      gdb_mode:


pls move below and add to the name prefix benchmark_

lukaszstolarczuk · 2025-12-15T13:17:56Z

devops/actions/run-tests/benchmark/action.yml

      echo "::endgroup::"
      echo "::group::run_benchmarks"

+      export LLVM_BENCHMARKS_USE_GDB=${LLVM_BENCHMARKS_USE_GDB}


when is LLVM_BENCHMARKS_USE_GDB set?

Must have deleted it accidentally. Now it is added

lukaszstolarczuk · 2025-12-15T13:20:16Z

devops/scripts/benchmarks/benches/compute.py

+        if is_gdb_benchmarks:
+            return [(0.0, 0.0)]
+
+        else:


no need for else, since you returned on the if (this way we don't need extra tab for the rest of the code

lukaszstolarczuk · 2025-12-15T13:22:48Z

devops/scripts/benchmarks/benches/compute.py

            f"-DUSE_SYSTEM_LEVEL_ZERO=OFF",
            f"-DCMAKE_CXX_COMPILER=clang++",
            f"-DCMAKE_C_COMPILER=clang",
+            f"-DCMAKE_CXX_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG -fdebug-info-for-profiling",


perhaps we should only add these when GDB is enabled...?

lukaszstolarczuk · 2025-12-15T13:24:09Z

devops/scripts/benchmarks/utils/utils.py

            command = command.split()

+        is_gdb_benchmarks = os.environ.get("LLVM_BENCHMARKS_USE_GDB", "") == "true"
+        if command[0] == "taskset" and is_gdb_benchmarks:


why do we check if taskset is the first part of the command?

I did it so gdb is used only for commands which are responsible for running benchmarks as it looks like that 'taskset', '-c', '0,1,2,3', '/tmp/tmp7nhle4ge/compute-benchmarks-build/bin/torch_benchmark_l0', '--test=KernelSubmitMultiQueue', '--csv', '--noHeaders', '--iterations=3', '--workgroupCount=4096', '--workgroupSize=512', '--kernelsPerQueue=20'
I don't think we need gdb for any other command which are executed in out benchmarks as this extends time of execution of the script.

can it be not the first one? e.g. in command like PATH=<...> taskset <...>? just asking, dunno.

In base.py the method taskset_cmd says, that the benchmark command starts with taskset and then it is used in run_bench method

PatKamin · 2025-12-16T09:17:19Z

devops/scripts/benchmarks/benches/compute.py

            f"-DCMAKE_CXX_COMPILER=clang++",
            f"-DCMAKE_C_COMPILER=clang",
+            f"-DCMAKE_CXX_FLAGS_RELWITHDEBINFO:STRING=-O2 -g -DNDEBUG -fdebug-info-for-profiling",
+            f"-DCMAKE_BUILD_TYPE=RelWithDebInfo",


Modify the GitProject class, there is a default Release version set for all projects.

luszczewskakasia1 · 2025-12-17T21:36:01Z

https://github.com/intel/llvm/actions/runs/20301652593/job/58309783774 - results with gdb enabled

…ugging benchmarks

PatKamin · 2025-12-18T08:07:05Z

https://github.com/intel/llvm/actions/runs/20301652593/job/58309783774 - results with gdb enabled

Please, fix the safe-path error. It looks like gdb refuses to run test binaries.

PatKamin · 2025-12-18T08:18:45Z

devops/scripts/benchmarks/utils/utils.py

            command = command.split()

+        is_gdb_mode = os.environ.get("LLVM_BENCHMARKS_USE_GDB", "") == "true"
+        if command[0] == "taskset" and is_gdb_mode:


I don't understand why taskset is crucial here? We shouldn't rely on taskset being used in benchmark commands (if this is the case), this could be changed in future.

If needed, perhaps add some wrapper function that modifies benchmarks command (or passes it through down here).

As I understand this check is to distinguish if we run a benchmark or some other command. I think we can (instead of looking for taskset) just check if a binary from .../compute-benchmarks-build/bin/ is run.

PatKamin · 2025-12-18T08:21:49Z

devops/scripts/benchmarks/benches/compute.py

                "compute-benchmarks",
                use_installdir=False,
            )
+        print(self._project)


Are these some debug prints? Why logger is not used?

luszczewskakasia1 force-pushed the luszczewskakasia1_add-gdb branch from 56320bf to 9de028f Compare December 15, 2025 09:12

luszczewskakasia1 had a problem deploying to WindowsCILock December 15, 2025 09:12 — with GitHub Actions Error

luszczewskakasia1 force-pushed the luszczewskakasia1_add-gdb branch from 9de028f to 42f6390 Compare December 15, 2025 09:18

luszczewskakasia1 had a problem deploying to WindowsCILock December 15, 2025 09:18 — with GitHub Actions Error

luszczewskakasia1 force-pushed the luszczewskakasia1_add-gdb branch from 42f6390 to e179b9f Compare December 15, 2025 09:20

luszczewskakasia1 temporarily deployed to WindowsCILock December 15, 2025 09:20 — with GitHub Actions Inactive

luszczewskakasia1 had a problem deploying to WindowsCILock December 15, 2025 09:51 — with GitHub Actions Error

luszczewskakasia1 force-pushed the luszczewskakasia1_add-gdb branch from e179b9f to 1cae956 Compare December 15, 2025 10:08

luszczewskakasia1 temporarily deployed to WindowsCILock December 15, 2025 10:08 — with GitHub Actions Inactive

luszczewskakasia1 temporarily deployed to WindowsCILock December 15, 2025 10:40 — with GitHub Actions Inactive

luszczewskakasia1 force-pushed the luszczewskakasia1_add-gdb branch from 1cae956 to a4c9fed Compare December 15, 2025 11:32

luszczewskakasia1 had a problem deploying to WindowsCILock December 15, 2025 11:32 — with GitHub Actions Error

luszczewskakasia1 force-pushed the luszczewskakasia1_add-gdb branch from a4c9fed to c2805eb Compare December 15, 2025 11:59

luszczewskakasia1 temporarily deployed to WindowsCILock December 15, 2025 12:00 — with GitHub Actions Inactive

luszczewskakasia1 had a problem deploying to WindowsCILock December 15, 2025 12:31 — with GitHub Actions Failure

luszczewskakasia1 marked this pull request as ready for review December 15, 2025 12:58

luszczewskakasia1 requested review from a team as code owners December 15, 2025 12:58

lukaszstolarczuk requested changes Dec 15, 2025

View reviewed changes

PatKamin reviewed Dec 16, 2025

View reviewed changes

luszczewskakasia1 force-pushed the luszczewskakasia1_add-gdb branch from c2805eb to 4fcea5d Compare December 17, 2025 10:28

luszczewskakasia1 had a problem deploying to WindowsCILock December 17, 2025 10:28 — with GitHub Actions Error

luszczewskakasia1 force-pushed the luszczewskakasia1_add-gdb branch from 4fcea5d to e0788dd Compare December 17, 2025 10:38

luszczewskakasia1 had a problem deploying to WindowsCILock December 17, 2025 10:39 — with GitHub Actions Error

luszczewskakasia1 force-pushed the luszczewskakasia1_add-gdb branch from e0788dd to 5d275a1 Compare December 17, 2025 10:41

luszczewskakasia1 had a problem deploying to WindowsCILock December 17, 2025 10:42 — with GitHub Actions Error

luszczewskakasia1 force-pushed the luszczewskakasia1_add-gdb branch from 5d275a1 to b5986cf Compare December 17, 2025 10:46

luszczewskakasia1 had a problem deploying to WindowsCILock December 17, 2025 10:47 — with GitHub Actions Error

luszczewskakasia1 force-pushed the luszczewskakasia1_add-gdb branch from b5986cf to a90fc02 Compare December 17, 2025 10:51

luszczewskakasia1 had a problem deploying to WindowsCILock December 17, 2025 10:52 — with GitHub Actions Error

luszczewskakasia1 force-pushed the luszczewskakasia1_add-gdb branch from a90fc02 to 113f984 Compare December 17, 2025 11:02

luszczewskakasia1 temporarily deployed to WindowsCILock December 17, 2025 11:03 — with GitHub Actions Inactive

luszczewskakasia1 temporarily deployed to WindowsCILock December 17, 2025 11:31 — with GitHub Actions Inactive

luszczewskakasia1 force-pushed the luszczewskakasia1_add-gdb branch 2 times, most recently from 1897a8b to 317c411 Compare December 17, 2025 21:45

luszczewskakasia1 temporarily deployed to WindowsCILock December 17, 2025 21:45 — with GitHub Actions Inactive

luszczewskakasia1 temporarily deployed to WindowsCILock December 17, 2025 22:45 — with GitHub Actions Inactive

Modify benchmark scripts and CI workflow to enable gdb_mode while deb…

317c411

…ugging benchmarks

PatKamin reviewed Dec 18, 2025

View reviewed changes

[CI][benchmarks] Modify benchmark scripts and CI workflow to enable gdb_mode #20891

Are you sure you want to change the base?

[CI][benchmarks] Modify benchmark scripts and CI workflow to enable gdb_mode #20891

Conversation

luszczewskakasia1 commented Dec 15, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

luszczewskakasia1 commented Dec 17, 2025

Uh oh!

PatKamin commented Dec 18, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants