Unable to run more than 1 tests set in a row on self-hosted runner

[KiritoMC03]

Hi! And thanks for your job!

Bug description
How to reproduce

I have a self-hosted Ubuntu (server) runner that I'm trying to run tests of my project on. If I run the runner for the first time after installation, the tests run correctly. However, when I try to run them a second time, I get an error in the preparation phase of the runner. Deleting the _work folder helps me, but it doesn't look like a fix for the problem. The problem is reproduced if I stop and start the runner.

• Expected behavior

I'm expected to be able to run a lot of tests without errors or having to clean up the runner, as it works with github owned runners.

Additional details

Logs:

2025-04-16T09:02:25.7565996Z Current runner version: '2.323.0'
2025-04-16T09:02:25.7577064Z Runner name: 'custom-ubuntu01'
2025-04-16T09:02:25.7578641Z Runner group name: 'Default'
2025-04-16T09:02:25.7580304Z Machine name: 'secret-ubuntu-name'
2025-04-16T09:02:25.7591918Z ##[group]GITHUB_TOKEN Permissions
2025-04-16T09:02:25.7596460Z Actions: write
2025-04-16T09:02:25.7597872Z Attestations: write
2025-04-16T09:02:25.7599203Z Checks: write
2025-04-16T09:02:25.7600365Z Contents: write
2025-04-16T09:02:25.7601523Z Deployments: write
2025-04-16T09:02:25.7602728Z Discussions: write
2025-04-16T09:02:25.7604055Z Issues: write
2025-04-16T09:02:25.7605233Z Metadata: read
2025-04-16T09:02:25.7606346Z Models: read
2025-04-16T09:02:25.7607433Z Packages: write
2025-04-16T09:02:25.7609271Z Pages: write
2025-04-16T09:02:25.7610413Z PullRequests: write
2025-04-16T09:02:25.7611709Z RepositoryProjects: write
2025-04-16T09:02:25.7612920Z SecurityEvents: write
2025-04-16T09:02:25.7614154Z Statuses: write
2025-04-16T09:02:25.7615652Z ##[endgroup]
2025-04-16T09:02:25.7619336Z Secret source: Actions
2025-04-16T09:02:25.7620724Z Prepare workflow directory
2025-04-16T09:02:25.8253398Z ##[error]One or more errors occurred. (Access to the path '/home/ubuntu/actions-runner/_work/_actions/game-ci/unity-test-runner/v4/dist/BlankProject/Temp/workerlic' is denied.) (Access to the path '/home/ubuntu/actions-runner/_work/_actions/game-ci/unity-test-runner/v4/dist/BlankProject/Temp/UnityLockfile' is denied.)

Workflow:

name: Run Unit Tests

on: [pull_request, workflow_dispatch]

jobs:
  test:
    runs-on: custom-ubuntu01
    strategy:
      fail-fast: false
    steps:      
      - name: Checkout repository
        uses: actions/checkout@v4
        with:
          submodules: recursive
          token: ${{ secrets.UNITY_GIT_ACCESS_TOKEN }}
          lfs: true

      - name: Run unit tests
        uses: game-ci/unity-test-runner@v4
        id: legacy_tests
        continue-on-error: true
        env:
          UNITY_SERIAL: ${{ secrets.unity_serial }}
          UNITY_EMAIL: ${{ secrets.unity_email }}
          UNITY_PASSWORD: ${{ secrets.unity_password }}
        with:
          projectPath: "sample"
          unityVersion: "auto"
          testMode: "EditMode"

      - name: Upload tests result
        uses: actions/upload-artifact@v4
        if: always()
        with:
          name: Test results
          path: artifacts

[GabLeRoux]

👋 Hey thanks for the kind words :)

Based on your description and the error logs, the problem seems to be a file locking or permissions issue related to leftover files in the Temp/ directory of your Unity project after a previous test run.

I am not using a similar setup at this time so I cannot reproduce, but maybe you can try adding a step before the tests that will clean the Temp/ folder. That would still be a workaround as I don't think this should be necessary, but it's worth trying.

- name: Clean Unity Temp files
  run: |
    echo "Cleaning Temp directory..."
    rm -rf sample/Temp/

But this might also fail with permission issues if the runner doesn't have the rights on the generated files.

I suggest exploring the files and folders after a first run to see what are the permissions and ownership of the files.

ls -la /home/ubuntu/actions-runner/_work/_actions/game-ci/unity-test-runner/v4/dist/BlankProject/

This should show the owner and the permissions for that folder. I would expect to see the same user as the one used by the runner for it to succeed so if it's owned by a different user (ex; root), we'd have to figure out why this happens.

[KiritoMC03]

Thanks for the detailed reply!
I've done what you suggested and I see that rm -rf sample/Temp/ fails because of the permission level

rm -rf sample/Temp/ logs:

rm: cannot remove 'sample/Temp/UnityLockfile': Permission denied
rm: cannot remove 'sample/Temp/workerlic': Permission denied
rm: cannot remove 'sample/Temp/FSTimeGet-c2ccbad8196f50b62aec112ea846a996': Permission denied

---

And I see that the Library, Logs and UserSettings folders are managed by the root user.
ls -la /home/ubuntu/actions-runner/_work/_actions/game-ci/unity-test-runner/v4/dist/BlankProject/ logs:

drwxrwxr-x 8 ubuntu ubuntu 4096 Apr 17 08:45 .
drwxrwxr-x 6 ubuntu ubuntu 4096 Apr 17 08:37 ..
-rw-rw-r-- 1 ubuntu ubuntu  619 Jun 15  2024 .gitignore
drwxrwxr-x 3 ubuntu ubuntu 4096 Apr 17 08:37 Assets
drwxr-xr-x 9 root   root   4096 Apr 17 08:45 Library
drwxr-xr-x 2 root   root   4096 Apr 17 08:45 Logs
drwxrwxr-x 2 ubuntu ubuntu 4096 Apr 17 08:38 Packages
drwxrwxr-x 2 ubuntu ubuntu 4096 Apr 17 08:38 ProjectSettings
drwxr-xr-x 2 root   root   4096 Apr 17 08:38 UserSettings

[webbertakken]

The docker image runs as root, so the files are owned by root.

You could try using ˋsudoˋ to do the removal.

[KiritoMC03]

Yes, I can sudo rm..., but do I want to? I mean, it doesn't seem like the right solution to the problem. It seems to me that my workflow file should not be different on self-hosted runners and github runners, and we should try to understand and solve the cause of this behavior on self-hosted runners
I don't know what mode and permissions the default github runners are running in, I know I can't call sudo ./run.sh and I think that's correct

[mfbrantner]

I might be wrong, but isn't this what runAsHostUser is for?
(unity-builder also supports this configuration option)

[webbertakken]

Yes, I can sudo rm..., but do I want to? I mean, it doesn't seem like the right solution to the problem. It seems to me that my workflow file should not be different on self-hosted runners and github runners, and we should try to understand and solve the cause of this behavior on self-hosted runners I don't know what mode and permissions the default github runners are running in, I know I can't call sudo ./run.sh and I think that's correct

Sure. And the correct solution and the difference between self-hosted and GH hosted runners has been widely discussed in previous issues and on our Discord (especially in the context the user).

I might be wrong, but isn't this what runAsHostUser is for?
(unity-builder also supports this configuration option)

That's right. And it looks like that change has never been ported to test runner.