Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

regtests: mitigate file permissions issue with podman compose #205

Merged
merged 1 commit into from
Aug 26, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
regtests: mitigate file permissions issue with podman compose
This change works around a file-permissions issue when running regtests via `podman compose up --build --exit-code-from regtest` (not Docker). CI is happy without this change.

Generally speaking, a `chmod -R go+rwx` is not good, however, it is only for tests running in a container.

```
[regtest] | t_cli/src/test_cli.py ....                                               [ 94%]
[regtest] | t_pyspark/src/test_spark_sql_s3_with_privileges.py ssssssssssss          [100%]
[regtest] |
[regtest] | =============================== warnings summary ===============================
[regtest] | ../polaris-venv/lib/python3.8/site-packages/_pytest/cacheprovider.py:479
[regtest] |   /home/spark/polaris-venv/lib/python3.8/site-packages/_pytest/cacheprovider.py:479: PytestCacheWarning: could not create cache path /home/spark/regtests/.pytest_cache/v/cache/nodeids: [Errno 13] Permission denied: '/home/spark/regtests/pytest-cache-files-nm7of75m'
[regtest] |     config.cache.set("cache/nodeids", sorted(self.cached_nodeids))
[regtest] |
[regtest] | ../polaris-venv/lib/python3.8/site-packages/_pytest/stepwise.py:57
[regtest] |   /home/spark/polaris-venv/lib/python3.8/site-packages/_pytest/stepwise.py:57: PytestCacheWarning: could not create cache path /home/spark/regtests/.pytest_cache/v/cache/stepwise: [Errno 13] Permission denied: '/home/spark/regtests/pytest-cache-files-gxv4jemz'
[regtest] |     session.config.cache.set(STEPWISE_CACHE_DIR, [])
[regtest] |
[regtest] | -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
[regtest] | =========== 218 passed, 12 skipped, 2 warnings in 139.01s (0:02:19) ============
[regtest] | Wed 21 Aug 2024 11:21:46 AM UTC: Test SUCCEEDED: t_cli:test_cli.py
[regtest] | Wed 21 Aug 2024 11:21:46 AM UTC: Starting test t_hello_world:hello_world.sh
mkdir: cannot create directory ‘/tmp/polaris-regtests/t_hello_world’: Permission denied
/home/spark/regtests/run.sh: line 121: /tmp/polaris-regtests/t_hello_world/hello_world.sh.stderr: No such file or directory
/home/spark/regtests/run.sh: line 121: /tmp/polaris-regtests/t_hello_world/hello_world.sh.stdout: No such file or directory
[regtest] | Wed 21 Aug 2024 11:21:46 AM UTC: Test run concluded for t_hello_world:hello_world.sh
/home/spark/regtests/run.sh: line 132: /tmp/polaris-regtests/t_hello_world/hello_world.sh.fixdiffs.sh: No such file or directory
[regtest] | Wed 21 Aug 2024 11:21:46 AM UTC: Test FAILED: t_hello_world:hello_world.sh
/home/spark/regtests/run.sh: line 133: /tmp/polaris-regtests/t_hello_world/hello_world.sh.fixdiffs.sh: No such file or directory
chmod: cannot access '/tmp/polaris-regtests/t_hello_world/hello_world.sh.fixdiffs.sh': No such file or directory
[regtest] | Wed 21 Aug 2024 11:21:46 AM UTC: To compare and fix diffs (if 'meld' installed): /tmp/polaris-regtests/t_hello_world/hello_world.sh.fixdiffs.sh
[regtest] | Wed 21 Aug 2024 11:21:46 AM UTC: Or manually diff: diff /tmp/polaris-regtests/t_hello_world/hello_world.sh.stdout /home/spark/regtests/t_hello_world/ref/hello_world.sh.ref
[regtest] | Wed 21 Aug 2024 11:21:46 AM UTC: See stderr from test run for additional diagnostics: /tmp/polaris-regtests/t_hello_world/hello_world.sh.stderr
diff: /tmp/polaris-regtests/t_hello_world/hello_world.sh.stdout: No such file or directory
[regtest] | Wed 21 Aug 2024 11:21:46 AM UTC: Starting pytest t_pyspark:test_spark_sql_s3_with_privileges.py
[regtest] | ============================= test session starts ==============================
[regtest] | platform linux -- Python 3.8.10, pytest-8.2.2, pluggy-1.5.0
[regtest] | rootdir: /home/spark/regtests
```
  • Loading branch information
snazy committed Aug 24, 2024
commit d90591803149681a16aca68b64fa06180f77d911
5 changes: 4 additions & 1 deletion regtests/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,7 @@ RUN ./setup.sh

COPY --chown=spark . /home/spark/regtests

CMD ["./run.sh"]
# /home/spark/regtests might not be writable in all situations, see https://github.com/apache/polaris/pull/205
RUN chmod -R go+rwx /home/spark/regtests

CMD ["./run.sh"]