Deploy test on theta

[wilke]

Setup psij testing suite to run daily, either as cron job or via gitlab.

[wilke]

Status: Failing

Steps:

1. loading python3.9
2. creating venv
3. installing requirements.txt and requirements-tests.txt
4. ./psij-ci-run

ModuleNotFoundError: No module named 'requests'

[wilke]

Status: Failing

manually installed: python -m pip install requests
./psij-ci-run:

./psij-ci-run
From https://github.com/ExaWorks/psi-j-python

• branch HEAD -> FETCH_HEAD
  Reading configuration... OK
  Cloning repository... OK
  Testing branch "main"... psij.venv/bin/python: No module named pytest
  OK
  Testing branch "combine-getting-started-quickstart-pages"... psij.venv/bin/python: No module named pytest
  OK
  Testing branch "wilke/logging"... psij.venv/bin/python: No module named pytest
  OK
  Testing branch "1.0.0-a0_release_branch"... FAILED

[hategan]

Did you follow the official instructions (README-testing.md)?

If you did, it would probably have told you to either run psij-ci-setup, which installs all needed dependencies, or to use requirements-dev.txt which does the same. Can you try that and submit an issue if that fails?

[hategan]

PS: requests is needed by Sphinx, which is listed in requirements-docs.txt, which is included from requirements-dev.txt.

It should probably be listed explicitly.

[wilke]

Seems to work but fails with the error below. Probably a problem in the branch but need to check.

Testing branch "wilke/logging"... ERROR: usage: __main__.py [options] [file_or_dir] [file_or_dir] [...]
__main__.py: error: unrecognized arguments: --minimal-uploads
  inifile: None
  rootdir: /tmp/tmpjxup64ta/code

OK
Testing branch "1.0.0-a0_release_branch"... FAILED
Traceback (most recent call last):
  File "/lus/theta-fs0/projects/CSC249ADTR01/psi-j-python/tests/ci_runner.py", line 195, in <module>
    run_tests(conf, site_ids, branches, clone)
  File "/lus/theta-fs0/projects/CSC249ADTR01/psi-j-python/tests/ci_runner.py", line 156, in run_tests
    run_branch_tests(conf, tmpp, run_id, clone, site_id, branch)
  File "/lus/theta-fs0/projects/CSC249ADTR01/psi-j-python/tests/ci_runner.py", line 135, in run_branch_tests
    subprocess.run(args, cwd=cwd.resolve(), env=env)
  File "/opt/python/3.9.4.1/lib/python3.9/subprocess.py", line 505, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/opt/python/3.9.4.1/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/opt/python/3.9.4.1/lib/python3.9/subprocess.py", line 1821, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'tests/branch_test_wrapper.sh'```

[wilke]

Did you follow the official instructions (README-testing.md)?

If you did, it would probably have told you to either run psij-ci-setup, which installs all needed dependencies, or to use requirements-dev.txt which does the same. Can you try that and submit an issue if that fails?

I did run psij-ci-setup but can do it again on a clean environment. On a side note, the setup script is not suitable for Jenkins or gitlab workflows due to it's interactive nature right now.

[hategan]

On a side note, the setup script is not suitable for Jenkins or gitlab workflows due to it's interactive nature right now.

It is not meant for CI workflows.

[hategan]

Testing branch "1.0.0-a0_release_branch"... FAILED

Yeah, branches created before minimal uploads were merged won't work with minimal uploads enabled. Do you actually need that on theta?

[wilke]

Testing branch "1.0.0-a0_release_branch"... FAILED

Yeah, branches created before minimal uploads were merged won't work with minimal uploads enabled. Do you actually need that on theta?

No

[wilke]

Manual test run worked.
Status: Success

Next step: Setting up gitlab workflow

[wilke]

Running as cronjob from project directory