Python Libraries Error

[leonardt]

Trying to write a simple test using the fault interface and running into what looks to be an environment error.

Have you seen this error before?

build/Queue.json
-- pybind11 v2.6.2 dev1
-- Configuring done
-- Generating done
-- Build files have been written to: /home/lenny/repos/mantle2/build/obj_dir/build
[100%] Built target libpysv
Running command: ./obj_dir/VQueue
<STDERR>
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Python path configuration:
  PYTHONHOME = (not set)
  PYTHONPATH = (not set)
  program name = 'python3'
  isolated = 0
  environment = 1
  user site = 1
  import site = 1
  sys._base_executable = ''
  sys.base_prefix = '/tmp/build/80754af9/python_1599203911753/_h_env_placehold_placehold_placehold_place
hold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold
_placehold_placehold_placehold_placehold_placehold_placehold_placeho'
  sys.base_exec_prefix = '/tmp/build/80754af9/python_1599203911753/_h_env_placehold_placehold_placehold_
placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac
ehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho'
  sys.executable = ''
  sys.prefix = '/tmp/build/80754af9/python_1599203911753/_h_env_placehold_placehold_placehold_placehold_
placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac
ehold_placehold_placehold_placehold_placehold_placehold_placeho'
  sys.exec_prefix = '/tmp/build/80754af9/python_1599203911753/_h_env_placehold_placehold_placehold_place
hold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold
_placehold_placehold_placehold_placehold_placehold_placehold_placeho'
  sys.path = [
    '/tmp/build/80754af9/python_1599203911753/_h_env_placehold_placehold_placehold_placehold_placehold_p
lacehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place
hold_placehold_placehold_placehold_placehold_placeho/lib/python38.zip',
    '/tmp/build/80754af9/python_1599203911753/_h_env_placehold_placehold_placehold_placehold_placehold_p
lacehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place
hold_placehold_placehold_placehold_placehold_placeho/lib/python3.8',
    '/tmp/build/80754af9/python_1599203911753/_h_env_placehold_placehold_placehold_placehold_placehold_p
lacehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place
hold_placehold_placehold_placehold_placehold_placeho/lib/lib-dynload',
  ]
Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding
Python runtime state: core initialized
ModuleNotFoundError: No module named 'encodings'

Current thread 0x00007f4079877740 (most recent call first):
<no Python frame>
</STDERR>

I'm suspecting this is some sort of environment issue, here are the steps if you'd like to try to reproduce

$ git clone -b queue-model https://github.com/leonardt/mantle2
$ pytest tests/test_queue.py::test_queue_model

The test is very similar to the fault test in the main test suite

I am using a miniconda environment (installed into $HOME/miniconda3), perhaps that's causing a problem with the python library integration with the pysv compile?

To sanity check, I just tried pytest tests/test_pysv.py in the fault repo and got the same error

[Kuree]

I was not able to reproduce this error with stock 3.8 but was able to reproduce inside the official moniconda docker image. From the error messages it seems to replated to some Pybind11 issues: pybind/pybind11#1930 and pybind/pybind11#2369.

The interesting thing is if you just run the VQueue directly, it runs without any issue (had to fix a minor bug in pysv, but it's not related).

So the issue only happens when using miniconda + pytest at the same time. Somehow setting PYTHONHOME will not get propagate inside the pytest tests, which might relate to the cause of the error.

I will keep poking and keep you updated.

[Kuree]

Found a solution that seems to work with miniconda+pytest. If PYTHONPATH and PYTHONHOME are hard-coded into the binary , the embedded interpreter can pick up Python modules properly.

[Kuree]

Should be fixed in the latest release. Feel free to reopen it if it doesn't work.

[leonardt]

Seems like it's working now