Mitogen fails on RHEL8 server with bash: /usr/bin/python: No such file or directory #673

rlinq · 2019-12-02T21:25:07Z

Incorrect identification of python interpreter on Red Hat Enterprise Linux 8.0. Cause playbook to fail instantly.

Hope this issue is easy to find and solve, this is a awesome plugin that speeds up our Ansible playbooks with 9x, absolutely mind blowing, can't live without it. Great work!

fatal: [rhel8-test]: UNREACHABLE! => changed=false
  msg: |-
    EOF on stream; last 100 lines received:
    bash: /usr/bin/python: No such file or directory
  unreachable: true

Which version of Ansible are you running?
2.8.5

ansible 2.8.5
  config file = /opt/tools/rlinq/dev/ansible/core/ansible.cfg
  configured module search path = [u'/export/home/rlinq/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/site-packages/ansible
  executable location = /bin/ansible
  python version = 2.7.5 (default, Jun 11 2019, 14:33:56) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]

Is your version of Ansible patched in any way?

No

Are you running with any custom modules, or module_utils loaded?

No

Have you tried the latest master version from Git?

No, but latest released version mitogen-0.2.9 is used in this issue report.
Reviewed diff between 0.2.9 and master found nothing related to python version detection.

How to reproduce the issue

Install a RHEL8 or possible a CENTOS8 server (haven't been able to test on CENTOS8 but should be the same)
Download mitogen 0.2.9 and extract that in a path of your choice. Configure this path in your ansible.cfg file in strategy_plugin. Set strategy = mitogen_linear

rlinq@rhel8-test $ grep mitogen ../ansible.cfg
strategy_plugins = /opt/tools/rlinq/tools/mitogen-0.2.9/ansible_mitogen/plugins/strategy
strategy = mitogen_linear

Create a simple ansible playbook that connects to your self using hostname.

rlinq@rhel8-test $ cat mitogen_test_rhel8.yml
- hosts: rhel8-test
  gather_facts: 'no'
  strategy: mitogen_linear
  pre_tasks:
    - setup:
        gather_subset: 'network'

rlinq@rhel8-test $ ansible-playbook mitogen_test_rhel8.yml -i hosts -vvv'

Result

rlinqmgr@rhel8-test: /.../rlinq_ansible/roles $ ansible-playbook mitogen_test_rhel8.yml -i ../environments/test/hosts -vvv
ansible-playbook 2.9.1
  config file = /opt/tools/rlinq/clones/rlinq_ansible/roles/ansible.cfg
  configured module search path = ['/export/home/rlinqmgr/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.6/site-packages/ansible
  executable location = /usr/local/bin/ansible-playbook
  python version = 3.6.8 (default, Jun 12 2019, 01:12:31) [GCC 8.2.1 20180905 (Red Hat 8.2.1-3)]
Using /opt/tools/rlinq/clones/rlinq_ansible/roles/ansible.cfg as config file
host_list declined parsing /opt/tools/rlinq/clones/rlinq_ansible/environments/test/hosts as it did not pass its verify_file() method
script declined parsing /opt/tools/rlinq/clones/rlinq_ansible/environments/test/hosts as it did not pass its verify_file() method
auto declined parsing /opt/tools/rlinq/clones/rlinq_ansible/environments/test/hosts as it did not pass its verify_file() method
Parsed /opt/tools/rlinq/clones/rlinq_ansible/environments/test/hosts inventory source with ini plugin

PLAYBOOK: mitogen_test_rhel8.yml ****************************************************************************************************************************
1 plays in mitogen_test_rhel8.yml

PLAY [rhel8-test] **************************************************************************************************************************************
[top  18273] 16:11:06.072438 D ansible_mitogen.affinity: CPU mask for Ansible top-level process: 0x000002
[top  18273] 16:11:06.072951 D ansible_mitogen.process: inherited open file limits: soft=1024 hard=4096
[top  18273] 16:11:06.073176 D ansible_mitogen.process: raised soft open file limit from 1024 to 4096
[mux  18280] 16:11:06.076797 D ansible_mitogen.affinity: CPU mask for MuxProcess 0: 0x000001
[mux  18280] 16:11:06.086035 D mitogen.service: Pool(9828, size=32, th='MainThread'): initialized
[mux  18280] 16:11:06.087603 D ansible_mitogen.process: Service pool configured: size=32

TASK [setup] ************************************************************************************************************************************************
task path: /opt/tools/rlinq/clones/rlinq_ansible/roles/mitogen_test_rhel8.yml:5
[task 18314] 16:11:06.151910 D ansible_mitogen.affinity: CPU mask for WorkerProcess: 0x000002
[task 18314] 16:11:06.168864 D ansible_mitogen.process: will use multiplexer 0 (/tmp/mitogen_unix_acgc811n.sock) to connect to "rhel8-test"
[task 18314] 16:11:06.169072 D mitogen.unix: client: connecting to /tmp/mitogen_unix_acgc811n.sock
[task 18314] 16:11:06.169753 D mitogen.unix: client: local ID is 1, remote is 0
[mux  18280] 16:11:06.169885 D mitogen.unix: listener: accepted connection from PID 18314: unix_client.18314
[mux  18280] 16:11:06.173997 D mitogen.parent: creating connection to context 2 using mitogen.ssh
[mux  18280] 16:11:06.350272 D mitogen.parent: command line for Connection(None): ssh -o "LogLevel ERROR" -i /export/home/rlinqmgr/secret/rlinqmgr_secret_id_rsa -o "Compression yes" -o "ServerAliveInterval 30" -o "ServerAliveCountMax 10" -o "BatchMode yes" -o "StrictHostKeyChecking yes" -F ./ssh_config rhel8-test /usr/bin/python -c "'import codecs,os,sys;_=codecs.decode;exec(_(_(REMOVED_ON_PURPOSE=\".encode(),\"base64\"),\"zip\"))'"
[mux  18280] 16:11:06.357355 D mitogen.parent: child for Connection(None) started: pid:18317 stdin:76 stdout:76 stderr:78
[mux  18280] 16:11:06.869922 D mitogen.parent: ssh.rhel8-test: (unrecognized): bash: /usr/bin/python: No such file or directory
[mux  18280] 16:11:06.870307 D mitogen: <Side of ssh.rhel8-test fd 76>: empty read, disconnecting
[mux  18280] 16:11:06.871088 D mitogen: BootstrapProtocol(ssh.rhel8-test): disconnecting
[mux  18280] 16:11:06.871509 D mitogen: <Side of ssh.rhel8-test fd 78>: empty read, disconnecting
[mux  18280] 16:11:06.871673 D mitogen.parent: failing connection ssh.rhel8-test due to EofError('EOF on stream; last 100 lines received:\nbash: /usr/bin/python: No such file or directory',)
[mux  18280] 16:11:06.872036 D mitogen.parent: PopenProcess ssh.rhel8-test pid 18317: exited with return code 127
[mux  18280] 16:11:06.872191 D mitogen: SetupProtocol(ssh.rhel8-test): disconnecting
[mux  18280] 16:11:06.872615 D mitogen: SetupProtocol(ssh.rhel8-test): disconnecting
[task 18314] 16:11:06.884586 D mitogen: MitogenProtocol(unix_listener.18280): disconnecting
[task 18314] 16:11:06.885241 D mitogen: Waker(fd=10/11): disconnecting
[task 18314] 16:11:06.885705 D mitogen: Router(Broker(37f0)): stats: 0 module requests in 0 ms, 0 sent (0 ms minify time), 0 negative responses. Sent 0.0 kb total, 0.0 kb avg.
[mux  18280] 16:11:06.885811 D mitogen: <Side of unix_client.18314 fd 75>: empty read, disconnecting
[mux  18280] 16:11:06.885993 D mitogen: MitogenProtocol(unix_client.18314): disconnecting
fatal: [rhel8-test]: UNREACHABLE! => changed=false
  msg: |-
    EOF on stream; last 100 lines received:
    bash: /usr/bin/python: No such file or directory
  unreachable: true

PLAY RECAP **************************************************************************************************************************************************
rhel8-test            : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0

Running against localhost, this results in an OK identification of python3 version

Included only snippet until python path is identified and connection is setup.

TASK [Gathering Facts] **************************************************************************************************************************************
task path: /opt/tools/rlinq/clones/rlinq_ansible/roles/mitogen_test_rhel8.yml:1
[task 30354] 22:05:54.515629 D ansible_mitogen.affinity: CPU mask for WorkerProcess: 0x000002
[task 30354] 22:05:54.526037 D ansible_mitogen.process: will use multiplexer 0 (/tmp/mitogen_unix_n73udz9n.sock) to connect to "localhost"
[task 30354] 22:05:54.526285 D mitogen.unix: client: connecting to /tmp/mitogen_unix_n73udz9n.sock
[task 30354] 22:05:54.527010 D mitogen.unix: client: local ID is 1, remote is 0
[mux  30320] 22:05:54.527165 D mitogen.unix: listener: accepted connection from PID 30354: unix_client.30354
[mux  30320] 22:05:54.528993 D mitogen.parent: creating connection to context 2 using mitogen.parent
[mux  30320] 22:05:54.711130 D mitogen.parent: command line for Connection(None): /usr/bin/python3 -c "import codecs,os,sys;_=codecs.decode;exec(_(_(\"eNqFkU9Lw0AQxc/Np8htd+mablLEElhQehAPIgSxBy2SP5t2abq77G4a66d3TMUm9eBtfsybeY+ZjK64dpGRRmASWNoNSNYhQK3tDpM0mEBdtSbBjMaMkTNndEgWuvGJy0Y7gbMh2CGshtABgKE7gn2Te3Ddh5yHqMptJxUKc1X1TfEhytbnRSP69qx1dlZINTNHv9UKQc7JhWzK+8GDsE5q9ZrO172tUAdpgdFddv/C0JqPx04awAaPG3SMU4T30uuNUKkzTat2+429le7KbkWzKFvn2bWLtBJ5ZSMlfDpn84QRRAJY31npBY4penx4fmKMvSkEsUpdwfVJsOTv+Pv+lTZCwdWRLRCJLGzC8Q1LFoSiT2lgU234WbeiqCvQ90tq82Ow7OvTmS/U3X/qvynjccrfBybkC0uYucA=\".encode(),\"base64\"),\"zip\"))"
[mux  30320] 22:05:54.718144 D mitogen.parent: child for Connection(None) started: pid:30357 stdin:76 stdout:76 stderr:None
[mux  30320] 22:05:54.759414 D mitogen.parent: BootstrapProtocol(local.30357): first stage started succcessfully
[mux  30320] 22:05:54.764670 D mitogen.parent: BootstrapProtocol(local.30357): first stage received mitogen.core source
[mux  30320] 22:05:54.833924 D mitogen.parent: BootstrapProtocol(local.30357): new child booted successfully
[mux  30320] 22:05:54.834463 D mitogen.[local.30357]: Python version is 3.6.8 (default, Jun 12 2019, 01:12:31)
[GCC 8.2.1 20180905 (Red Hat 8.2.1-3)]
[mux  30320] 22:05:54.834621 D mitogen.[local.30357]: Parent is context 0 (master); my ID is 2
[mux  30320] 22:05:54.834739 D mitogen.[local.30357]: pid:30357 ppid:30320 uid:38001942/38001942, gid:38002279/38002279 host:'rhel8-test.company.com'
[mux  30320] 22:05:54.834848 D mitogen.[local.30357]: Recovered sys.executable: '/usr/bin/python3'
[mux  30320] 22:05:54.835892 D mitogen.parent: starting function call to local.30357: ansible_mitogen.target.init_child(log_level=10, candidate_temp_dirs=['/tmp', '/var/tmp', '/tmp'])

Do you have some idea of what the underlying problem may be?

Redhat 8 Server doesn't have /usr/bin/python, the mitogen code handles remote connection different from localhost. Where localhost successfully identifies correct python path a remote connection to the same host fails.

Mention your host and target OS and versions
Same behavior seen using RHEL7 or RHEL8 as host towards a RHEL8 target. Logs only include from RHEL8 -> RHEL8. Tests conducted from RHEL7-RHEL8 with the same result.
Host1
- Red Hat Enterprise Linux release 8.0 (Oopta)
- Linux rhel8-test 4.18.0-80.11.2.el8_0.x86_64 #1 SMP Sun Sep 15 11:24:21 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Host2
- Red Hat Enterprise Linux Server release 7.7 (Maipo)
- Linux rhel7-test 3.10.0-1062.1.2.el7.x86_64 #1 SMP Mon Sep 16 14:19:51 EDT 2019 x86_64 x86_64 x86_64 GNU/Linux
Target
- Red Hat Enterprise Linux release 8.0 (Oopta)
- Linux rhel8-test 4.18.0-80.11.2.el8_0.x86_64 #1 SMP Sun Sep 15 11:24:21 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Mention your host and target Python versions
Host1
- Python 3.6.8
Host2
- Python 2.7.5
Target
- Python 3.6.8
If reporting a performance issue, mention the number of targets and a rough
description of your workload (lots of copies, lots of tiny file edits, etc.)
If reporting a crash or hang in Ansible, please rerun with -vvv and include
200 lines of output around the point of the error, along with a full copy of
any traceback or error text in the log. Beware "-vvv" may include secret
data! Edit as necessary before posting.
If reporting any kind of problem with Ansible, please include the Ansible
version along with output of "ansible-config dump --only-changed".

ANSIBLE_SSH_ARGS(/opt/tools/rlinq/clones/rlinq_ansible/roles/ansible.cfg) = -F ./ssh_config
DEFAULT_CALLBACK_PLUGIN_PATH(/opt/tools/rlinq/clones/rlinq_ansible/roles/ansible.cfg) = ['/usr/share/ansible/plugins/callback', '/opt/tools/rlinq/dev/a>
DEFAULT_FORKS(/opt/tools/rlinq/clones/rlinq_ansible/roles/ansible.cfg) = 100
DEFAULT_HOST_LIST(/opt/tools/rlinq/clones/rlinq_ansible/roles/ansible.cfg) = ['/opt/tools/rlinq/clones/rlinq_ansible/roles/environments/test']
DEFAULT_LOAD_CALLBACK_PLUGINS(/opt/tools/rlinq/clones/rlinq_ansible/roles/ansible.cfg) = True
DEFAULT_PRIVATE_KEY_FILE(/opt/tools/rlinq/clones/rlinq_ansible/roles/ansible.cfg) = /export/home/rlinqmgr/secret/rlinqmgr_secret_id_rsa
DEFAULT_STDOUT_CALLBACK(/opt/tools/rlinq/clones/rlinq_ansible/roles/ansible.cfg) = yaml
DEFAULT_STRATEGY(/opt/tools/rlinq/clones/rlinq_ansible/roles/ansible.cfg) = mitogen_linear
DEFAULT_STRATEGY_PLUGIN_PATH(/opt/tools/rlinq/clones/rlinq_ansible/roles/ansible.cfg) = ['/opt/tools/rlinq/tools/mitogen-0.2.9/ansible_mitogen/plugins/>
DEFAULT_TIMEOUT(/opt/tools/rlinq/clones/rlinq_ansible/roles/ansible.cfg) = 60

The text was updated successfully, but these errors were encountered:

bradh352 · 2020-01-17T20:47:02Z

duplicate of #597 ?

s1113950 · 2020-02-27T23:14:13Z

@rlinq can you try my patch here and see if it works for you? #658

rlinq · 2020-03-02T15:26:47Z

@rlinq can you try my patch here and see if it works for you? #658

I downloaded your patch from https://github.com/s1113950/mitogen/tree/complexAnsiblePythonInterpreterArg Hoping that it's correct way of testing you patch.
Installed it on my RedHat 7 server, and updated ansible.cfg to point at your patch.
strategy_plugins = /opt/nordea/splunk/tools/mitogen-patch2/ansible_mitogen/plugins/strategy strategy = mitogen_linear

Unfortunately the result is as before, python version is not detected on RHEL8 destination servers.

Let me know if there is more that I can test for you?

`ansible-playbook 2.9.2
config file = /opt/company/splunk/dev/ansible/core/roles/ansible.cfg
configured module search path = [u'/export/home/splunkmgr/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /bin/ansible-playbook
python version = 2.7.5 (default, Jun 11 2019, 14:33:56) [GCC 4.8.5 20150623 (Red Hat 4.8.5-39)]
Using /opt/company/splunk/dev/ansible/core/roles/ansible.cfg as config file
host_list declined parsing /opt/company/splunk/dev/ansible/core/environments/test/hosts as it did not pass its verify_file() method
script declined parsing /opt/company/splunk/dev/ansible/core/environments/test/hosts as it did not pass its verify_file() method
auto declined parsing /opt/company/splunk/dev/ansible/core/environments/test/hosts as it did not pass its verify_file() method
Parsed /opt/company/splunk/dev/ansible/core/environments/test/hosts inventory source with ini plugin

PLAYBOOK: manage_splunk_forwarders.yml **********************************************************************************************************************
5 plays in manage_splunk_forwarders.yml

PLAY [heavy_forwarders:&heavy_forwarders_dmz] ***************************************************************************************************************
skipping: no hosts matched

PLAY [heavy_forwarders:&heavy_forwarders_blue] **************************************************************************************************************
[top 7110] 16:14:21.124795 D ansible_mitogen.affinity: CPU mask for Ansible top-level process: 0x000002
[top 7110] 16:14:21.126470 D ansible_mitogen.process: inherited open file limits: soft=1024 hard=4096
[top 7110] 16:14:21.127179 D ansible_mitogen.process: raised soft open file limit from 1024 to 4096
[mux 7119] 16:14:21.132028 D ansible_mitogen.affinity: CPU mask for MuxProcess 0: 0x000001
[mux 7119] 16:14:21.151125 D mitogen.service: Pool(0b90, size=32, th='MainThread'): initialized
[mux 7119] 16:14:21.151836 D ansible_mitogen.process: Service pool configured: size=32

TASK [setup] ************************************************************************************************************************************************
task path: /opt/company/splunk/dev/ansible/core/roles/manage_splunk_forwarders.yml:31
[task 7154] 16:14:21.215836 D ansible_mitogen.affinity: CPU mask for WorkerProcess: 0x000002
[task 7154] 16:14:21.254403 D ansible_mitogen.process: will use multiplexer 0 (/tmp/mitogen_unix_P7PHJ5.sock) to connect to "rhel8-host"
[task 7154] 16:14:21.255483 D mitogen.unix: client: connecting to /tmp/mitogen_unix_P7PHJ5.sock
[task 7154] 16:14:21.256706 D mitogen.unix: client: local ID is 1, remote is 0
[mux 7119] 16:14:21.256782 D mitogen.unix: listener: accepted connection from PID 7154: unix_client.7154
[mux 7119] 16:14:21.262320 D mitogen.parent: creating connection to context 2 using mitogen.ssh
[mux 7119] 16:14:21.500107 D mitogen.parent: command line for Connection(None): ssh -o "LogLevel ERROR" -l splunkmgr -i /export/home/splunkmgr/secret/splunkmgr_secret_id_rsa -o "Compression yes" -o "ServerAliveInterval 30" -o "ServerAliveCountMax 10" -o "BatchMode yes" -o "StrictHostKeyChecking yes" -F ./ssh_config rhel8-host /usr/bin/python -c "'import codecs,os,sys;=codecs.decode;exec((_("e.../Hl9YpAydHvkAk8bAI8xmb3hCKvrSDTaUTZ92CorZAx3+U7sdg3tWnG1+o2//Uf1PyYcrf703JN15yuSE=".encode(),"base64"),"zip"))'"
[mux 7119] 16:14:21.510204 D mitogen.parent: child for Connection(None) started: pid:7157 stdin:77 stdout:77 stderr:79
[mux 7119] 16:14:21.981576 D mitogen.parent: ssh.rhel8-host: (unrecognized): bash: /usr/bin/python: No such file or directory
[mux 7119] 16:14:21.982482 D mitogen: <Side of ssh.rhel8-host fd 77>: empty read, disconnecting
[mux 7119] 16:14:21.982724 D mitogen: BootstrapProtocol(ssh.rhel8-host): disconnecting
[mux 7119] 16:14:21.982946 D mitogen: <Side of ssh.rhel8-host fd 79>: empty read, disconnecting
[mux 7119] 16:14:21.983172 D mitogen.parent: failing connection ssh.rhel8-host due to EofError(u'EOF on stream; last 100 lines received:\nbash: /usr/bin/python: No such file or directory',)
[mux 7119] 16:14:21.983480 D mitogen.parent: PopenProcess ssh.rhel8-host pid 7157: exited with return code 127
[mux 7119] 16:14:21.983796 D mitogen: SetupProtocol(ssh.rhel8-host): disconnecting
[mux 7119] 16:14:21.984817 D mitogen: SetupProtocol(ssh.rhel8-host): disconnecting
[task 7154] 16:14:21.986162 D mitogen: MitogenProtocol(unix_listener.7119): disconnecting
[task 7154] 16:14:21.986598 D mitogen: Waker(fd=11/12): disconnecting
[mux 7119] 16:14:21.986630 D mitogen: <Side of unix_client.7154 fd 76>: empty read, disconnecting
[task 7154] 16:14:21.986999 D mitogen: Router(Broker(3f10)): stats: 0 module requests in 0 ms, 0 sent (0 ms minify time), 0 negative responses. Sent 0.0 kb total, 0.0 kb avg.
[mux 7119] 16:14:21.987384 D mitogen: MitogenProtocol(unix_client.7154): disconnecting
fatal: [rhel8-host]: UNREACHABLE! => changed=false
msg: |-
EOF on stream; last 100 lines received:
bash: /usr/bin/python: No such file or directory
unreachable: true`

s1113950 · 2020-03-02T18:01:34Z

That's the correct way; @rlinq can you run your playbook again with -vvv so I can see what discovery tried to do on your setup? 🙏

rlinq · 2020-03-02T18:27:00Z

Sure, I though I ran it with -vvv last time, now with -vvvvv. Unfortunately I don't think you get what you want as I can't see any discovery messages in the output.

Let me know what you find and I will rerun the tests for you.

mitogen_debug_rhel8_issue658.log

s1113950 · 2020-03-02T23:38:10Z

You're correct; there wasn't much in that log :/ I got it passing tests on a centos:8 container using https://github.com/s1113950/mitogen-test but I guess it's not the same as RHEL8. I think I know the problem though 🤔 it has to do with transport_config.py. I'll try and replicate using the RHEL8 container I found here: https://hub.docker.com/u/roboxes/ . Let me know if that won't properly replicate your issue!

s1113950 · 2020-03-02T23:40:33Z

Unfortunately both Azure and Travis only support Ubuntu (according to https://docs.travis-ci.com/user/reference/overview/#linux and https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted?view=azure-devops#use-a-microsoft-hosted-agent ), so the best we have is to run tests targeting a container rather than baremetal

s1113950 · 2020-03-03T00:17:49Z

aHa! @rlinq I can reproduce the error running make complex-args-test CONTAINER_IMAGE=roboxes/rhel8:2.0.6 USE_DOCKER=true from https://github.com/s1113950/mitogen-test . Sorry about that; I tested centos:8 but not rhel8 (also installed python on centos:8 which is probably why it passed tests).

rlinq · 2020-03-03T07:28:52Z

@s1113950 Great that you were able to reproduce it, I didn't see this coming, python differences between RHEL8 and CentOS 8.

Let me know if I can assist when you got your tests to run through successfully

s1113950 · 2020-03-03T22:48:58Z

#658 (comment) this is proving to be a pain 😓 I'm gonna have to rethink some things in order to get it to work on RHEL8 (might end up with a loop of the different python interpreters around a try/catch statement to avoid infinite recursion 😞 )

s1113950 · 2020-03-04T08:13:21Z

@rlinq I gotta get my PR passing tests again but I got specifically the interpreter discovery tests running on rhel8 :) I should have time tomorrow to get the PR fully passing tests again

rlinq · 2020-03-04T09:30:25Z

@s1113950 I downloaded your latest patch and it's working great, I really appreciate your work!

Looking at your code, I think one optimization can be done in the list of potential python versions.
I added python3 as the second option, it's at least on RHEL linked to the specific python version binaries. Then the try/catch will only be looped 2 times on a proper system configuration, trying python(2) and then python3. Fallback to the rest of your list if the two first is not working.

Tested it and it works, unfortunately I have no access from where I work to do a pull request.

possible_pythons = [ '/usr/bin/python', 'python3', 'python3.7', 'python3.6', 'python3.5', 'python2.7', 'python2.6', 'usr/libexec/platform-python', 'usr/bin/python3', 'python' ]

RHEL8 linked python binaries from /usr/bin/python3
[root@is-rhel8-host ~]$ which python3 /usr/bin/python3 [root@is-rhel8-host ~]$ ls -ltr /usr/bin/python3 lrwxrwxrwx. 1 root root 25 Feb 9 15:58 /usr/bin/python3 -> /etc/alternatives/python3 [root@is-rhel8-host ~]$ ls -l /etc/alternatives/python3 lrwxrwxrwx. 1 root root 18 Feb 9 15:58 /etc/alternatives/python3 -> /usr/bin/python3.6

s1113950 · 2020-03-04T18:29:56Z

np, glad to read that it's working for you! :D

I'll add in python3 as an option. Unfortunately, that might not be what Ansible returns though:
https://github.com/ansible/ansible/blob/v2.8.8/lib/ansible/executor/interpreter_discovery.py#L64 checks the INTERPRETER_PYTHON_FALLBACK options which are https://github.com/ansible/ansible/blob/v2.8.8/lib/ansible/config/base.yml#L1475 (maybe there needs to be a PR made to Ansible to add python3 as an option then 🤔 )

Since I call discover_interpreter() it uses whatever Ansible returns for us. Our issue with Mitogen is it needs to know which Python to run on the other system to even call this function in the first place 😅 so we're sorta double-detecting the interpreter, but I'm letting Ansible handle the bulk of the logic.

s1113950 · 2020-03-15T19:51:38Z

@rlinq can you try my patch one more time? I got it passing tests/made some more tweaks. Sorry it took so long; I just switched jobs and didn't have as much time as before to work on it til now.

s1113950 · 2020-03-22T05:52:22Z

Another person confirmed it worked for them as well so I went ahead and merged the patch after tests passed.

joinmaster66 · 2020-05-07T07:52:10Z

Where i can find this patch? https://github.com/s1113950/mitogen/tree/complexAnsiblePythonInterpreterArg I've the same issue with RHEL8. It doesnt work for me with the acctually version (latest)...
Thanks for Help!

s1113950 · 2020-05-07T18:50:51Z

oh shoot 🤔 someone else also reported it didn't work for them either, and I suggested #658 (comment) . Can you try the suggestions I put in that comment and see if it works?

bradh352 · 2020-05-07T18:56:55Z

not sure I understand what is being said. @joinmaster66 are you running the latest mitogen release version (0.2.9), the latest mitogen github master, or .... ?

joinmaster66 · 2020-05-08T06:35:56Z

yes i use the latest mitogen version (0.2.9). In my case i use mitogen for multiple systems (rhel7, rhel8, etc.) in each case it works but in Rhel8 it doesnt. The ansible version i use (2.9.3) @bradh352

bradh352 · 2020-05-08T10:12:28Z

That is your problem, the patch is NOT in a release. You need to install mitogen from master. Something like this:
sudo pip3 install git+https://github.com/dw/mitogen

oliverlj · 2020-05-24T15:30:35Z

i have same issue on willhallonline/ansible:2.9-alpine container

s1113950 · 2020-05-24T15:42:07Z

@oliverlj I'll try and repro with that container this weekend

oliverlj · 2020-05-24T16:28:31Z

here is the ansible.cfg is used

strategy = mitogen_linear
strategy_plugins = /usr/lib/python3.8/site-packages/ansible_mitogen/plugins/strategy

and the inventory workaround

ansible_python_interpreter: /usr/bin/python3

s1113950 · 2020-05-26T00:02:00Z

@oliverlj I'm able to run Mitogen master on willhallonline/ansible:2.9-alpine. Please update your Mitogen version via something like pip3 install git+https://github.com/dw/mitogen. A new release of Mitogen is planned once Ansible 2.10 comes out, but until then master has the ansible_python_interpreter discovery fix.

le0m · 2020-08-04T17:16:24Z

I encountered the same problem on Ubuntu 20.04, but installing Mitogen from master with pip3 install git+https://github.com/dw/mitogen fixed it! 😄

That said, I've already been bitten in the past by not pinning versions. I saw from your comment on your PR (#658 (comment)) that we can pin to a commit, but would you consider to release a patch version 0.2.9.1?

s1113950 · 2020-08-05T01:01:42Z

@le0m I don't have the ability to push to Mitogen's PyPi account, so that would be up to @dw . There are already plans to cut a release after Ansible 2.10 comes out though, which is expected on August 13th.

amarao · 2020-10-29T11:24:28Z

Ansible 2.10 was released few months ago. Can you release it, please?

s1113950 · 2020-12-28T01:27:16Z

New rc available here: https://github.com/dw/mitogen/releases/tag/v0.3.0-rc.0 for Ansible 2.10+
For Ansible < 2.10, there's also https://github.com/dw/mitogen/releases/tag/v0.2.10-rc.0
both have the fix in them

frankruegamer mentioned this issue Dec 18, 2019

ANSIBLE_PYTHON_INTERPRETER env variable not used? #650

Closed

s1113950 mentioned this issue Feb 27, 2020

Adds support for special ansible_python_interpreter values, ansible_python_interpreter discovery, and fixes tests #658

Merged

s1113950 closed this as completed in #658 Mar 22, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mitogen fails on RHEL8 server with bash: /usr/bin/python: No such file or directory #673

Mitogen fails on RHEL8 server with bash: /usr/bin/python: No such file or directory #673

rlinq commented Dec 2, 2019 •

edited

Loading

bradh352 commented Jan 17, 2020

s1113950 commented Feb 27, 2020

rlinq commented Mar 2, 2020 •

edited

Loading

s1113950 commented Mar 2, 2020

rlinq commented Mar 2, 2020

s1113950 commented Mar 2, 2020

s1113950 commented Mar 2, 2020

s1113950 commented Mar 3, 2020

rlinq commented Mar 3, 2020

s1113950 commented Mar 3, 2020

s1113950 commented Mar 4, 2020

rlinq commented Mar 4, 2020

s1113950 commented Mar 4, 2020

s1113950 commented Mar 15, 2020

s1113950 commented Mar 22, 2020

joinmaster66 commented May 7, 2020 •

edited

Loading

s1113950 commented May 7, 2020

bradh352 commented May 7, 2020

joinmaster66 commented May 8, 2020 •

edited

Loading

bradh352 commented May 8, 2020

oliverlj commented May 24, 2020

s1113950 commented May 24, 2020

oliverlj commented May 24, 2020

s1113950 commented May 26, 2020

le0m commented Aug 4, 2020 •

edited

Loading

s1113950 commented Aug 5, 2020

amarao commented Oct 29, 2020

s1113950 commented Dec 28, 2020

Mitogen fails on RHEL8 server with bash: /usr/bin/python: No such file or directory #673

Mitogen fails on RHEL8 server with bash: /usr/bin/python: No such file or directory #673

Comments

rlinq commented Dec 2, 2019 • edited Loading

bradh352 commented Jan 17, 2020

s1113950 commented Feb 27, 2020

rlinq commented Mar 2, 2020 • edited Loading

s1113950 commented Mar 2, 2020

rlinq commented Mar 2, 2020

s1113950 commented Mar 2, 2020

s1113950 commented Mar 2, 2020

s1113950 commented Mar 3, 2020

rlinq commented Mar 3, 2020

s1113950 commented Mar 3, 2020

s1113950 commented Mar 4, 2020

rlinq commented Mar 4, 2020

s1113950 commented Mar 4, 2020

s1113950 commented Mar 15, 2020

s1113950 commented Mar 22, 2020

joinmaster66 commented May 7, 2020 • edited Loading

s1113950 commented May 7, 2020

bradh352 commented May 7, 2020

joinmaster66 commented May 8, 2020 • edited Loading

bradh352 commented May 8, 2020

oliverlj commented May 24, 2020

s1113950 commented May 24, 2020

oliverlj commented May 24, 2020

s1113950 commented May 26, 2020

le0m commented Aug 4, 2020 • edited Loading

s1113950 commented Aug 5, 2020

amarao commented Oct 29, 2020

s1113950 commented Dec 28, 2020

rlinq commented Dec 2, 2019 •

edited

Loading

rlinq commented Mar 2, 2020 •

edited

Loading

joinmaster66 commented May 7, 2020 •

edited

Loading

joinmaster66 commented May 8, 2020 •

edited

Loading

le0m commented Aug 4, 2020 •

edited

Loading