docker_image.present fails first time when python3-docker module is installed #54449

golpa · 2019-09-10T20:53:31Z

Description of Issue

I'm trying to use docker_image.present but salt fails to detect the module on first state.apply because the required python modules are installed in that run. On second state.apply the module is detected and used. This is even the case with reload_modules: true set.

On the first run I see:

----------
          ID: Docker
    Function: pkg.installed
      Result: True
     Comment: The following packages were installed/updated: python3-docker
              The following packages were already installed: docker-ce
     Started: 16:12:02.098635
    Duration: 1403.352 ms
     Changes:   
              ----------
              python3-docker:
                  ----------
                  new:
                      1.9.0-1
                  old:
              python3-websocket:
                  ----------
                  new:
                      0.37.0-2
                  old:
----------
          ID: Kubernetes Dependency Image k8s.gcr.io/pause
    Function: docker_image.present
        Name: k8s.gcr.io/pause
      Result: False
     Comment: State 'docker_image.present' was not found in SLS 'kube-minion'
              Reason: 'docker_image' __virtual__ returned False: 'docker.version' is not available.
     Changes:

On the minion itself the salt-minion logs have this error:

salt-minion[138912]: [ERROR   ] State 'docker_image.present' was not found in SLS 'kube-minion'
salt-minion[138912]: Reason: 'docker_image' __virtual__ returned False: 'docker.version' is not available.

However, it works on second run:

----------
          ID: Docker
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed and are at the desired version
     Started: 16:29:44.739748
    Duration: 26.875 ms
     Changes:  
----------
          ID: Kubernetes Dependency Image k8s.gcr.io/pause
    Function: docker_image.present
        Name: k8s.gcr.io/pause
      Result: True
     Comment: Image 'k8s.gcr.io/pause:3.1' was pulled
     Started: 16:29:44.859688
    Duration: 1079.729 ms
     Changes:   
              ----------
              Layers:
                  ----------
                  Pulled:
                      - 67ddbfb20a22
              Status:
                  Downloaded newer image for k8s.gcr.io/pause:3.1
              Time_Elapsed:
                  0.4564225673675537
              retcode:
                  0

Setup

I have two state files. docker.sls:

Docker:
  pkg.installed:
    - pkgs:
        - docker-ce: 18.06.1~ce~3-0~debian
        - python3-docker
    - install_recommends: False
    - reload_modules: true

and kube.sls:

include:
  - docker

Kubernetes Dependency Image k8s.gcr.io/pause:
  docker_image.present:
    - name: k8s.gcr.io/pause
    - tag: 3.1
    - reload_modules: True
    - require:
      - pkg: Docker
      - service: Docker
      - pkg: Kubernetes Dependencies

This may be related to #54332 but in that case pip is the installation method and the error message is different.

Steps to Reproduce Issue

Run salt <minion> state.apply
Get Failure
Run salt <minion> state.apply
Success!

Versions Report

On Master:

Salt Version:
           Salt: 2019.2.0
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.3
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.5.3 (default, Sep 27 2018, 17:25:39)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.4.3
            ZMQ: 4.2.1
 
System Versions:
           dist: debian 9.9 
         locale: UTF-8
        machine: x86_64
        release: 4.9.0-9-amd64
         system: Linux
        version: debian 9.9

On the Minion (after python3-docker has been installed):

Salt Version:
           Salt: 2019.2.0
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.5.3
      docker-py: 1.9.0
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.9.4
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.8
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.5.3 (default, Sep 27 2018, 17:25:39)
   python-gnupg: Not Installed
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.4.3
            ZMQ: 4.2.1
 
System Versions:
           dist: debian 9.9 
         locale: UTF-8
        machine: x86_64
        release: 4.9.0-9-amd64
         system: Linux
        version: debian 9.9

The text was updated successfully, but these errors were encountered:

xeacott · 2019-10-30T18:52:25Z

I'm not reproducing this entirely, getting a different error but may be with my sls file...

linux:
----------
          ID: Docker
    Function: pkg.installed
      Result: True
     Comment: All specified packages are already installed and are at the desired version
     Started: 18:51:18.833487
    Duration: 95.318 ms
     Changes:
----------
          ID: Kubernetes Dependency Image k8s.gcr.io/pause
    Function: docker_image.present
        Name: k8s.gcr.io/pause
      Result: False
     Comment: The following requisites were not found:
                                 require:
                                     service: Docker
                                     pkg: Kubernetes Dependencies
     Started: 18:51:18.950491
    Duration: 0.179 ms
     Changes:

Summary for linux
------------
Succeeded: 1
Failed:    1
------------
Total states run:     2
Total run time:  95.497 ms

I did use the provided states above

xeacott · 2019-10-30T18:54:46Z

I probably am missing some kubernetes package

xpicio · 2019-11-29T14:26:20Z

Hello i'm reporting the same issue using docker formula (it seems that in docker formula they use reload_modules: true)

Here a snippet of my sls file:

include:
  - docker

internal_network:
  docker_network.present:
    - subnet: 172.20.0.0/16

redis service:
  docker_container.running:
    - name: redis
    - image: registry.***.***/***/redis:v1
    - binds:
      - /opt/redis/data:/data
    - ports:
      - 6379
    - networks:
      - internal_network
      - bridge
    - restart_policy: on-failure:3
    - log_driver: syslog
    - log_opt:
      - syslog-facility: daemon

The error on first run:

----------
          ID: internal_network
    Function: docker_network.present
      Result: False
     Comment: State 'docker_network.present' was not found in SLS 'init-gtw'
              Reason: 'docker_network' __virtual__ returned False: 'docker.version' is not available.
     Changes:
----------
          ID: redis service
    Function: docker_container.running
        Name: redis
      Result: False
     Comment: State 'docker_container.running' was not found in SLS 'init-gtw'
              Reason: 'docker_container' __virtual__ returned False: 'docker.version' is not available.
     Changes:
----------

The result on second run:

----------
          ID: internal_network
    Function: docker_network.present
      Result: True
     Comment: Network 'internal_network' created
     Started: 01:48:06.168053
    Duration: 157.235 ms
     Changes:
              ----------
              created:
                  True
----------
          ID: redis service
    Function: docker_container.running
        Name: redis
      Result: True
     Comment: Created container 'redis'
     Started: 01:48:06.453425
    Duration: 12383.978 ms
     Changes:
              ----------
              container:
                  ----------
                  Networks:
                      ----------
                      internal_network:
                          ----------
                          Gateway:
                              ----------
                              new:
                                  172.20.0.1
                              old:
                                  None
                          IPAddress:
                              ----------
                              new:
                                  172.20.0.2
                              old:
                                  None
              container_id:
                  ----------
                  added:
                      128201edd8beb49da5321278b2ae5cf01dd8e36577357a09c0f474cb6936a800
              image:
                  ----------
                  Layers:
                      ----------
                      Pulled:
                          - e42672a2ca6a
                          - 823e4275fef6
                          - 327e62bcfef7
                          - be5997a801f5
                          - dc62338c8c74
                          - 6c40cc604d8e
                          - 79ed9185c842
                          - c2ad7a3b6c3f
                  Status:
                      Downloaded newer image for registry.***.***/***/redis:v1
                  Time_Elapsed:
                      3.1893012523651123
                  retcode:
                      0
              state:
                  ----------
                  new:
                      running
                  old:
                      None

Versions on server:

Salt Version:
           Salt: 2019.2.2

Dependency Versions:
           cffi: 1.11.5
       cherrypy: unknown
       dateutil: 2.2
      docker-py: Not Installed
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10.3
        libgit2: 0.28.2
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.6.2
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 2.6.1
   pycryptodome: 3.9.4
         pygit2: 0.28.2
         Python: 3.7.5 (default, Oct 17 2019, 12:25:15)
   python-gnupg: Not Installed
         PyYAML: 3.13
          PyZMQ: 18.1.1
           RAET: Not Installed
          smmap: Not Installed
        timelib: 0.2.4
        Tornado: 4.5.3
            ZMQ: 4.3.2

System Versions:
           dist:
         locale: UTF-8
        machine: x86_64
        release: 4.14.138+
         system: Linux
        version: Not Installed

Versions on minion:

root@ubuntu-18:/tmp# salt-call --versions-report
Salt Version:
           Salt: 2019.2.2

Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: 2.5.1
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.9 (default, Nov  7 2019, 10:44:02)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5

System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-29-generic
         system: Linux
        version: Ubuntu 18.04 bionic

stale · 2020-01-16T18:01:36Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

lsh-0 · 2020-01-20T05:13:16Z

I'm getting this too on Salt 2018.3

Salt Version:
           Salt: 2018.3.4
 
Dependency Versions:
           cffi: Not Installed
       cherrypy: Not Installed
       dateutil: 2.6.1
      docker-py: 4.1.0
          gitdb: Not Installed
      gitpython: Not Installed
          ioflo: Not Installed
         Jinja2: 2.10
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.5.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 3.6.9 (default, Nov  7 2019, 10:44:02)
   python-gnupg: 0.4.1
         PyYAML: 3.12
          PyZMQ: 16.0.2
           RAET: Not Installed
          smmap: Not Installed
        timelib: Not Installed
        Tornado: 4.5.3
            ZMQ: 4.2.5
 
System Versions:
           dist: Ubuntu 18.04 bionic
         locale: UTF-8
        machine: x86_64
        release: 4.15.0-72-generic
         system: Linux
        version: Ubuntu 18.04 bionic

stale · 2020-01-20T05:13:19Z

Thank you for updating this issue. It is no longer marked as stale.

lsh-0 · 2020-01-20T06:38:34Z

FWIW, ensuring the docker python library is installed prior to calling highstate seems to work. The python-mysqldb dependency doesn't seem to fail like this but historically we had problems with upgrading python using salt states as well (reload_modules: true never seemed to work for us either) so I think it's just safest to call the Python docker a hard prerequisite before running Salt.

darkn3rd · 2020-02-06T21:48:29Z

I run into the same issue with salt-minion 2019.2.3 (Fluorine) when running on Ubuntu 18.04:

salt-call state.highstate --retcode-passthrough --local --log-level=debug --force-color

I install the docker module, but then run docker_container.running, which does a Traceback. Second time running it, it works.

The workaround is to run the state for docker and docker python module separately from the other states that depend on it. Otherwise, a Traceback is the current UI to users of Salt Stack.

supfors · 2020-02-21T09:43:57Z

Running into the same issue using salt 3000 on ubuntu 18.04. Installing python3-docker in advance is working for me. Any chance for a fix in 3000.1 ?

234d · 2020-03-06T09:38:45Z

Same issue here on Amazon Linux 2 and salt-minion 3000.
Tired setting both reload_modules and force_reinstall to True without sucess.

max-arnold · 2020-04-17T10:27:19Z

This looks like a potential cause: #56669 (comment)

mattiasr · 2021-05-11T17:43:25Z

I installed python3-docker, no change, then I started a python3 shell and imported docker and got error No module named 'distutils.spawn'

Installed python3-distutils and now the steps works.

Running: 3003+ds-1 on Ubuntu 20.04

devrushit · 2021-05-21T10:01:59Z

still getting this issue with
- python3-distutils
- python3-docker

    googlecompute:           ID: Configure Networks
    googlecompute:     Function: docker_network.present
    googlecompute:         Name: traefik_web
    googlecompute:       Result: False
    googlecompute:      Comment: State 'docker_network.present' was not found in SLS 'docker'
    googlecompute:               Reason: 'docker_network' __virtual__ returned False: 'docker.version' is not available.
    googlecompute:      Changes:

mattiasr · 2021-05-28T14:11:46Z

still getting this issue with

python3-distutils
python3-docker

    googlecompute:           ID: Configure Networks
    googlecompute:     Function: docker_network.present
    googlecompute:         Name: traefik_web
    googlecompute:       Result: False
    googlecompute:      Comment: State 'docker_network.present' was not found in SLS 'docker'
    googlecompute:               Reason: 'docker_network' __virtual__ returned False: 'docker.version' is not available.
    googlecompute:      Changes:

Dump question, but did you restart the service?

rhbvkleef · 2021-08-26T16:00:38Z

I have the same issue with:

SaltStack 3003.2
Python 3.7.3
Debian 10
python3-docker 3.4.1 installed using apt OR docker 5.0.0 installed by pip
python3-requests 2.21.0 installed using apt

I made sure to entirely restart the minion each time I changed the python packages.

This is the trace I get:

An exception occurred in this state: Traceback (most recent call last):
  File "/usr/share/python-wheels/urllib3-1.24.1-py2.py3-none-any.whl/urllib3/util/timeout.py", line 124, in _validate_timeout
    float(value)
TypeError: float() argument must be a string or a number, not 'Timeout'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/docker/api/client.py", line 214, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
  File "/usr/local/lib/python3.7/dist-packages/docker/api/daemon.py", line 181, in version
    return self._result(self._get(url), json=True)
  File "/usr/local/lib/python3.7/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.7/dist-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/share/python-wheels/requests-2.21.0-py2.py3-none-any.whl/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/usr/share/python-wheels/requests-2.21.0-py2.py3-none-any.whl/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.7/dist-packages/raven/breadcrumbs.py", line 341, in send
    resp = real_send(self, request, *args, **kwargs)
  File "/usr/share/python-wheels/requests-2.21.0-py2.py3-none-any.whl/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/share/python-wheels/requests-2.21.0-py2.py3-none-any.whl/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/share/python-wheels/urllib3-1.24.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 587, in urlopen
    timeout_obj = self._get_timeout(timeout)
  File "/usr/share/python-wheels/urllib3-1.24.1-py2.py3-none-any.whl/urllib3/connectionpool.py", line 300, in _get_timeout
    return Timeout.from_float(timeout)
  File "/usr/share/python-wheels/urllib3-1.24.1-py2.py3-none-any.whl/urllib3/util/timeout.py", line 154, in from_float
    return Timeout(read=timeout, connect=timeout)
  File "/usr/share/python-wheels/urllib3-1.24.1-py2.py3-none-any.whl/urllib3/util/timeout.py", line 94, in __init__
    self._connect = self._validate_timeout(connect, 'connect')
  File "/usr/share/python-wheels/urllib3-1.24.1-py2.py3-none-any.whl/urllib3/util/timeout.py", line 127, in _validate_timeout
    "int, float or None." % (name, value))
ValueError: Timeout value connect was Timeout(connect=60, read=60, total=None), but it must be an int, float or None.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/salt/state.py", line 2172, in call
    *cdata["args"], **cdata["kwargs"]
  File "/usr/lib/python3/dist-packages/salt/loader.py", line 1241, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader.py", line 2274, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader.py", line 2289, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader.py", line 2322, in wrapper
    return f(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/states/docker_container.py", line 1687, in running
    image_id = _resolve_image(ret, image, client_timeout)
  File "/usr/lib/python3/dist-packages/salt/states/docker_container.py", line 187, in _resolve_image
    image_id = __salt__["docker.resolve_image_id"](image)
  File "/usr/lib/python3/dist-packages/salt/loader.py", line 1241, in __call__
    return self.loader.run(run_func, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader.py", line 2274, in run
    return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/loader.py", line 2289, in _run_as
    return _func_or_method(*args, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/modules/dockermod.py", line 2030, in resolve_image_id
    inspect_result = inspect_image(name)
  File "/usr/lib/python3/dist-packages/salt/modules/dockermod.py", line 1962, in inspect_image
    ret = _client_wrapper("inspect_image", name)
  File "/usr/lib/python3/dist-packages/salt/modules/dockermod.py", line 437, in wrapper
    __context__["docker.client"] = _get_client(timeout=timeout, **kwargs)
  File "/usr/lib/python3/dist-packages/salt/modules/dockermod.py", line 397, in _get_client
    ret = docker.APIClient(**client_kwargs)
  File "/usr/local/lib/python3.7/dist-packages/docker/api/client.py", line 197, in __init__
    self._version = self._retrieve_server_version()
  File "/usr/local/lib/python3.7/dist-packages/docker/api/client.py", line 222, in _retrieve_server_version
    'Error while fetching server API version: {0}'.format(e)
docker.errors.DockerException: Error while fetching server API version: Timeout value connect was Timeout(connect=60, read=60, total=None), but it must be an int, float or None.

Looking at this trace, it seems that requests submits a Timeout instance (called TimeoutSauce in the requests source) to the urlopen method, which is supposed to accept such a Timeout instance. Somehow this Timeout instance is not an instance of the Timeout class, as checked by if isinstance(timeout, Timeout): in _get_timeout? I would say that this is either an issue of urllib3 not accepting a Timeout instance even though it is supposed to, or requests sending an instance of the incorrect Timeout class.

I have verified that I only have one version of urllib3 and requests installed:

--REDACTED--:/usr/local/lib/python3.7/dist-packages> sudo find / -type d -name "requests" 2>/dev/null
/usr/lib/python2.7/dist-packages/requests
/usr/lib/python3/dist-packages/requests
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests
/usr/local/lib/python3.4/dist-packages/pip/_vendor/requests
--REDACTED--:/usr/local/lib/python3.7/dist-packages> sudo find / -type d -name "urllib3" 2>/dev/null
/usr/lib/python2.7/dist-packages/urllib3
/usr/lib/python3/dist-packages/urllib3
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3
/usr/local/lib/python3.4/dist-packages/pip/_vendor/requests/packages/urllib3

DaAwesomeP · 2021-08-29T21:54:07Z

I am in nearly the exact same environment as @rhbvkleef and have the same issue. I have tried both python3-docker via apt and docker==4.4.4, urllib3==1.26.6, and requests==2.26.0 via pip. However, I cannot get it to work at all (I always see the above issue).

Another issue mentions installing docker, urllib3, and requests via pip, but I think this might not work because the salt apt packages depend on python3-requests, etc.

DaAwesomeP · 2021-09-03T18:38:51Z

I think some of the problem may be in that salt is installed via the system package repository and relies on packages like python3-requests and python3-urllib3 of whichever version the OS repo provides. However, the Python docker library is installed via pip and and brings its own versions of requests and urllib3 which salt may not actually use. Docker no longer offers docker-py as an system package; it can only be installed as a pip package.

The solution to isolate these would be to either

Reinstall salt from pip (not ideal)
Move docker-py to a virtualenv and run some other salt connector process in that virtualenv (proper separation, most ideal)
Somehow force salt to use the pip versions of urllib3 and requests

rhbvkleef · 2021-09-04T20:21:21Z

@DaAwesomeP, if you read my previous comment, you'll see that requests and urllib3 are only installed once each, so there is no such conflict, although somehow the stacktrace looks like that's the only option. Is my find not complete somehow? Where else could these packages be installed?

DaAwesomeP · 2021-09-04T20:36:29Z

Saltstack depends on python3-requests at the OS level (i.e. this is install by APT on Debian as it is a dependency of salt-common). These are installed to /usr/lib/python3/dist-packages/ .You will also notice a urllib3 and requests here if you run apt-get install python3-urllib3 python3-requests (though requests should already be there if salt is installed).

However, pip is able to provide newer dependencies when it installs packages. These are stored in /usr/local/lib/python3.7/dist-packages/ on Debian 10. This is where the docker library is installed from pip. You will also notice a urllib3 and requests here if you run pip3 install urllib3 requests. These should be here if you installed docker via pip.

So I think what may be happening, and maybe I am very wrong here, but salt-common installed via APT may be preferring the python3-requests library over the pip-installed requests library. But the docker library installed via pip expects the newer version that pip installed (as a dependency for docker) and so is generating this error.

There is always a danger in mixing and matching pip and os-provided Python libraries for this reason: there is no version matching between the two systems (apt and pip). I think this may not have used to be an issue because Docker used to ship a docker-py OS/apt package (which would properly depend on the right version of python3-requests), but now they have moved to pip only (probably for greater system/OS compatibility and newer dependency versions).

Typically for CLI tools and individual applications the solution would be to isolate docker into a virtualenv/venv, but since it needs to be accessed programmatically via Python, that may not be possible.

The best solution I can think of would be to introduce a salt-docker-connector package to pip/Pypi. This package would depend on docker on Pypi. It would provide a connection between the docker package and salt via non-Python programmatic means (i.e. socket, bus, CLI, etc.). This way docker and salt-docker-connector could live in their own in a separate virtualenv and not mix with the salt or other system dependencies. This would also allow for salt-docker-connector to explicitly mark which versions of the docker library it is compatible with, which would probably help with avoiding breaking changes in the docker library.

whytewolf · 2021-10-25T17:57:33Z

I'm having trouble replicating this issue with the latest 3004. the reload_modules seems to have taken without the need to reload and I am not getting the tracepath shown in #54449 (comment) when with python3-requests at 2.21.0 and python3-docker installed with pkg.installed at 3.4.1 on debian 10.

Can someone give a full testcase that replicates the issue?

dmurphy18 · 2022-01-14T00:52:47Z

There is no response to the issue for some months, if none is received soon, the issue will be closed as considered dormant, and no longer relevant.

DaAwesomeP · 2022-02-02T20:24:04Z

@whytewolf where did you install the python3-docker module from? Did you install the Debian-provided Docker or the one from the Docker APT repo?

whytewolf · 2022-02-02T20:48:23Z

docker apt doesn't supply python3-docker https://download.docker.com/linux/debian/dists/buster/pool/stable/amd64/ that can only come from the debian repos.

but i did use the docker apt repos for docker itself

for the record. I also run a debian 11 system that has a pip installed version of docker. that runs fine. without issue.

whytewolf · 2022-02-14T18:14:19Z

Closing this issue since no one has been able to give a test case that fails, and I have tried the suggested test cases. as well as some things not suggested in an attempt to get it to fail. If you are still having this issue with a 3004 or later version then please open a new ticket.

keithpjolley · 2022-08-23T14:03:40Z

When trying to use the docker_container.running state the 'docker.version' not available error hit. Resolved by reading the fine manual which says to run salt minion pip.install docker. After this the error went away. Will put that as a prerequisite in the .sls file.

markdueck1 · 2022-10-12T01:18:08Z

I have recently run into this on raspbian bullseye, with the latest salt (3005.1). This time the pip docker version 6 is the issue, downgrading it to 5.0.3 fixes the issue for me.

whytewolf · 2022-10-12T01:39:38Z

I have recently run into this on raspbian bullseye, with the latest salt (3005.1). This time the pip docker version 6 is the issue, downgrading it to 5.0.3 fixes the issue for me.

the issue with docker 6 is not related to this issue. docker 6.0.0 broke how they handle versioning which is how salt determines if a proper docker python module is installed. that is much different than what was causing this issue which is loader issues and python path based.

* comment out geerlingguy.docker and disable docker images preload. geerlingguy.docker (5:23.0.1-1~ubuntu.20.04~focal) and preloading of docker images may be an issue as securityonion is pinning older version (5:20.10.8~3-0~ubuntu-focal). trying to force version in geerlingguy.docker did not work. * use python3-docker (4.1.0-1) like securityonion and not pip docker (6.0.1). else noise `Reason: 'docker_container' __virtual__ returned False: 'docker.version' is not available.` saltstack/salt#54449 (comment) https://github.com/saltstack/salt/pull/62882/files * give more retries for so-setup ~2h30

garethgreenaway added the needs-triage label Oct 14, 2019

garethgreenaway assigned xeacott Oct 14, 2019

xeacott added v2019.2 unsupported version Bug broken, incorrect, or confusing behavior P4 Priority 4 and removed needs-triage labels Oct 30, 2019

sagetherage added this to the Approved milestone Dec 17, 2019

stale bot added the stale label Jan 16, 2020

stale bot removed the stale label Jan 20, 2020

lsh-0 mentioned this issue Jan 20, 2020

python 3 upgrade elifesciences/iiif-formula#17

Closed

lsh-0 mentioned this issue Jan 20, 2020

Feature: containerised loris elifesciences/iiif-formula#18

Merged

8 tasks

Ch3LL added the ZRelease-Sodium retired label label Mar 18, 2020

sagetherage assigned waynew and unassigned xeacott Mar 19, 2020

sagetherage removed the v2019.2 unsupported version label Apr 1, 2020

sagetherage removed the ZRelease-Sodium retired label label May 11, 2020

sagetherage added severity-medium 3rd level, incorrect or bad functionality, confusing and lacks a work around and removed severity-high 2nd top severity, seen by most users, causes major problems labels Mar 24, 2021

sagetherage modified the milestones: Silicon, Approved Aug 4, 2021

ali-ghasempor mentioned this issue Sep 9, 2021

Salt is unable to communicate and execute Docker functionalities #60856

Closed

anilsil assigned whytewolf and dmurphy18 Oct 20, 2021

anilsil removed the Silicon v3004.0 Release code name label Oct 20, 2021

anilsil modified the milestones: Approved, Phosphorus v3005.0 Oct 20, 2021

dmurphy18 added the Phosphorus v3005.0 Release code name and version label Oct 20, 2021

whytewolf closed this as completed Feb 14, 2022

dariosusman mentioned this issue Nov 1, 2022

[BUG] docker_image.present erroring out: docker_image' __virtual__ returned False: 'docker.version' is not available. #62602

Closed

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docker_image.present fails first time when python3-docker module is installed #54449

docker_image.present fails first time when python3-docker module is installed #54449

golpa commented Sep 10, 2019

xeacott commented Oct 30, 2019

xeacott commented Oct 30, 2019

xpicio commented Nov 29, 2019

stale bot commented Jan 16, 2020

lsh-0 commented Jan 20, 2020

stale bot commented Jan 20, 2020

lsh-0 commented Jan 20, 2020

darkn3rd commented Feb 6, 2020 •

edited

Loading

supfors commented Feb 21, 2020

234d commented Mar 6, 2020

max-arnold commented Apr 17, 2020

mattiasr commented May 11, 2021

devrushit commented May 21, 2021

mattiasr commented May 28, 2021

rhbvkleef commented Aug 26, 2021 •

edited

Loading

DaAwesomeP commented Aug 29, 2021 •

edited

Loading

DaAwesomeP commented Sep 3, 2021

rhbvkleef commented Sep 4, 2021

DaAwesomeP commented Sep 4, 2021 •

edited

Loading

whytewolf commented Oct 25, 2021

dmurphy18 commented Jan 14, 2022

DaAwesomeP commented Feb 2, 2022

whytewolf commented Feb 2, 2022 •

edited

Loading

whytewolf commented Feb 14, 2022

keithpjolley commented Aug 23, 2022 •

edited

Loading

markdueck1 commented Oct 12, 2022

whytewolf commented Oct 12, 2022

docker_image.present fails first time when python3-docker module is installed #54449

docker_image.present fails first time when python3-docker module is installed #54449

Comments

golpa commented Sep 10, 2019

Description of Issue

Setup

Steps to Reproduce Issue

Versions Report

xeacott commented Oct 30, 2019

xeacott commented Oct 30, 2019

xpicio commented Nov 29, 2019

stale bot commented Jan 16, 2020

lsh-0 commented Jan 20, 2020

stale bot commented Jan 20, 2020

lsh-0 commented Jan 20, 2020

darkn3rd commented Feb 6, 2020 • edited Loading

supfors commented Feb 21, 2020

234d commented Mar 6, 2020

max-arnold commented Apr 17, 2020

mattiasr commented May 11, 2021

devrushit commented May 21, 2021

mattiasr commented May 28, 2021

rhbvkleef commented Aug 26, 2021 • edited Loading

DaAwesomeP commented Aug 29, 2021 • edited Loading

DaAwesomeP commented Sep 3, 2021

rhbvkleef commented Sep 4, 2021

DaAwesomeP commented Sep 4, 2021 • edited Loading

whytewolf commented Oct 25, 2021

dmurphy18 commented Jan 14, 2022

DaAwesomeP commented Feb 2, 2022

whytewolf commented Feb 2, 2022 • edited Loading

whytewolf commented Feb 14, 2022

keithpjolley commented Aug 23, 2022 • edited Loading

markdueck1 commented Oct 12, 2022

whytewolf commented Oct 12, 2022

darkn3rd commented Feb 6, 2020 •

edited

Loading

rhbvkleef commented Aug 26, 2021 •

edited

Loading

DaAwesomeP commented Aug 29, 2021 •

edited

Loading

DaAwesomeP commented Sep 4, 2021 •

edited

Loading

whytewolf commented Feb 2, 2022 •

edited

Loading

keithpjolley commented Aug 23, 2022 •

edited

Loading