ssl.SSLWantReadError in version 3.3.0

[liormizr]

Version: 3.3.0

Platform: Python3.7 aws Lambda and Python 3.6 Ubuntu 18.04

Description:

Hello

We upgraded today version to the last one - 3.3.0 from 3.2.1.
When we did that we started to get ssl error.

reverting version to 3.2.1 fix the issue...

code:

from redis import StrictRedis
cache = StrictRedis(
    host=SESSION_CACHE_HOST,
    db=int(SESSION_CACHE_DB),
    password=SESSION_CACHE_PASS,
    ssl=True)
print(cache.keys())

traceback:

Traceback (most recent call last):
...
 File "/var/task/redis/client.py", line 1307, in exists
 return self.execute_command('EXISTS', *names)
 File "/var/task/redis/client.py", line 836, in execute_command
 conn = self.connection or pool.get_connection(command_name, **options)
 File "/var/task/redis/connection.py", line 1049, in get_connection
 if connection.can_read():
 File "/var/task/redis/connection.py", line 666, in can_read
 return self._parser.can_read(timeout)
 File "/var/task/redis/connection.py", line 280, in can_read
 return self._buffer and self._buffer.can_read(timeout)
 File "/var/task/redis/connection.py", line 190, in can_read
 raise_on_timeout=False)
 File "/var/task/redis/connection.py", line 159, in _read_from_socket
 data = recv(self._sock, socket_read_size)
 File "/var/task/redis/_compat.py", line 58, in recv
 return sock.recv(*args, **kwargs)
 File "/var/lang/lib/python3.7/ssl.py", line 1037, in recv
 return self.read(buflen)
 File "/var/lang/lib/python3.7/ssl.py", line 913, in read
 return self._sslobj.read(len)
ssl.SSLWantReadError: The operation did not complete (read) (_ssl.c:2488)

[delijati]

Got the same error in AWS Beanstalk ...

[jzillioux]

Also seeing this behavior in AWS Lambda Python3.7.

[andymccurdy]

Based on the traceback I suspect that SSL connections and non-SSL connections raise different exceptions in nonblocking mode. I’ll have a hot fix out in a few hours.

[andymccurdy]

Confirmed my suspicion here: https://docs.python.org/3/library/ssl.html under notes about nonblocking sockets.

I’ll work on a fix when I’m at my desk.

[andymccurdy]

Just pushed a fix to Github. Will release 3.3.1 once Travis gives the all clear.

[andymccurdy]

redis-py 3.3.1 is now available on PyPI. Thanks for reporting this. Please confirm that this fixes your issue.

[jzillioux]

Still getting an error, however it is slightly different than before.
Code:

import redis

r = redis.StrictRedis(host=host, port=port, password=pw, ssl=True)
r.keys('*')

traceback:

Traceback (most recent call last):
  File "test-redis.py", line 8, in <module>
    r.keys('*')
  File "/tmp/env/lib/python3.6/site-packages/redis/client.py", line 1387, in keys
    return self.execute_command('KEYS', pattern)
  File "/tmp/env/lib/python3.6/site-packages/redis/client.py", line 836, in execute_command
    conn = self.connection or pool.get_connection(command_name, **options)
  File "/tmp/env/lib/python3.6/site-packages/redis/connection.py", line 1060, in get_connection
    if connection.can_read():
  File "/tmp/env/lib/python3.6/site-packages/redis/connection.py", line 677, in can_read
    return self._parser.can_read(timeout)
  File "/tmp/env/lib/python3.6/site-packages/redis/connection.py", line 291, in can_read
    return self._buffer and self._buffer.can_read(timeout)
  File "/tmp/env/lib/python3.6/site-packages/redis/connection.py", line 201, in can_read
    raise_on_timeout=False)
  File "/tmp/env/lib/python3.6/site-packages/redis/connection.py", line 170, in _read_from_socket
    data = recv(self._sock, socket_read_size)
  File "/tmp/env/lib/python3.6/site-packages/redis/_compat.py", line 58, in recv
    return sock.recv(*args, **kwargs)
  File "/usr/lib/python3.6/ssl.py", line 997, in recv
    return self.read(buflen)
  File "/usr/lib/python3.6/ssl.py", line 874, in read
    return self._sslobj.read(len, buffer)
  File "/usr/lib/python3.6/ssl.py", line 633, in read
    v = self._sslobj.read(len)
ssl.SSLWantReadError: The operation did not complete (read) (_ssl.c:2305)

[andymccurdy]

@jzillioux Thanks. I'm looking into this now. I have an assumption and need to setup an SSL connection to test.

[andymccurdy]

I just pushed another fix. SSL connections in non-blocking mode raise SSL-specific exceptions to indicate that there is no data to read on the socket. Additionally, they have their own error numbers which do not correspond to non-SSL socket error numbers.

I've pushed 3.3.2 to PyPI. Please let me know if this fixes the issue for you.

[JokerQyou]

I'm getting an ImportError related to the SSLWantReadError class. I use Python 2.7.6 so the ssl module in stdlib is known to be outdated. But I did not find anything regarding dropping support for Python < 2.7.9 in the changelog, so I think this still count as a regression?

File "eggs/redis-3.3.2-py2.7.egg/redis/__init__.py", line 1, in <module>
    from redis.client import Redis, StrictRedis
File "eggs/redis-3.3.2-py2.7.egg/redis/client.py", line 13, in <module>
    from redis.connection import (ConnectionPool, UnixDomainSocketConnection,
File "eggs/redis-3.3.2-py2.7.egg/redis/connection.py", line 42, in <module>
    ssl.SSLWantReadError,
AttributeError: 'module' object has no attribute 'SSLWantReadError'

The same code worked with redis==3.3.0.

This seems to be hard to monkey patch from my side, because a simple line import redis would raise ImportError, so there is no way to get version number of redis ahead.

[jzillioux]

Tested with python 3.6.8 and python 2.7.15 on Ubuntu 18.04 and the changes in 3.3.2 resolved the issues I was seeing.

[andymccurdy]

@jzillioux Awesome, thanks for testing.

@JokerQyou Thanks for reporting this. I'll install an older 2.7.x release and see how difficult it will be to support the ssl module in Python < 2.7.9.