Add SSL support for Redis Cluster using jedis [DATAREDIS-974]

[spring-projects-issues]

Deep Shiv opened DATAREDIS-974 and commented

Hello,

I'm trying to connect to AWS ElastiCache Redis using Spring Data Redis + Jedis combination. [Redis Cluster enabled, so it has Cluster Config endpoint, with 3 shard - each shard has 1 primary node + 2 replica nodes ]

I'm getting Read timed out error.

Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out

{{}}

AWS Redis Server Version : 5.0.3 / Cluster Mode : Enabled / SSL : Enabled / Auth : Enabled ( by password )

Library -- Spring-data-redis : 2.1.6.Release / jedis : 2.9.0

Telnet works to AWS Redis all nodes and cluster config endpoint at 6379 ports.

So, no issues with Redis itself, issue with Spring Data Redis in combination with Jedis.

This is how my code looks like for Cluster 

      RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration();
       redisClusterConfiguration.setClusterNodes(listOfRedisNode);
       redisClusterConfiguration.setPassword(passwordString);


       JedisClientConfiguration.JedisClientConfigurationBuilder jedisClientConfiguration = JedisClientConfiguration.builder();
       jedisClientConfiguration.connectTimeout(Duration.ofSeconds(60));
       jedisClientConfiguration.useSsl();
       jedisClientConfiguration.usePooling();


       JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory(redisClusterConfiguration, jedisClientConfiguration.build() );
       jedisConnectionFactory.afterPropertiesSet();

       final RedisTemplate<String, Serializable> redisTemplate = new RedisTemplate<>();
       redisTemplate.setConnectionFactory(jedisConnectionFactory);
       redisTemplate.setKeySerializer(new JdkSerializationRedisSerializer());
       redisTemplate.setValueSerializer(new JdkSerializationRedisSerializer());
       redisTemplate.afterPropertiesSet();

       System.out.println(redisTemplate.getClientList().size());

       StringRedisConnection stringRedisConnectionlettuce = new DefaultStringRedisConnection(redisTemplate.getConnectionFactory().getConnection());
       final String message2 = stringRedisConnectionlettuce.echo("Hello");
       System.out.println("Hello".equals(message2));

 

Error Stack 

Exception in thread "main" java.lang.reflect.InvocationTargetException
       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
       at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       at java.lang.reflect.Method.invoke(Method.java:498)
       at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
       at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
       at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
       at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
       Caused by: redis.clients.jedis.exceptions.JedisConnectionException: java.net.SocketTimeoutException: Read timed out
       at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:202)
       at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40)
       at redis.clients.jedis.Protocol.process(Protocol.java:151)
       at redis.clients.jedis.Protocol.read(Protocol.java:215)
       at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:340)
       at redis.clients.jedis.Connection.getStatusCodeReply(Connection.java:239)
       at redis.clients.jedis.BinaryClient.connect(BinaryClient.java:96)
       at redis.clients.jedis.Connection.sendCommand(Connection.java:126)
       at redis.clients.jedis.Connection.sendCommand(Connection.java:117)
       at redis.clients.jedis.BinaryClient.auth(BinaryClient.java:564)
       at redis.clients.jedis.BinaryJedis.auth(BinaryJedis.java:2138)
       at redis.clients.jedis.JedisClusterConnectionHandler.initializeSlotsCache(JedisClusterConnectionHandler.java:36)
       at redis.clients.jedis.JedisClusterConnectionHandler.<init>(JedisClusterConnectionHandler.java:17)
       at redis.clients.jedis.JedisSlotBasedConnectionHandler.<init>(JedisSlotBasedConnectionHandler.java:24)
       at redis.clients.jedis.BinaryJedisCluster.<init>(BinaryJedisCluster.java:54)
       at redis.clients.jedis.JedisCluster.<init>(JedisCluster.java:93)
       at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.createCluster(JedisConnectionFactory.java:423)
       at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.createCluster(JedisConnectionFactory.java:393)
       at org.springframework.data.redis.connection.jedis.JedisConnectionFactory.afterPropertiesSet(JedisConnectionFactory.java:350)
       at io.github.deepshiv126.springdataredis.example.MySpringBootApplication.main(MySpringBootApplication.java:208)
... 8 more
       Caused by: java.net.SocketTimeoutException: Read timed out
       at java.net.SocketInputStream.socketRead0(Native Method)
       at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
       at java.net.SocketInputStream.read(SocketInputStream.java:171)
       at java.net.SocketInputStream.read(SocketInputStream.java:141)
       at java.net.SocketInputStream.read(SocketInputStream.java:127)
       at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:196)
... 27 more

---

 I looked into Spring Source Code and Jedis Source Code -- My assumption its Jedis not using SSL Connection ;

JedisConnectionFactory - afterPropertiesSet() --> trying to create Cluster --> under that it's trying initializeSlotsCache, which issued AUTH command to Redis Server, with password -- This is where "Read timed out" is occuring;

JedisClusterConnectionHandler - isnt suppose to use Jedis with SSL??

!https://user-images.githubusercontent.com/12883918/56941800-ade6cc00-6acb-11e9-8671-169e4e260ef4.png|width=675,height=197!

I understand local redis - you can go inside and run auth command to get authenticate. But I guess AWS Redis may not able to do that , its needs to have SSL Connection even before it runs AUTH command - Why Jedis is not using SSL Connection ?

I found another suggestion to use JedisPool - but spring-data-redis' JedisConnectionFactory doesn't accepts JedisPool. Is there any other way to do that ?

JedisPool jedisPool = new JedisPool("rediss://" + clusterConfigEndPoint + ":6379");

Another question - other libraries use redis ssl connection as rediss:// - how to Jedis Client to use SSL connection??

---

Attachments:

• image-2019-05-01-17-46-18-027.png (77.42 kB)

Issue Links:

• DATAREDIS-1015 Upgrade to Jedis 3.1.0
  ("depends on")
• DATAREDIS-575 Support client name for JedisCluster
  ("supersedes")

Referenced from: pull request #472, and commits ac4909f, 849091d

6 votes, 6 watchers

[spring-projects-issues]

Deep Shiv commented

JedisConnectionFactory - afterPropertiesSet()-- trying to create Cluster -- under that its trying initializeSlotsCache, which issued AUTH command, with password -- This is where "Read timed out" is occuring;

I understand local redis - you can go inside and run auth command to get authenticate. But I guess AWS Redis need SSL Connection, but I dont seems this program is using SSL Connection. 

redis.clients.jedis.JedisClusterConnectionHandler – this Jedis is not using SSL Connection.

!image-2019-04-29-20-24-15-649.png!

 

Other Library ( like Redisson ) uses URI with SSL - something like this - rediss://hostname:port/ .
Does Jedis uses same way to connect to Redis ?

 

[spring-projects-issues]

Deep Shiv commented

Using JedisShardInfo  - it perfectly works - BUT its deprecated. 

 

JedisShardInfo info = new JedisShardInfo("rediss://:" + password + "@dev-r63-0001-001.dev-r63.3mlfp2.use1.cache.amazonaws.com:6379");
JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory(info);


 final RedisTemplate<String, Serializable> redisTemplate = new RedisTemplate<>();
 redisTemplate.setConnectionFactory(jedisConnectionFactory);
 redisTemplate.setKeySerializer(new JdkSerializationRedisSerializer());
 redisTemplate.setValueSerializer(new JdkSerializationRedisSerializer());
 redisTemplate.afterPropertiesSet();


 StringRedisConnection stringRedisConnectionlettuce = new DefaultStringRedisConnection(redisTemplate.getConnectionFactory().getConnection());
 final String message2 = stringRedisConnectionlettuce.echo("Hello");
 System.out.println("Hello".equals(message2));

 redisTemplate.opsForValue().set("d1", "value111");
 System.out.println("FINALLY :: "+ redisTemplate.opsForValue().get("d1"));// code placeholder

 

This is clearly a bug. Isn't it ?

 

[spring-projects-issues]

Deep Shiv commented

After spending a lot of hours - I have figured out the issue 

1. Jedis added SSL Support for Cluster - very recently they merged it 
   1. https://github.com/xetorthio/jedis/blob/master/src/main/java/redis/clients/jedis/JedisCluster.java
   2. Add SSL support to JedisCluster redis/jedis#1896
2. Spring Data Redis - NOT SUPPORTING SSL support for cluster. 
3. !image-2019-05-01-17-46-18-027.png!

 

 

Question - When do Spring Plan to provide SSL support for Cluster ? 

 

[spring-projects-issues]

hradilf commented

Hi, please, is there any estimation available?

[spring-projects-issues]

hradilf commented

Hi Deep Shiv,

you mentioned that it works with JedisShardInfo. Please, which version of Jedis & Spring data did you use?

Thank you

[spring-projects-issues]

Mark Paluch commented

Jedis 3.1 is not released yet therefore we cannot provide SSL support. Please use Lettuce in the meantime

[spring-projects-issues]

Deep Shiv commented

Mark Paluch / Christoph Strobl - Jedis had made official release that support "SSL with Cluster"; 
https://github.com/xetorthio/jedis/releases/tag/jedis-3.1.0

Any tentative date when Spring Data Redis will provide support for Jedis ?

[spring-projects-issues]

Christoph Strobl commented

thanks for the ping!

[spring-projects-issues]

Deep Shiv commented

Any tentative date, when Spring will make an official release with Cluster+SSL Support?

We moved to Lettuce, but we are having a lot of Performance issues with Lettuce Library ( no offense, but it is what is and we are having problems with lettuce lib).

So, your dates will really help us to align our goals.  

Thanks

Deepak

[spring-projects-issues]

Christoph Strobl commented

Please have a look at planned release dates in our wiki.
If you've got time please report issues with lettuce here