Support for Redis Cluster

[pcejrowski]

Hi, does this project support Redis cluster configuration?

[KarelCemus]

Hi, not right now, there was no need so far. Though the configuration is quite opened, I hope. What exactly do you need? Feel free to propose the default configuration or highlight missing components or configuration properties.

[pcejrowski]

Ok, thanks.
I thought about something like it's done in scredis

I've just started reading the code and I think my PR for this will take place.

[KarelCemus]

Oh, I see. This might require more significant internal change, give me couple days, I will look at it myself.

But I'd appreciate if you provide me a proposal of configuration, e.g., how it could look in the configuration file or so. I haven't used the cluster for real.

Would it work for you?

[pcejrowski]

Yes, sure.

I think, that simple HOCON list might be the most natural way.
For more advanced configuration of nodes this set of values might be a good insipration.

play {
  cache {
    redis {
	  cluster [
	  	{ host: "redis.node-1.org", port: 6383},
	  	{ host: "redis.node-2.org", port: 6383}
	  ]
	  ...
    }
  }
}

[KarelCemus]

Ok, I'll look at it hopefully around a weekend.

[KarelCemus]

I looked at it and there are several issues.

First, scredis recently moved into the new repository. The latest released version in the original repository is 2.0.8, but it does not have merged the cluster implementation.

Second, the latest released version in the new repository is 2.0.6, which also does not implement it. However, there is unreleased 2.1.0-SNAPSHOT version having the clusters merged and implemented support of Scala 2.12.

Third, to make things more complicated, it turned out the scredis repository is basically dead. They accept and merge PRs, but there is no active development. Furthermore, there is not even established the release process, so releasing 2.1.0 is quite challenging.

Finally, as I looked, their implementation of clusters does not support authentication.

So, to sum it up:

• there is no released version implementing clusters
• the repository has no active development but they react to issues and accept PRs
• there is no release process so releasing 2.1.0 is challenging
• support of Scala 2.12 is becoming important as the release of Play 2.6 is coming.
• their implementation of clusters does not support authentication.

What is next

We have a major decision to make. The current version of scredis is insufficient as at least support of Scala 2.12 is required. There are alternatives I have in mind

1. we help them to set up the release process and have 2.1.0 released. That will be enough for a while but it would have only partial implementation of clusters
2. I will fork their repo and publish it myself, this is a fallback to the previous case
3. We will change the internal engine again, but I am not aware of any other async implementation with active development. Furthermore, changing the engine twice in that short term seems a bit disturbing.

So, what do you suggest to do?

[KarelCemus]

In case of chaging the connector (again), I am thinking of etaty/rediscala. It does not seem much more active than scredis but there are many more contributors, GitHub stars, and implemented features including clusters.

[pcejrowski]

Hi Karel,

thanks for your deep analysis! Terrible story...

In my opinion still option 3 should be optimal. The problem is that we cannot predict which client is going to be maintained properly.

If you decide how to share tasks, I can happily help you with development.

PS. In my team, we use etaty/rediscala from Spark applications and it works well.

[KarelCemus]

Alright then, I'll look at rediscala more deeply to estimate the efforts and let you know the roadmap of migration.

Should I release it for Play 2.5.x or it would wait til 2.6.x?

[pcejrowski]

I think that support for 2.5 would be nice, even after 2.6 is published, since they are changing backend (as well...), thus some projects won't be migrated just immediately.

In my case, it doesn't matter, because I won't have it on production before 2.6 is published, so I am going to upgrade to 2.6 before my first release.

[KarelCemus]

Well, support is one thing, new features (redis cluster) the other, which is why I ask. I believe that current revision for 2.5.x works fine.

[pcejrowski]

Yes, sure. It works perfectly. You are right in aiming in 2.6.x.

[pcejrowski]

Moreover, as Play 2.6.x provides two API for sync and async operations on the DI level (see: docs ), even more changes are necessary.

[KarelCemus]

I am aware of it. I haven't check yet but I am curious about the compatibility with my APIs. I hope the transition will be smooth, but we will see.

Anyway, next release will be huge for play-redis: Scala 2.12, Play 2.6 with new APIs and probably new internal connector. Maybe I should start working on it :)

[pcejrowski]

Yup, sounds like a huge contribution.

If it was up to me, I would have split those releases:

1. Play 2.6 on Scala 2.11
2. Support for Scala 2.12 after the connector supports it.

Best!

[KarelCemus]

Well, it seems rediscala already supports 2.12, so after the transition to Play 2.6, the support of Scala 2.12 should be really simple.

[pcejrowski]

Ohhh, sorry, I didn't check that.

[KarelCemus]

I am sorry, I was a bit short on time and didn't manage to implement it for 1.5.0, which I have to release now as Play 2.6 is out. I'm postponing it into next release, which I hope will be soon. There are some things I need to address.