Upgrade kafka 2.6.0

[sunshine69]

No description provided.

[sunshine69]

This has been a request #1137 and we are currently suffering from this kafka for too long.

[sunshine69]

Please note that I have tested this PR and successfully build a epiph cluster with kafka version 2.6.0.

From the original request (not mine) I saw that we are not planned for 0.8.0 but it is not clear to me the reason. Can someone please share a bit more about it? I would like to know what is the technical issues, or procedure issues that prevent us to move forward?

Thanks

[seriva]

Hey

Few minor comments:

• Missing changelog entry
• Update version in component list (https://github.com/epiphany-platform/epiphany/blob/develop/docs/home/COMPONENTS.md#epiphany-cluster-components)

Few bigger things:

• Missing upgrade for existing clusters, this will break existing installations.
• We need to check Zookeeper as well. If I remember there are some issues regarding compatibility or do we even still need it? (Upgrade ZooKeeper #1136)

Cheers
Luuk

[sunshine69]

Thanks Luuk, I will look into it as soon as I can

[sunshine69]

@seriva Are we meant to support upgrade for kafka? To my best of knowledge we don't support it even for current - it is complex process.

Re-run epiphany and update component is still a test/try progress atm and depending on many components, some might work, some might not. I saw a page somewhere listing state of it and I remember we mostly don't support it globally. Am I correct?

I will try to look into upgrade as well by the way admiting it would be very complex situation .

[seriva]

@sunshine69 Sorry for the late reply but yes this upgrade should be taken into consideration looking at the issue Tomasz made for that #1137

Otherwise existing cluster might have issues.

[plirglo]

Hi @sunshine69

As @seriva mentioned while changing version of component we should also create automation to upgrade this component (by running epicli upgrade).

To see how it was done for upgrade between Kafka 2.0.0 to recent 2.3.1 please look at this part of upgrade code here and ansible tasks located here.

Also with Kafka 2.6 Apache released new version of Zookeeper here, so in this case we should also upgrade Zookeeper to keep configuration consistent.

Regards,
Irek

[erzetpe]

Why you have chosen this mirror?

[erzetpe]

Why you have chosen this mirror?

[atsikham]

This PR seems to be ok as we already do Zookeeper upgrade in another task and there is Kafka upgrade part in upgrade role, but @erzetpe questions still have no answers and there are no components/changelog updates. We support only offline upgrade and I see no changes to make there.

[atsikham]

@toszo so this PR can be closed as implemented in #1890?