Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

6.7 docker build_type breaking 6.6 client #40511

Closed
brackxm opened this issue Mar 27, 2019 · 13 comments · Fixed by #40705
Closed

6.7 docker build_type breaking 6.6 client #40511

brackxm opened this issue Mar 27, 2019 · 13 comments · Fixed by #40705
Labels
>bug team-discuss

Comments

@brackxm
Copy link
Contributor

brackxm commented Mar 27, 2019

a 6.6 high level rest client cannot parse the MainResponse from a 6.7 server
due to the new docker build_type (#39378 )
previously a docker distribution was returning the tar type

this prevents 6.6 -> 6.7 upgrades with server before client
client before server upgrades are also not possible due to other 7.0 forward compatibility changes in 6.7
this leaves no 6.6 -> 6.7 or 7 upgrade path
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features

@javanna javanna added the >bug label Mar 27, 2019
@javanna
Copy link
Member

javanna commented Mar 27, 2019

Thanks for reporting this @brackxm , this is indeed a problem. We will discuss possible solutions with the team.

@javanna
Copy link
Member

javanna commented Mar 27, 2019

cc @hub-cap I marked this team-discuss .

@jasontedor
Copy link
Member

Thanks for reporting this @brackxm and sorry for the issue. As @javanna mentions, we will consider some options and see what we can come up with.

hub-cap added a commit to hub-cap/elasticsearch that referenced this issue Apr 1, 2019
@hub-cap
HLRC: Convert xpack methods to client side objects
0b7d88f 
This commit fixes a problem with BWC that was brought up in elastic#40511. A
newer version of the code was emitting a new value for an enum to an
older version, and the older version could not handle that. It caused
the response to error. The MainResponse is now relaxed, and will accept
whatever values the server expose, and holds most of them as Strings
instead of complex objects.

Fixes elastic#40511
@hub-cap hub-cap mentioned this issue Apr 1, 2019
Merged
@hub-cap
Copy link
Contributor

hub-cap commented Apr 1, 2019

Hi @brackxm, Thank you for pointing this out. This did not get caught due to some testing inefficiencies, which I will be hopefully addressing soon. But in the meantime I plan on going thru all of our response parsers manually to see if they suffer from this issue potentially, and if so, correct them such they will not hamper compatibility between versions. #40705 is the first which addresses the parsing problem of this class.

@iremmats
Copy link

iremmats commented Apr 2, 2019

Would there be any risc overriding the /usr/share/elasticsearch/bin/elasticsearch-env file and at the bottom of it change ES_DISTRIBUTION_TYPE=docker to ES_DISTRIBUTION_TYPE=tar? I just tried this and it made it possible to connect a 6.7.0 node to a 6.6.2 cluster.

DaveCTurner added a commit to DaveCTurner/elasticsearch that referenced this issue Apr 2, 2019
@DaveCTurner
Use TAR instead of DOCKER build type before 6.7.0
3ccc4ac 
In 6.7.0 (elastic#39378) we added a build type of DOCKER for the docker images, but
unfortunately earlier versions do not understand this and will reject any
transport messages that mention this build type.

This commit fixes this by reporting TAR instead of DOCKER when talking to older
nodes.

Relates (but does not fix) elastic#40511
Relates elastic#39378
@DaveCTurner DaveCTurner mentioned this issue Apr 2, 2019
Merged
DaveCTurner added a commit that referenced this issue Apr 2, 2019
@DaveCTurner
Use TAR instead of DOCKER build type before 6.7.0 (#40723)
2ff0bdd 
In 6.7.0 (#39378) we added a build type of DOCKER for the docker images, but
unfortunately earlier versions do not understand this and will reject any
transport messages that mention this build type.

This commit fixes this by reporting TAR instead of DOCKER when talking to older
nodes.

Relates (but does not fix) #40511
Relates #39378
DaveCTurner added a commit that referenced this issue Apr 2, 2019
@DaveCTurner
Use TAR instead of DOCKER build type before 6.7.0 (#40723)
6f00952 
In 6.7.0 (#39378) we added a build type of DOCKER for the docker images, but
unfortunately earlier versions do not understand this and will reject any
transport messages that mention this build type.

This commit fixes this by reporting TAR instead of DOCKER when talking to older
nodes.

Relates (but does not fix) #40511
Relates #39378
DaveCTurner added a commit that referenced this issue Apr 2, 2019
@DaveCTurner
Use TAR instead of DOCKER build type before 6.7.0 (#40723)
952ad85 
In 6.7.0 (#39378) we added a build type of DOCKER for the docker images, but
unfortunately earlier versions do not understand this and will reject any
transport messages that mention this build type.

This commit fixes this by reporting TAR instead of DOCKER when talking to older
nodes.

Relates (but does not fix) #40511
Relates #39378
DaveCTurner added a commit that referenced this issue Apr 2, 2019
@DaveCTurner
Use TAR instead of DOCKER build type before 6.7.0 (#40723)
abe509a 
In 6.7.0 (#39378) we added a build type of DOCKER for the docker images, but
unfortunately earlier versions do not understand this and will reject any
transport messages that mention this build type.

This commit fixes this by reporting TAR instead of DOCKER when talking to older
nodes.

Relates (but does not fix) #40511
Relates #39378
DaveCTurner added a commit that referenced this issue Apr 2, 2019
@DaveCTurner
Use TAR instead of DOCKER build type before 6.7.0 (#40723)
2f32220 
In 6.7.0 (#39378) we added a build type of DOCKER for the docker images, but
unfortunately earlier versions do not understand this and will reject any
transport messages that mention this build type.

This commit fixes this by reporting TAR instead of DOCKER when talking to older
nodes.

Relates (but does not fix) #40511
Relates #39378

This commit is the changes that were supposed to be included in the preceding
one, abe509a, but weren't because of a missing
`git add`.
@dongs0104 dongs0104 mentioned this issue Apr 3, 2019
Closed
@dongs0104
Copy link

@iremmats #40723 will return same result as you #40511 (comment)

@lcawl lcawl mentioned this issue Apr 3, 2019
Merged
@dongs0104
Copy link

  1. build new docker file from official build and change ES_DISTRIBUTION_TYPE
FROM docker.elastic.co/elasticsearch/elasticsearch:6.7.0

RUN grep ES_DISTRIBUTION_TYPE=docker /usr/share/elasticsearch/bin/elasticsearch-env \
    && sed -ie 's/ES_DISTRIBUTION_TYPE=docker/ES_DISTRIBUTION_TYPE=tar/' /usr/share/elasticsearch/bin/elasticsearch-env
  1. rolling upgrade for every nodes MUST Check Elastic cluster version > 6.7.
  • that issue is shown under 6.7.0 Cluster only. (docker type shown at 6.7.0)
  1. repeat rolling upgrade(Change Docker images) using official build from ES(docker.elastic.co/elasticsearch/elasticsearch:6.7.0)

It should be work.

@ItamarBenjamin
Copy link
Contributor

@dongs0104
if i got this right, need to roll upgrade the entire cluster using the modified build, and once the entire cluster runs on that build do another rolling upgrade to the original build?

@dongs0104
Copy link

@dongs0104
if i got this right, need to roll upgrade the entire cluster using the modified build, and once the entire cluster runs on that build do another rolling upgrade to the original build?

I think when you changed every master nodes it will be work but that is not tested

@ItamarBenjamin
Copy link
Contributor

@dongs0104
thanks, it worked! i didn't do this just for the masters though. installed the entire cluster with the fixed version, then installed the real version again.

hub-cap added a commit that referenced this issue Apr 4, 2019
@hub-cap
HLRC: Convert xpack methods to client side objects (#40705)
86e4b04 
This commit fixes a problem with BWC that was brought up in #40511. A
newer version of the code was emitting a new value for an enum to an
older version, and the older version could not handle that. It caused
the response to error. The MainResponse is now relaxed, and will accept
whatever values the server expose, and holds most of them as Strings
instead of complex objects.

Fixes #40511
hub-cap added a commit that referenced this issue Apr 4, 2019
@hub-cap
HLRC: Convert xpack methods to client side objects (#40705)
fb5a065 
This commit fixes a problem with BWC that was brought up in #40511. A
newer version of the code was emitting a new value for an enum to an
older version, and the older version could not handle that. It caused
the response to error. The MainResponse is now relaxed, and will accept
whatever values the server expose, and holds most of them as Strings
instead of complex objects.

Fixes #40511
@brackxm
Copy link
Contributor Author

brackxm commented Apr 5, 2019

will #40705 be backported to 6.6.x?

@jasontedor
Copy link
Member

@brackxm No, as we do not intend to do another 6.6.x release.

ns-codereview pushed a commit to couchbase/couchbase-elasticsearch-connector that referenced this issue Apr 29, 2019
@dnault
CBES-123 Support Elasticsearch 6.7 docker image
c353337 
Upgrade Elasticsearch High Level REST Client to version 6.7.1 to avoid
elastic/elasticsearch#40511

Change-Id: I0a4514db4a67330589798a901508ef0e90f95b7d
Reviewed-on: http://review.couchbase.org/108402
Tested-by: Build Bot <[email protected]>
Reviewed-by: David Nault <[email protected]>
gurkankaymak pushed a commit to gurkankaymak/elasticsearch that referenced this issue May 27, 2019
@DaveCTurner
Use TAR instead of DOCKER build type before 6.7.0 (elastic#40723)
8761ba6 
In 6.7.0 (elastic#39378) we added a build type of DOCKER for the docker images, but
unfortunately earlier versions do not understand this and will reject any
transport messages that mention this build type.

This commit fixes this by reporting TAR instead of DOCKER when talking to older
nodes.

Relates (but does not fix) elastic#40511
Relates elastic#39378
gurkankaymak pushed a commit to gurkankaymak/elasticsearch that referenced this issue May 27, 2019
@hub-cap
HLRC: Convert xpack methods to client side objects (elastic#40705)
06bd239 
This commit fixes a problem with BWC that was brought up in elastic#40511. A
newer version of the code was emitting a new value for an enum to an
older version, and the older version could not handle that. It caused
the response to error. The MainResponse is now relaxed, and will accept
whatever values the server expose, and holds most of them as Strings
instead of complex objects.

Fixes elastic#40511
@hashhar hashhar mentioned this issue Jun 24, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
>bug team-discuss
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

HLRC: Convert xpack methods to client side objects hub-cap/elasticsearch
8 participants
@hub-cap @javanna @brackxm @iremmats @jasontedor @ItamarBenjamin @dongs0104 @elasticmachine