Bug 1815634: bump etcd v3.3.22 #53

hexfusion · 2020-07-15T12:21:20Z

This PR bumps etcd to v3.3.22 which includes a few important bugfixes:

etcd server

Fix corruption bug in defrag. mvcc/backend: Fix corruption bug in defrag etcd-io/etcd#11613
Fix wal not found. etcdserver: wait purge file loop to exit before stopping raft etcd-io/etcd#11308

client v3

Fix "hasleader" metadata embedding. clientv3: fix "hasleader" metadata key, embed client version etcd-io/etcd#11687

gRPC Proxy

Fix panic on error for metrics handler. proxy/grpcproxy: add return on error for metrics handler etcd-io/etcd#11694

3.3.18 changelog: https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.3.md#v3318-2019-11-26
3.3.19 changelog: https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.3.md#v3319-2020-03-18
3.3.20 changelog https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.3.md#v3320-2020-04-01
3.3.21 changelog https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.3.md#v3321-2020-05-18
3.3.22 changelog https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.3.md#v3321-2020-05-18

CARRY: 7e95ccf

scripts: avoid release builds on darwin machine.

Signed-off-by: Gyuho Lee <[email protected]>

…-of-#10257-upstream-release-3.3 cherry pick "etcd_cluster_version" metric" (etcd-io#10257, etcd-io#11233, etcd-io#11254, etcd-io#11265) to release-3.3

…of-#11247-origin-release-3.3 Automated cherry pick of etcd-io#11247

Signed-off-by: Gyuho Lee <[email protected]>

To prevent the purge file loop from accidentally acquiring the file lock and remove the files during server shutdowm.

…f-#11308-upstream-release-3.3 Automated cherry pick of etcd-io#11308 on release-3.3

Signed-off-by: Gyuho Lee <[email protected]>

Once chk(ai) fails with auth.ErrAuthOldRevision it will always do, regardless how many times you retry. So the error is better be returned to fail the pending request and make the client re-authenticate.

Remove auth validation loop in v3_server.raftRequest(). Re-validation when error ErrAuthOldRevision occurs should be handled on client side.

Disable TestV3AuthOldRevConcurrent for now. See etcd-io#10468 (comment)

…put request.

…f-#11374-upstream-release-3.3 Automated cherry pick of etcd-io#11374 on release 3.3

Signed-off-by: Gyuho Lee <[email protected]>

…f-#10218-etcd-io#10468-upstream-release-3.3 Automated cherry pick of etcd-io#10218 etcd-io#10468 on release 3.3

…-#11613-origin-release-3.3 Automated cherry pick of etcd-io#11613 to release-3.3

Check if boltOpenOptions is nil before use it.

…f-#11630-upstream-release-3.3 Automated cherry pick of etcd-io#11630 to release-3.3

Use members information from member add response, which is guaranteed to be up to date.

…f-#11638-upstream-release-3.3 Automated cherry pick of etcd-io#11638 on release-3.3

Signed-off-by: Sam Batschelet <[email protected]>

ref. etcd-io#11704 Signed-off-by: Gyuho Lee <[email protected]>

ref. etcd-io#11687 Signed-off-by: Gyuho Lee <[email protected]> clientv3: fix racy writes to context key === RUN TestWatchOverlapContextCancel ================== WARNING: DATA RACE Write at 0x00c42110dd40 by goroutine 99: runtime.mapassign() /usr/local/go/src/runtime/hashmap.go:485 +0x0 github.com/coreos/etcd/clientv3.metadataSet() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:61 +0x8c github.com/coreos/etcd/clientv3.withVersion() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/ctx.go:47 +0x137 github.com/coreos/etcd/clientv3.newStreamClientInterceptor.func1() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/client.go:309 +0x81 google.golang.org/grpc.NewClientStream() /go/src/github.com/coreos/etcd/gopath/src/google.golang.org/grpc/stream.go:101 +0x10e github.com/coreos/etcd/etcdserver/etcdserverpb.(*watchClient).Watch() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/etcdserver/etcdserverpb/rpc.pb.go:3193 +0xe9 github.com/coreos/etcd/clientv3.(*watchGrpcStream).openWatchClient() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:788 +0x143 github.com/coreos/etcd/clientv3.(*watchGrpcStream).newWatchClient() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:700 +0x5c3 github.com/coreos/etcd/clientv3.(*watchGrpcStream).run() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:431 +0x12b Previous read at 0x00c42110dd40 by goroutine 130: reflect.maplen() /usr/local/go/src/runtime/hashmap.go:1165 +0x0 reflect.Value.MapKeys() /usr/local/go/src/reflect/value.go:1090 +0x43b fmt.(*pp).printValue() /usr/local/go/src/fmt/print.go:741 +0x1885 fmt.(*pp).printArg() /usr/local/go/src/fmt/print.go:682 +0x1b1 fmt.(*pp).doPrintf() /usr/local/go/src/fmt/print.go:998 +0x1cad fmt.Sprintf() /usr/local/go/src/fmt/print.go:196 +0x77 github.com/coreos/etcd/clientv3.streamKeyFromCtx() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:825 +0xc8 github.com/coreos/etcd/clientv3.(*watcher).Watch() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:265 +0x426 github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e Goroutine 99 (running) created at: github.com/coreos/etcd/clientv3.(*watcher).newWatcherGrpcStream() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:236 +0x59d github.com/coreos/etcd/clientv3.(*watcher).Watch() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/watch.go:278 +0xbb6 github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel.func1() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:959 +0x23e Goroutine 130 (running) created at: github.com/coreos/etcd/clientv3/integration.testWatchOverlapContextCancel() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:979 +0x76d github.com/coreos/etcd/clientv3/integration.TestWatchOverlapContextCancel() /go/src/github.com/coreos/etcd/gopath/src/github.com/coreos/etcd/clientv3/integration/watch_test.go:922 +0x44 testing.tRunner() /usr/local/go/src/testing/testing.go:657 +0x107 ================== Signed-off-by: Gyuho Lee <[email protected]>

ref. etcd-io#11687 Signed-off-by: Gyuho Lee <[email protected]>

Signed-off-by: Gyuho Lee <[email protected]>

CARRY: This bump resolves and issue where max frame size (16kb) was not defined which has security implications. Signed-off-by: Sam Batschelet <[email protected]>

openshift-ci-robot · 2020-07-15T12:21:21Z

@hexfusion: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

In response to this:

bump etcd v3.3.22

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-07-15T12:30:38Z

@hexfusion: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

In response to this:

bump etcd v3.3.22

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-07-15T12:32:16Z

@hexfusion: No Bugzilla bug is referenced in the title of this pull request.
To reference a bug, add 'Bug XXX:' to the title of this pull request and request another bug refresh with /bugzilla refresh.

In response to this:

bump etcd v3.3.22

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-07-15T12:33:21Z

@hexfusion: This pull request references Bugzilla bug 1815634, which is valid. The bug has been updated to refer to the pull request using the external bug tracker.

8 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.3.z) matches configured target release for branch (4.3.z)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
dependent bug Bugzilla bug 1801196 is in the state CLOSED (ERRATA), which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA))
dependent bug Bugzilla bug 1801237 is in the state CLOSED (ERRATA), which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA))
dependent Bugzilla bug 1801196 targets the "4.4.0" release, which is one of the valid target releases: 4.4.0, 4.4.z
dependent Bugzilla bug 1801237 targets the "4.4.0" release, which is one of the valid target releases: 4.4.0, 4.4.z
bug has dependents

In response to this:

Bug 1815634: bump etcd v3.3.22

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

hexfusion · 2020-07-15T15:52:47Z

failed unit

TestLessorRenewExtendPileup
TestServer_TCP_Secure_DelayTx

/retest

hexfusion · 2020-07-15T16:33:17Z

soak
/test all

hexfusion · 2020-07-15T20:58:03Z

soak
/test all

hexfusion · 2020-07-16T00:08:02Z

soak ( I think this could be last this is very stable)
/test all

hexfusion · 2020-07-16T11:39:40Z

/retest

retroflexer · 2020-07-16T15:25:47Z

/lgtm

hexfusion · 2020-07-16T15:25:54Z

cc @sdodson

openshift-ci-robot · 2020-07-16T15:26:11Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hexfusion, retroflexer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [hexfusion]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-bot · 2020-07-17T14:06:56Z

/retest

Please review the full test history for this PR and help us cut down flakes.

hexfusion · 2020-07-17T14:13:12Z

unit flake TestServer_Unix_Secure_DelayAccept
we dont use unix sockets with etcd.

/test unit

https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift_etcd/53/pull-ci-openshift-etcd-openshift-4.3-unit/1284121149805957120

openshift-ci-robot · 2020-07-17T15:47:01Z

@hexfusion: All pull requests linked via external trackers have merged: openshift/etcd#53. Bugzilla bug 1815634 has been moved to the MODIFIED state.

In response to this:

Bug 1815634: bump etcd v3.3.22

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

YoyinZyc and others added 30 commits October 11, 2019 11:27

scripts: avoid release builds on darwin machine.

ffcddac

Merge pull request etcd-io#11245 from YoyinZyc/prevent-darwin-build-3.3

660dc83

scripts: avoid release builds on darwin machine.

Add version, tag and branch checks to release script

683a643

etcdserver/*: add "etcd_cluster_version" metric

5c19bd2

Signed-off-by: Gyuho Lee <[email protected]>

tests/e2e: test cluster version

7fbfdc2

Signed-off-by: Gyuho Lee <[email protected]>

Add cluster version fix etcd-io#11233, etcd-io#11254, etcd-io#11265

e788880

Merge pull request etcd-io#11261 from wenjiaswe/automated-cherry-pick…

aedfe54

…-of-#10257-upstream-release-3.3 cherry pick "etcd_cluster_version" metric" (etcd-io#10257, etcd-io#11233, etcd-io#11254, etcd-io#11265) to release-3.3

Merge pull request etcd-io#11253 from YoyinZyc/automated-cherry-pick-…

a977795

…of-#11247-origin-release-3.3 Automated cherry pick of etcd-io#11247

scripts/release: list GPG key only when tagging is needed

ff5fb05

Signed-off-by: Gyuho Lee <[email protected]>

etcdserver: wait purge file loop during shutdown

7c164a8

To prevent the purge file loop from accidentally acquiring the file lock and remove the files during server shutdowm.

Merge pull request etcd-io#11314 from jingyih/automated-cherry-pick-o…

069bce1

…f-#11308-upstream-release-3.3 Automated cherry pick of etcd-io#11308 on release-3.3

clientv3: fix retry/streamer error message

5cf80a6

Signed-off-by: Gyuho Lee <[email protected]>

etcdserver: Remove infinite loop in doSerialize

95095f8

Once chk(ai) fails with auth.ErrAuthOldRevision it will always do, regardless how many times you retry. So the error is better be returned to fail the pending request and make the client re-authenticate.

etcdserver: remove auth validation loop

5a48217

Remove auth validation loop in v3_server.raftRequest(). Re-validation when error ErrAuthOldRevision occurs should be handled on client side.

integration: disable TestV3AuthOldRevConcurrent

e1508f9

Disable TestV3AuthOldRevConcurrent for now. See etcd-io#10468 (comment)

mvcc: add "etcd_mvcc_put_size_in_bytes" to monitor the throughput of …

aaa8571

…put request.

mvcc: update to "etcd_debugging_mvcc_total_put_size_in_bytes"

47d3dea

Merge pull request etcd-io#11394 from jingyih/automated-cherry-pick-o…

9cd3eef

…f-#11374-upstream-release-3.3 Automated cherry pick of etcd-io#11374 on release 3.3

version: 3.3.18

3c8740a

Signed-off-by: Gyuho Lee <[email protected]>

Merge pull request etcd-io#11378 from jingyih/automated-cherry-pick-o…

70853d6

…f-#10218-etcd-io#10468-upstream-release-3.3 Automated cherry pick of etcd-io#10218 etcd-io#10468 on release 3.3

mvcc/backend: Delete orphaned db.tmp files before defrag

b3d9e29

Merge pull request etcd-io#11623 from jpbetz/automated-cherry-pick-of…

b0a4038

…-#11613-origin-release-3.3 Automated cherry pick of etcd-io#11613 to release-3.3

mvcc/backend: check for nil boltOpenOptions

7b1a92c

Check if boltOpenOptions is nil before use it.

Merge pull request etcd-io#11632 from jingyih/automated-cherry-pick-o…

e21e355

…f-#11630-upstream-release-3.3 Automated cherry pick of etcd-io#11630 to release-3.3

etcdctl: fix member add command

c58133b

Use members information from member add response, which is guaranteed to be up to date.

Merge pull request etcd-io#11665 from jingyih/automated-cherry-pick-o…

eb1df6d

…f-#11638-upstream-release-3.3 Automated cherry pick of etcd-io#11638 on release-3.3

proxy/grpcproxy: add return on error for metrics handler

1228d6c

Signed-off-by: Sam Batschelet <[email protected]>

etcdserver/api/etcdhttp: log server-side /health checks

30aaceb

ref. etcd-io#11704 Signed-off-by: Gyuho Lee <[email protected]>

etcdserver/api/v3rpc: handle api version metadata, add metrics

d9027ce

ref. etcd-io#11687 Signed-off-by: Gyuho Lee <[email protected]>

tangcong and others added 4 commits May 20, 2020 11:39

wal: fix crc mismatch crash bug

8ce10ea

wal: add TestValidSnapshotEntriesAfterPurgeWal testcase

a9d14cb

Signed-off-by: Gyuho Lee <[email protected]>

version: 3.3.22

282cce7

Signed-off-by: Gyuho Lee <[email protected]>

vendor: bump gRPC-go to v1.23.1

7e95ccf

CARRY: This bump resolves and issue where max frame size (16kb) was not defined which has security implications. Signed-off-by: Sam Batschelet <[email protected]>

openshift-ci-robot requested review from smarterclayton and wking July 15, 2020 12:21

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 15, 2020

hexfusion mentioned this pull request Jul 15, 2020

[testing] bump etcd v3.3.19 #41

Closed

hexfusion changed the title ~~bump etcd v3.3.22~~ Bug 1815634: bump etcd v3.3.22 Jul 15, 2020

openshift-ci-robot added bugzilla/severity-high Referenced Bugzilla bug's severity is high for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. labels Jul 15, 2020

openshift-ci-robot assigned retroflexer Jul 16, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 16, 2020

sdodson added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Jul 17, 2020

openshift-merge-robot merged commit 87f03ed into openshift:openshift-4.3 Jul 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug 1815634: bump etcd v3.3.22 #53

Bug 1815634: bump etcd v3.3.22 #53

hexfusion commented Jul 15, 2020 •

edited

Loading

openshift-ci-robot commented Jul 15, 2020

openshift-ci-robot commented Jul 15, 2020

openshift-ci-robot commented Jul 15, 2020

openshift-ci-robot commented Jul 15, 2020

hexfusion commented Jul 15, 2020

hexfusion commented Jul 15, 2020

hexfusion commented Jul 15, 2020

hexfusion commented Jul 16, 2020

hexfusion commented Jul 16, 2020

retroflexer commented Jul 16, 2020

hexfusion commented Jul 16, 2020

openshift-ci-robot commented Jul 16, 2020

openshift-bot commented Jul 17, 2020

hexfusion commented Jul 17, 2020 •

edited

Loading

openshift-ci-robot commented Jul 17, 2020

Bug 1815634: bump etcd v3.3.22 #53

Bug 1815634: bump etcd v3.3.22 #53

Conversation

hexfusion commented Jul 15, 2020 • edited Loading

openshift-ci-robot commented Jul 15, 2020

openshift-ci-robot commented Jul 15, 2020

openshift-ci-robot commented Jul 15, 2020

openshift-ci-robot commented Jul 15, 2020

hexfusion commented Jul 15, 2020

hexfusion commented Jul 15, 2020

hexfusion commented Jul 15, 2020

hexfusion commented Jul 16, 2020

hexfusion commented Jul 16, 2020

retroflexer commented Jul 16, 2020

hexfusion commented Jul 16, 2020

openshift-ci-robot commented Jul 16, 2020

openshift-bot commented Jul 17, 2020

hexfusion commented Jul 17, 2020 • edited Loading

openshift-ci-robot commented Jul 17, 2020

hexfusion commented Jul 15, 2020 •

edited

Loading

hexfusion commented Jul 17, 2020 •

edited

Loading