Bug 1815637: bump etcd v3.3.20

[hexfusion]

This PR bumps etcd to v3.3.20 which includes a few important bugfixes:

etcd server

• Fix corruption bug in defrag. mvcc/backend: Fix corruption bug in defrag etcd-io/etcd#11613

client v3

• Fix "hasleader" metadata embedding. clientv3: fix "hasleader" metadata key, embed client version etcd-io/etcd#11687

gRPC Proxy

• Fix panic on error for metrics handler. proxy/grpcproxy: add return on error for metrics handler etcd-io/etcd#11694

3.3.19 changelog: https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.3.md#v3319-2020-03-18
3.3.20 changelog https://github.com/etcd-io/etcd/blob/master/CHANGELOG-3.3.md#v3320-2020-04-01

Carry: 151bb84

[openshift-ci-robot]

@hexfusion: This pull request references Bugzilla bug 1815637, which is invalid:

• expected the bug to target the "4.4.0" release, but it targets "4.5.0" instead
• expected Bugzilla bug 1815637 to depend on a bug targeting a release in 4.5.0, 4.5.z and in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), but no dependents were found

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 1815637: bump etcd v3.3.20

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hexfusion

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [hexfusion]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[hexfusion]

validate integrity of rebase.

git diff openshift-4.4-bump-etcd-v3.3.20..v3.3.20

diff --git a/.dockerignore b/.dockerignore
index e69de29bb..6b8710a71 100644
--- a/.dockerignore
+++ b/.dockerignore
@@ -0,0 +1 @@
+.git
diff --git a/Dockerfile.openshift b/Dockerfile.openshift
deleted file mode 100644
index 78303e525..000000000
--- a/Dockerfile.openshift
+++ /dev/null
@@ -1,20 +0,0 @@
-FROM registry.svc.ci.openshift.org/openshift/release:golang-1.12 AS builder
-
-WORKDIR /go/src/github.com/coreos/etcd
-
-COPY . .
-
-RUN ./build
-
-# stage 2
-FROM registry.svc.ci.openshift.org/openshift/origin-v4.0:base
-
-ENTRYPOINT ["/usr/bin/etcd"]
-
-COPY --from=builder /go/src/github.com/coreos/etcd/bin/etcd /usr/bin/
-COPY --from=builder /go/src/github.com/coreos/etcd/bin/etcdctl /usr/bin/
-COPY --from=builder /go/src/github.com/coreos/etcd/bin/discover-etcd-initial-cluster /usr/bin/
-
-LABEL io.k8s.display-name="etcd server" \
-      io.k8s.description="etcd is a distributed key-value store which stores the persistent master state for Kubernetes and OpenShift." \
-      maintainer="Sam Batschelet <[email protected]>"
diff --git a/Dockerfile.rhel b/Dockerfile.rhel
deleted file mode 100644
index bd353e1a3..000000000
--- a/Dockerfile.rhel
+++ /dev/null
@@ -1,20 +0,0 @@
-FROM openshift/golang-builder:1.12 AS builder
-
-WORKDIR /go/src/github.com/coreos/etcd
-
-COPY . .
-
-RUN ./build
-
-# stage 2
-FROM openshift/origin-base
-
-ENTRYPOINT ["/usr/bin/etcd"]
-
-COPY --from=builder /go/src/github.com/coreos/etcd/bin/etcd /usr/bin/
-COPY --from=builder /go/src/github.com/coreos/etcd/bin/etcdctl /usr/bin/
-COPY --from=builder /go/src/github.com/coreos/etcd/bin/discover-etcd-initial-cluster /usr/bin/
-
-LABEL io.k8s.display-name="etcd server" \
-      io.k8s.description="etcd is a distributed key-value store which stores the persistent master state for Kubernetes and OpenShift." \
-      maintainer="Sam Batschelet <[email protected]>"
diff --git a/OWNERS b/OWNERS
deleted file mode 100644
index 572c3af9d..000000000
--- a/OWNERS
+++ /dev/null
@@ -1,8 +0,0 @@
-approvers:
-- hexfusion
-reviewers:
-- deads2k
-- crawford
-- hexfusion
-- smarterclayton
-- wking
diff --git a/build b/build
index e0787aaa8..5cf4d7c64 100755
--- a/build
+++ b/build
@@ -64,20 +64,6 @@ etcd_build() {
 		-o "${out}/etcdctl" ${REPO_PATH}/etcdctl || return
 }
 
-
-openshift_tools_build() {
-	out="bin"
-	if [[ -n "${BINDIR}" ]]; then out="${BINDIR}"; fi
-	toggle_failpoints_default
-
-	# Static compilation is useful when etcd is run in a container. $GO_BUILD_FLAGS is OK
-	# shellcheck disable=SC2086
-	CGO_ENABLED=0 go build $GO_BUILD_FLAGS \
-		-installsuffix cgo \
-		-ldflags "$GO_LDFLAGS" \
-		-o "${out}/discover-etcd-initial-cluster" "github.com/coreos/etcd/openshift-tools/discover-etcd-initial-cluster" || return
-}
-
 tools_build() {
 	out="bin"
 	if [[ -n "${BINDIR}" ]]; then out="${BINDIR}"; fi
@@ -105,5 +91,4 @@ fi
 # only build when called directly, not sourced
 if echo "$0" | grep "build$" >/dev/null; then
 	etcd_build
-	openshift_tools_build
 fi
diff --git a/glide.lock b/glide.lock
index 31c787a2d..5fcf059ca 100644
--- a/glide.lock
+++ b/glide.lock
@@ -1,5 +1,5 @@
-hash: fb475a41a3ff034787f702d39abce833083d7fc3c37aa80e2bf35f82655c0a31
-updated: 2020-04-17T10:25:50.74702498-04:00
+hash: 3fece95d13153dbb91034bd2708f8f7a39510c682dfda4c3ff691f09a61a2fc1
+updated: 2019-08-19T20:45:29.93927922-07:00
 imports:
 - name: github.com/beorn7/perks
   version: 37c8de3658fcb183f997c4e13e8337516ab753e6
@@ -184,7 +184,7 @@ imports:
   - googleapis/api/annotations
   - googleapis/rpc/status
 - name: google.golang.org/grpc
-  version: 39e8a7b072a67ca2a75f57fa2e0d50995f5b22f6
+  version: 6eaf6f47437a6b4e2153a190160ef39a92c7eceb
   subpackages:
   - balancer
   - balancer/base
diff --git a/glide.yaml b/glide.yaml
index f9cbb2348..00b10851a 100644
--- a/glide.yaml
+++ b/glide.yaml
@@ -122,7 +122,7 @@ import:
   subpackages:
   - rate
 - package: google.golang.org/grpc
-  version: v1.23.1
+  version: v1.23.0
   subpackages:
   - balancer
   - codes
diff --git a/openshift-tools/discover-etcd-initial-cluster/main.go b/openshift-tools/discover-etcd-initial-cluster/main.go
deleted file mode 100644
index de62e32a9..000000000
--- a/openshift-tools/discover-etcd-initial-cluster/main.go
+++ /dev/null
@@ -1,35 +0,0 @@
-package main
-
-import (
-	goflag "flag"
-	"fmt"
-	"math/rand"
-	"os"
-	"strings"
-	"time"
-
-	discover_etcd_initial_cluster "github.com/coreos/etcd/openshift-tools/pkg/discover-etcd-initial-cluster"
-	"github.com/spf13/pflag"
-)
-
-// copy from `utilflag "k8s.io/component-base/cli/flag"`
-// WordSepNormalizeFunc changes all flags that contain "_" separators
-func WordSepNormalizeFunc(f *pflag.FlagSet, name string) pflag.NormalizedName {
-	if strings.Contains(name, "_") {
-		return pflag.NormalizedName(strings.Replace(name, "_", "-", -1))
-	}
-	return pflag.NormalizedName(name)
-}
-
-func main() {
-	rand.Seed(time.Now().UTC().UnixNano())
-
-	pflag.CommandLine.SetNormalizeFunc(WordSepNormalizeFunc)
-	pflag.CommandLine.AddGoFlagSet(goflag.CommandLine)
-
-	command := discover_etcd_initial_cluster.NewDiscoverEtcdInitialClusterCommand()
-	if err := command.Execute(); err != nil {
-		fmt.Fprintf(os.Stderr, "%v\n", err)
-		os.Exit(1)
-	}
-}
diff --git a/openshift-tools/pkg/discover-etcd-initial-cluster/initial-cluster.go b/openshift-tools/pkg/discover-etcd-initial-cluster/initial-cluster.go
deleted file mode 100644
index cbcb5de5f..000000000
--- a/openshift-tools/pkg/discover-etcd-initial-cluster/initial-cluster.go
+++ /dev/null
@@ -1,276 +0,0 @@
-package discover_etcd_initial_cluster
-
-import (
-	"context"
-	"fmt"
-	"os"
-	"path/filepath"
-	"strings"
-	"time"
-
-	"github.com/coreos/etcd/etcdserver/etcdserverpb"
-
-	"github.com/coreos/etcd/pkg/transport"
-	"google.golang.org/grpc"
-
-	"github.com/coreos/etcd/clientv3"
-
-	"github.com/spf13/pflag"
-
-	"github.com/spf13/cobra"
-)
-
-type DiscoverEtcdInitialClusterOptions struct {
-	// TargetPeerURLHost is the host portion of the peer URL.  It is used to match on. (either IP or hostname)
-	TargetPeerURLHost string
-	// TargetName is the name to assign to this peer if we create it
-	TargetName string
-
-	// CABundleFile is the file to use to trust the etcd server
-	CABundleFile string
-	// ClientCertFile is the client cert to use to authenticate this binary to etcd
-	ClientCertFile string
-	// ClientKeyFile is the client key to use to authenticate this binary to etcd
-	ClientKeyFile string
-	// Endpoints is a list of all the endpoints to use to try to contact etcd
-	Endpoints []string
-
-	// DataDir is the directory created when etcd starts the first time
-	DataDir string
-}
-
-func NewDiscoverEtcdInitialCluster() *DiscoverEtcdInitialClusterOptions {
-	return &DiscoverEtcdInitialClusterOptions{}
-}
-
-func NewDiscoverEtcdInitialClusterCommand() *cobra.Command {
-	o := NewDiscoverEtcdInitialCluster()
-
-	cmd := &cobra.Command{
-		Use:   "discover-etcd-initial-cluster",
-		Short: "output the value for ETCD_INITIAL_CLUSTER in openshift etcd static pod",
-		Long: `output the value for ETCD_INITIAL_CLUSTER in openshift etcd static pod
-
-1. If --data-dir exists, output a marker value and exit.
-2. It tries to contact every available etcd to get a list of member.
-3. Check each member to see if any one of them is the target.
-4. If so, and if it is started, use the member list to create the ETCD_INITIAL_CLUSTER value and print it out.
-5. If so, and if it it not started, use the existing member list and append the target value to create the ETCD_INITIAL_CLUSTER value and print it out.
-6. If not, try again until either you have it or you time out.
-`,
-		Run: func(cmd *cobra.Command, args []string) {
-			if err := o.Validate(); err != nil {
-				fmt.Fprint(os.Stderr, err)
-				os.Exit(1)
-			}
-
-			if err := o.Run(); err != nil {
-				fmt.Fprint(os.Stderr, err)
-				os.Exit(1)
-			}
-		},
-	}
-	o.BindFlags(cmd.Flags())
-
-	return cmd
-}
-
-func (o *DiscoverEtcdInitialClusterOptions) BindFlags(flags *pflag.FlagSet) {
-	flags.StringVar(&o.CABundleFile, "cacert", o.CABundleFile, "file to use to verify the identity of the etcd server")
-	flags.StringVar(&o.ClientCertFile, "cert", o.ClientCertFile, "client cert to use to authenticate this binary to etcd")
-	flags.StringVar(&o.ClientKeyFile, "key", o.ClientKeyFile, "client key to use to authenticate this binary to etcd")
-	flags.StringSliceVar(&o.Endpoints, "endpoints", o.Endpoints, "list of all the endpoints to use to try to contact etcd")
-	flags.StringVar(&o.DataDir, "data-dir", o.DataDir, "dir to stat for existence of the member directory")
-	flags.StringVar(&o.TargetPeerURLHost, "target-peer-url-host", o.TargetPeerURLHost, "host portion of the peer URL.  It is used to match on. (either IP or hostname)")
-	flags.StringVar(&o.TargetName, "target-name", o.TargetName, "name to assign to this peer if we create it")
-}
-
-func (o *DiscoverEtcdInitialClusterOptions) Validate() error {
-	if len(o.CABundleFile) == 0 {
-		return fmt.Errorf("missing --cacert")
-	}
-	if len(o.ClientCertFile) == 0 {
-		return fmt.Errorf("missing --cert")
-	}
-	if len(o.ClientKeyFile) == 0 {
-		return fmt.Errorf("missing --key")
-	}
-	if len(o.Endpoints) == 0 {
-		return fmt.Errorf("missing --endpoints")
-	}
-	if len(o.DataDir) == 0 {
-		return fmt.Errorf("missing --data-dir")
-	}
-	if len(o.TargetPeerURLHost) == 0 {
-		return fmt.Errorf("missing --target-peer-url-host")
-	}
-	if len(o.TargetName) == 0 {
-		return fmt.Errorf("missing --target-name")
-	}
-	return nil
-}
-
-func (o *DiscoverEtcdInitialClusterOptions) Run() error {
-
-	//Temporary hack to work with the current pod.yaml
-	var memberDir string
-	if strings.HasSuffix(o.DataDir, "member") {
-		memberDir = o.DataDir
-		o.DataDir = filepath.Dir(o.DataDir)
-	} else {
-		memberDir = filepath.Join(o.DataDir, "member")
-	}
-
-	memberDirExists := false
-	_, err := os.Stat(memberDir)
-	switch {
-	case os.IsNotExist(err):
-		// do nothing. This just means we fall through to the polling logic
-
-	case err == nil:
-		fmt.Fprintf(os.Stderr, "memberDir %s is present on %s\n", memberDir, o.TargetName)
-		memberDirExists = true
-
-	case err != nil:
-		return err
-	}
-
-	client, err := o.getClient()
-	if err != nil && memberDirExists {
-		// we weren't able to get client and need to return based previous memberDir so we can restart.  This is the off and on again flow.
-		fmt.Fprintf(os.Stderr, "Couldn't get client, but memberDir %s is present on %s, err=%s. Returning with no error.\n", memberDir, o.TargetName, err)
-		fmt.Printf(o.TargetName)
-		return nil
-	} else if err != nil {
-		return err
-	}
-	defer client.Close()
-
-	var targetMember *etcdserverpb.Member
-	var allMembers []*etcdserverpb.Member
-	for i := 0; i < 10; i++ {
-		fmt.Fprintf(os.Stderr, "#### attempt %d\n", i)
-		targetMember, allMembers, err = o.checkForTarget(client)
-
-		for _, member := range allMembers {
-			fmt.Fprintf(os.Stderr, "      member=%v\n", stringifyMember(member))
-		}
-		fmt.Fprintf(os.Stderr, "      target=%v, err=%v\n", stringifyMember(targetMember), err)
-
-		// we're done because we found what we want.
-		if targetMember != nil && err == nil {
-			break
-		}
-
-		fmt.Fprintf(os.Stderr, "#### sleeping...\n")
-		time.Sleep(1 * time.Second)
-	}
-
-	switch {
-	case targetMember == nil && memberDirExists:
-		// we weren't able to locate other members and need to return based previous memberDir so we can restart.  This is again the off and on flow.
-		fmt.Fprintf(os.Stderr, "Couldn't get targetMember, but memberDir %s is present on %s. Returning with no error.\n", memberDir, o.TargetName)
-		fmt.Printf(o.TargetName)
-		return nil
-
-	case err != nil:
-		fmt.Fprintf(os.Stderr, "Couldn't get targetMember. Returning error.\n")
-		return err
-
-	case targetMember == nil && !memberDirExists:
-		// our member has not been added to the cluster and we have no previous data to start based on.
-		return fmt.Errorf("timed out")
-
-	case targetMember != nil && len(targetMember.Name) == 0 && memberDirExists:
-		// our member has been added to the cluster and has never been started before, but a data directory exists. This means that we have dirty data we must remove
-		fmt.Fprintf(os.Stderr, "Found targetMember but is unstarted and memberDir exists. Archiving memberrDir\n")
-		archiveDataDir(memberDir)
-
-	default:
-		// a target member was found, but no exception circumstances.
-	}
-
-	etcdInitialClusterEntries := []string{}
-	for _, member := range allMembers {
-		if len(member.Name) == 0 { // this is the signal for whether or not a given peer is started
-			continue
-		}
-		for _, peerURL := range member.PeerURLs{
-			etcdInitialClusterEntries = append(etcdInitialClusterEntries, fmt.Sprintf("%s=%s", member.Name, peerURL))
-		}
-	}
-	if len(targetMember.Name) == 0 {
-		fmt.Fprintf(os.Stderr, "Adding the unstarted member to the end %s\n", o.TargetName)
-		etcdInitialClusterEntries = append(etcdInitialClusterEntries, fmt.Sprintf("%s=%s", o.TargetName, targetMember.PeerURLs[0]))
-	}
-
-	fmt.Printf(strings.Join(etcdInitialClusterEntries, ","))
-
-	return nil
-}
-
-// TO DO: instead of archiving, we should remove the directory to avoid any confusion with the backups.
-func archiveDataDir(sourceDir string) error {
-	targetDir := filepath.Join(sourceDir + "-removed-archive-" + time.Now().Format("2006-01-02-030405"))
-
-	fmt.Fprintf(os.Stderr, "attempting to archive %s to %s", sourceDir, targetDir)
-	if err := os.Rename(sourceDir, targetDir); err != nil {
-		return err
-	}
-	return nil
-}
-
-func stringifyMember(member *etcdserverpb.Member) string {
-	if member == nil {
-		return "nil"
-	}
-
-	return fmt.Sprintf("{name=%q, peerURLs=[%s}, clientURLs=[%s]", member.Name, strings.Join(member.PeerURLs, ","), strings.Join(member.ClientURLs, ","))
-}
-
-func (o *DiscoverEtcdInitialClusterOptions) checkForTarget(client *clientv3.Client) (*etcdserverpb.Member, []*etcdserverpb.Member, error) {
-	ctx, cancel := context.WithTimeout(context.Background(), 1*time.Second)
-	defer cancel()
-
-	memberResponse, err := client.MemberList(ctx)
-	if err != nil {
-		return nil, nil, err
-	}
-
-	var targetMember *etcdserverpb.Member
-	for i := range memberResponse.Members {
-		member := memberResponse.Members[i]
-		for _, peerURL := range member.PeerURLs {
-			if strings.Contains(peerURL, o.TargetPeerURLHost) {
-				targetMember = member
-			}
-		}
-	}
-
-	return targetMember, memberResponse.Members, err
-}
-
-func (o *DiscoverEtcdInitialClusterOptions) getClient() (*clientv3.Client, error) {
-	dialOptions := []grpc.DialOption{
-		grpc.WithBlock(), // block until the underlying connection is up
-	}
-
-	tlsInfo := transport.TLSInfo{
-		CertFile:      o.ClientCertFile,
-		KeyFile:       o.ClientKeyFile,
-		TrustedCAFile: o.CABundleFile,
-	}
-	tlsConfig, err := tlsInfo.ClientConfig()
-	if err != nil {
-		return nil, err
-	}
-
-	cfg := &clientv3.Config{
-		DialOptions: dialOptions,
-		Endpoints:   o.Endpoints,
-		DialTimeout: 15 * time.Second,
-		TLS:         tlsConfig,
-	}
-
-	return clientv3.New(*cfg)
-}
diff --git a/vendor/google.golang.org/grpc/internal/transport/http2_server.go b/vendor/google.golang.org/grpc/internal/transport/http2_server.go
index 4e26f6a1d..83439b562 100644
--- a/vendor/google.golang.org/grpc/internal/transport/http2_server.go
+++ b/vendor/google.golang.org/grpc/internal/transport/http2_server.go
@@ -138,10 +138,7 @@ func newHTTP2Server(conn net.Conn, config *ServerConfig) (_ ServerTransport, err
 	}
 	framer := newFramer(conn, writeBufSize, readBufSize, maxHeaderListSize)
 	// Send initial settings as connection preface to client.
-	isettings := []http2.Setting{{
-		ID:  http2.SettingMaxFrameSize,
-		Val: http2MaxFrameLen,
-	}}
+	var isettings []http2.Setting
 	// TODO(zhaoq): Have a better way to signal "no limit" because 0 is
 	// permitted in the HTTP2 spec.
 	maxStreams := config.MaxStreams
diff --git a/vendor/google.golang.org/grpc/internal/transport/http_util.go b/vendor/google.golang.org/grpc/internal/transport/http_util.go
index 8f5f3349d..9d212867c 100644
--- a/vendor/google.golang.org/grpc/internal/transport/http_util.go
+++ b/vendor/google.golang.org/grpc/internal/transport/http_util.go
@@ -667,7 +667,6 @@ func newFramer(conn net.Conn, writeBufferSize, readBufferSize int, maxHeaderList
 		writer: w,
 		fr:     http2.NewFramer(w, r),
 	}
-	f.fr.SetMaxReadFrameSize(http2MaxFrameLen)
 	// Opt-in to Frame reuse API on framer to reduce garbage.
 	// Frames aren't safe to read from after a subsequent call to ReadFrame.
 	f.fr.SetReuseFrames()
diff --git a/vendor/google.golang.org/grpc/version.go b/vendor/google.golang.org/grpc/version.go
index 588850563..5411a73a2 100644
--- a/vendor/google.golang.org/grpc/version.go
+++ b/vendor/google.golang.org/grpc/version.go
@@ -19,4 +19,4 @@
 package grpc
 
 // Version is the current grpc version.
-const Version = "1.23.1"
+const Version = "1.23.0"

[hexfusion]

/retest

[hexfusion]

/retest

[hexfusion]

/test e2e-gcp-upgrade

[hexfusion]

/test e2e-gcp-upgrade

[openshift-ci-robot]

@hexfusion: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/e2e-gcp-upgrade	151bb84	link	/test e2e-gcp-upgrade

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[hexfusion]

/test all