sync-tags: add support for v0.x.y tags #210

nikhita · 2019-12-04T17:45:21Z

Implements changes to the go modules KEP made in
kubernetes/enhancements#1350.

This commit updates sync-tags to publish v0.x.y tags for corresponding
v1.x.y Kubernetes tag. The old kubernetes-1.x.y tags are still also
published. Both kubernetes-1.x.y and v0.x.y point to the same sha.

For v0.x.y:

At this tag, go.mod uses v0.x.y tags for dependencies.
When this tag is manually specified in a go.mod file, it will be
retained as v0.x.y.

For kubernetes-1.x.y:

At this tag, go.mod uses v0.x.y tags for dependencies. Note: this
differs from the previous way, where pseudoversions were used.
When this tag is manually specified in a go.mod file, go will resolve
it to the v0.x.y tag, so pseudoversions would never be needed.

Verified locally. A sample run for client-go (and api + apimachinery) is shown below:

/cc @sttts @liggitt @dims
/assign @sttts

cmd/sync-tags/gomod.go

cmd/sync-tags/main.go

nikhita · 2019-12-05T06:40:33Z

Addressed all comments except #210 (comment). Also, updated the main PR body and commit messages to reflect the new behaviour and added links to a new test run in the PR body.

If --publish-semver-tags is true, we won't use pseudoversions, so the problem described in #210 (comment) is not relevant anymore.

cmd/sync-tags/main.go

cmd/sync-tags/gomod.go

sttts · 2019-12-05T07:57:46Z

cmd/sync-tags/main.go

-			var changed, goModChanged bool
-			_, err = os.Stat("go.mod")
-			if os.IsNotExist(err) {
+			if !goModExists {
 				if _, err2 := os.Stat("Godeps/Godeps.json"); err2 == nil {
 					fmt.Printf("Updating Godeps.json to point to %s tag.\n", bName)


should we replace Godeps.json with go.mod everywhere?

or is this really still about godeps?

This is for the case where there is no go.mod but only Godeps.json file in the repo. We don't need it for k8s repos but might be useful for non-k8s ones.

Or we could remove it here and force them to use the godeps branch in this repo, but that'd break compatibility.

does this script still run on 1.14.x, or did we split the job to use the godeps branch for publishing that branch? dropping godeps manipulation seems ok, but I'd do it in a separate PR

does this script still run on 1.14.x, or did we split the job to use the godeps branch for publishing that branch?

The latter. Will remove this in another PR. 👍

cmd/sync-tags/main.go

liggitt · 2019-12-05T13:44:52Z

The tags published to https://github.com/prowtest look good

cmd/sync-tags/main.go

nikhita · 2019-12-09T13:00:45Z

Just adding an explicit hold until we've tested this thoroughly
/hold

nikhita · 2019-12-09T20:27:11Z

Updated with the latest changes, PTAL. The current workflow (as discussed on slack) is:

figure out all the tags we want to create
if any already exist remotely, we're done
if any already exist locally, delete them (and also delete the corresponding go mod cache .mod/.zip files)
update+tidy+commit the go.mod (with the semver tag if we're semver-tagging, with the pseudoversion otherwise), create all tags, then push all tags with --atomic

Note: for 3) we don't overwrite the existing .mod/.info/.zip files in the go mod cache dir while building tags instead because with the current logic, we avoid creating the .mod, etc files if they already exist.

publishing-bot/cmd/sync-tags/gomod.go

Lines 157 to 159 in dde80ae

    
           if _, err := os.Stat(goModFile); err == nil { 
        
           	fmt.Printf("%s for %s is already packaged up.\n", pseudoVersionOrTag, depPkg) 
        
           	return nil

Even if we tried to keep track of overwriting it once, it isn't possible to do it across multiple runs (for multiple repos) of sync-tags.

cmd/sync-tags/main.go

Implements changes to the go modules KEP made in kubernetes/enhancements#1350. This commit updates sync-tags to publish `v0.x.y` tags for corresponding `v1.x.y` Kubernetes tag. The old `kubernetes-1.x.y` tags are still also published. Both `kubernetes-1.x.y` and `v0.x.y` point to the same sha. For `v0.x.y`: - At this tag, `go.mod` uses `v0.x.y` tags for dependencies. - When this tag is manually specified in a `go.mod` file, it will be retained as `v0.x.y`. For `kubernetes-1.x.y`: - At this tag, `go.mod` uses `v0.x.y` tags for dependencies. Note: this differs from the previous way, where pseudoversions were used. - When this tag is manually specified in a `go.mod` file, go will resolve it to the appropriate pseudoversion.

nikhita · 2019-12-10T14:18:34Z

Rebased on top of master (which brought in #211)

liggitt · 2019-12-10T15:38:29Z

/lgtm
/hold pending run against kubernetes-nightly org

liggitt · 2019-12-10T17:34:50Z

test run to kubernetes-nightly org looks good

all repos got v0.17.0 and kubernetes-1.17.0 tags on the same sha. examples:
all go.mod files referenced v0.17.0 tags for peer k8s.io dependencies

liggitt · 2019-12-10T17:34:59Z

/hold cancel

sttts · 2019-12-10T19:56:23Z

/approve

k8s-ci-robot · 2019-12-10T19:56:40Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: liggitt, nikhita, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [sttts]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot assigned sttts Dec 4, 2019

k8s-ci-robot requested review from dims, liggitt and sttts December 4, 2019 17:45

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Dec 4, 2019

liggitt reviewed Dec 4, 2019

View reviewed changes

cmd/sync-tags/gomod.go Show resolved Hide resolved

liggitt reviewed Dec 4, 2019

View reviewed changes