Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to grpc-gateway v2 #6567

Merged
merged 11 commits into from
Sep 15, 2021

Conversation

briandealwis
Copy link
Member

@briandealwis briandealwis commented Sep 3, 2021

Fixes #5986
Related: #6556

Description
This PR separates out an upgrade of grpc-gateway that @baleshk incorporated into his CloudEvents PR in #6556.

Protobufs must be passed by-reference and not by-value

A side-effect is that I had to change our code pass protobuf objects by reference instead of by value (more below). The newer proto generation now marks protobuf objects with a DoNotCopy marker to generate warnings when they are copied. Protobuf objects are not meant to be copied by value: they include an internal field that is meant to be accessed atomic access, and this field has been there for a while.

Swagger/OpenAPI output has changed

@baleshk noted that error handling is removed in the newer gateway. As a result the swagger output is changed.
The gRPC Gateway migration guide notes that they now use the RPC status return.

This change is for the better:

OLD:
swagger-old

NEW:
skaffold-new


User facing changes (remove if N/A)

  • Swagger API docs have changed

Follow-up Work (remove if N/A)
Filed #6600 to track follow-on work: the Swagger editor complains about our Swagger output as we're missing a version field, and we have a lot of badly named types (protoState) that would be nice to clear up.

@@ -16,16 +16,18 @@

set -e

docker build -t gen-proto -f hack/proto/Dockerfile --target generate-files proto
docker build --platform linux/amd64 -t gen-proto -f hack/proto/Dockerfile --target generate-files proto
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need to mention the platform switch?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops. I was trying to build from an M1 and it was failing. I'll back this out.

@codecov
Copy link

codecov bot commented Sep 10, 2021

Codecov Report

Merging #6567 (78bc627) into main (290280e) will decrease coverage by 0.10%.
The diff coverage is 66.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #6567      +/-   ##
==========================================
- Coverage   70.48%   70.38%   -0.11%     
==========================================
  Files         515      517       +2     
  Lines       23150    23365     +215     
==========================================
+ Hits        16317    16445     +128     
- Misses       5776     5857      +81     
- Partials     1057     1063       +6     
Impacted Files Coverage Δ
cmd/skaffold/app/cmd/flags.go 89.00% <0.00%> (-1.82%) ⬇️
cmd/skaffold/skaffold.go 0.00% <ø> (ø)
pkg/diag/recommender/container_errors.go 0.00% <0.00%> (ø)
pkg/diag/validator/pod.go 1.32% <0.00%> (ø)
pkg/skaffold/build/buildpacks/logger.go 0.00% <ø> (ø)
pkg/skaffold/build/cluster/logs.go 0.00% <ø> (-16.67%) ⬇️
pkg/skaffold/config/options.go 100.00% <ø> (ø)
pkg/skaffold/debug/apply_transforms.go 22.64% <0.00%> (-0.44%) ⬇️
pkg/skaffold/debug/transform.go 95.91% <ø> (ø)
pkg/skaffold/docker/logger/log.go 22.64% <0.00%> (-0.44%) ⬇️
... and 58 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7b7aad1...78bc627. Read the comment docs.

@aaron-prindle
Copy link
Contributor

CI/CD failing with lint error:

pkg/skaffold/event/event.go:112:2: commentFormatting: put a space between `//` and comment text (gocritic)
	// fields are known to be nil'd out when empty
	^
FAILED hack/golangci-lint.sh

The kokoro failure is the known flake -

    panic.go:617: time="2021-09-13T15:49:17Z" level=warning msg="Assuming the secret e2esecret is mounted inside Kaniko pod with the filename kaniko-secret. If your secret is mounted at different path, please specify using config key `pullSecretPath`.\nSee https://skaffold.dev/docs/references/yaml/#build-cluster-pullSecretPath" subtask=-1 task=Build
        Get "https://34.135.186.128:5000/v2/": dial tcp 34.135.186.128:5000: i/o timeout; Get "http://34.135.186.128:5000/v2/": dial tcp 34.135.186.128:5000: i/o timeout
--- FAIL: TestBuildKanikoInsecureRegistry (693.43s)

@briandealwis
Copy link
Member Author

Thanks @aaron-prindle — I left that code in to see as the integration tests were failing when cloning the state using the JSON marshaller as State.StatusCheckState.Resources would be nil'd out when empty. Seems to be ok using protobuf's Clone() though 🎉

@briandealwis briandealwis marked this pull request as ready for review September 13, 2021 19:02
@briandealwis briandealwis requested a review from a team as a code owner September 13, 2021 19:02
@briandealwis
Copy link
Member Author

Marking as blocked: hold off until next release.

Copy link
Contributor

@MarlonGamez MarlonGamez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple quick questions but everything looks good

Comment on lines +61 to +65
var suggestions2 protoV2.Suggestion
suggestions2.Action = suggestion.Action
suggestions2.SuggestionCode = suggestion.SuggestionCode
suggestions2.Action = suggestion.Action
suggestionsV2[i] = &suggestions2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

feel free to add a todo here for me to put this into a function since we use it a couple times

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there other places?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there are just 2, here and in func V2fromV1() a bit further down in the same file

Copy link
Contributor

@MarlonGamez MarlonGamez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the clarifications @briandealwis

@briandealwis briandealwis merged commit 7abfe09 into GoogleContainerTools:main Sep 15, 2021
@briandealwis briandealwis deleted the grpc-gateway-v2 branch September 15, 2021 21:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update proto toolchain
4 participants