Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: sort resources when building MeshContext #5391

Merged
merged 7 commits into from
Dec 2, 2022
Merged

fix: sort resources when building MeshContext #5391

merged 7 commits into from
Dec 2, 2022

Conversation

lobkovilya
Copy link
Contributor

@lobkovilya lobkovilya commented Dec 1, 2022 •
edited by lahabana
Loading

Kuma passes /cluster/transport_socket_matches without sorting. This creates changes in CDS and unnecessary updates.

There are 2 approaches we can take to guarantee a stable snapshot:

  • Sort arrays "ad hoc" in configurers like ClientSideTLS and ProvidedEndpointClusterConfigurer
  • Guarantee that all arrays we pass to generators are stable

Today we have pieces of both approaches. For example, we sort endpoints in ProvidedEndpointClusterConfigurer but we sort the list of external services before passing them to Ingress and Egress generators.

I like the second approach a little bit more, I think if we provide stable input for generators and restrict using maps in generators package (for example, requiring using OrderedMap) we can guarantee a stable output.

Current PR adds sorting of external services for MeshGateway and DPP generators.

Checklist prior to review

  • Link to docs PR or issue --
  • Link to UI issue or PR --
  • Is the issue worked on linked? --
  • The PR does not hardcode values that might break projects that depend on kuma (e.g. "kumahq" as a image registry) --
  • The PR will work for both Linux and Windows, system specific functions like syscall.Mkfifo have equivalent implementation on the other OS --
  • Unit Tests --
  • E2E Tests --
  • Manual Universal Tests --
  • Manual Kubernetes Tests --
  • Do you need to update UPGRADE.md? --
  • Does it need to be backported according to the backporting policy? -- Yes
  • Do you need to explicitly set a > Changelog: entry here or add a ci/ label to run fewer/more tests?

Changelog: fix(xds): sort resources when building MeshContext

@lobkovilya lobkovilya requested a review from a team as a code owner December 1, 2022 09:53
lahabana
lahabana reviewed Dec 1, 2022
View reviewed changes
@lobkovilya
fix: review
eda52e3 
Signed-off-by: Ilya Lobkov <[email protected]>
@lobkovilya
fix: review
f7a955a 
Signed-off-by: Ilya Lobkov <[email protected]>
lahabana
lahabana requested changes Dec 1, 2022
View reviewed changes
Copy link
Contributor

@lahabana lahabana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm really not convinced doing the sorting in the store is the right approach. To me sorting what matters in the mesh_context_builder is sufficient and more predictible

@lobkovilya
Copy link
Contributor Author

Only mesh_context_builder is passing a store.ListOrdered option to the store, other store clients won't see sorted lists

@lobkovilya
fix: review
30db669 
Signed-off-by: Ilya Lobkov <[email protected]>
@lahabana lahabana changed the title fix: pass unsorted arrays to envoy fix: sort resources when building MeshContext Dec 2, 2022
lahabana
lahabana approved these changes Dec 2, 2022
View reviewed changes
Copy link
Contributor

@lahabana lahabana left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM except one minor comment

@lobkovilya lobkovilya enabled auto-merge (squash) December 2, 2022 12:31
@lobkovilya lobkovilya disabled auto-merge December 2, 2022 13:24
@michaelbeaumont
Copy link
Contributor

@Mergifyio backport release-2.0

@mergify
Copy link
Contributor

mergify bot commented Dec 2, 2022 •
edited
Loading

backport release-2.0

✅ Backports have been created

@lobkovilya lobkovilya merged commit ff4463e into kumahq:master Dec 2, 2022
@lobkovilya lobkovilya deleted the fix/es-order branch December 2, 2022 14:27
@mergify mergify bot mentioned this pull request Dec 2, 2022
Merged
mergify bot pushed a commit that referenced this pull request Dec 2, 2022
@lobkovilya @mergify
fix: sort resources when building MeshContext (#5391)
4577d33 
Signed-off-by: Ilya Lobkov <[email protected]>
(cherry picked from commit ff4463e)

# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/xds/server/v3/snapshot_generator_test.go
@lobkovilya
Copy link
Contributor Author

@Mergifyio backport release-1.8 release-1.7 release-1.6 release-1.5

@mergify mergify bot mentioned this pull request Dec 2, 2022
Merged
mergify bot pushed a commit that referenced this pull request Dec 2, 2022
@lobkovilya @mergify
fix: sort resources when building MeshContext (#5391)
07ef4d8 
Signed-off-by: Ilya Lobkov <[email protected]>
(cherry picked from commit ff4463e)

# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/plugins/runtime/gateway/generator.go
#	pkg/test/resources/builders/dataplane_builder.go
#	pkg/xds/server/v3/reconcile.go
#	pkg/xds/server/v3/snapshot_generator_test.go
@mergify mergify bot mentioned this pull request Dec 2, 2022
Merged
mergify bot pushed a commit that referenced this pull request Dec 2, 2022
@lobkovilya @mergify
fix: sort resources when building MeshContext (#5391)
48050aa 
Signed-off-by: Ilya Lobkov <[email protected]>
(cherry picked from commit ff4463e)

# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/plugins/runtime/gateway/gateway_route_generator_test.go
#	pkg/plugins/runtime/gateway/generator.go
#	pkg/plugins/runtime/gateway/testdata/tcp/tcp-route-no-egress.yaml
#	pkg/test/resources/builders/dataplane_builder.go
#	pkg/xds/server/v3/reconcile.go
#	pkg/xds/server/v3/snapshot_generator_test.go
@mergify mergify bot mentioned this pull request Dec 2, 2022
Merged
mergify bot pushed a commit that referenced this pull request Dec 2, 2022
@lobkovilya @mergify
fix: sort resources when building MeshContext (#5391)
8f1bd2b 
Signed-off-by: Ilya Lobkov <[email protected]>
(cherry picked from commit ff4463e)

# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/plugins/runtime/gateway/gateway_route_generator_test.go
#	pkg/plugins/runtime/gateway/generator.go
#	pkg/plugins/runtime/gateway/testdata/tcp/tcp-route-no-egress.yaml
#	pkg/test/resources/builders/dataplane_builder.go
#	pkg/xds/context/mesh_context_builder.go
#	pkg/xds/server/v3/reconcile.go
#	pkg/xds/server/v3/snapshot_generator_test.go
mergify bot pushed a commit that referenced this pull request Dec 2, 2022
@lobkovilya @mergify
fix: sort resources when building MeshContext (#5391)
8035f56 
Signed-off-by: Ilya Lobkov <[email protected]>
(cherry picked from commit ff4463e)

# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/plugins/runtime/gateway/gateway_route_generator_test.go
#	pkg/plugins/runtime/gateway/generator.go
#	pkg/plugins/runtime/gateway/testdata/tcp/tcp-route-no-egress.yaml
#	pkg/test/resources/builders/dataplane_builder.go
#	pkg/xds/context/mesh_context_builder.go
#	pkg/xds/server/v3/components.go
#	pkg/xds/server/v3/reconcile.go
#	pkg/xds/server/v3/snapshot_generator_test.go
#	pkg/xds/server/v3/testdata/dataplane.input.yaml
#	pkg/xds/sync/ingress_proxy_builder.go
@mergify mergify bot mentioned this pull request Dec 2, 2022
Merged
@mergify
Copy link
Contributor

mergify bot commented Dec 2, 2022

backport release-1.8 release-1.7 release-1.6 release-1.5

✅ Backports have been created

lobkovilya added a commit that referenced this pull request Dec 2, 2022
@lobkovilya
fix: sort resources when building MeshContext (#5391)
05653d7 
Signed-off-by: Ilya Lobkov <[email protected]>
# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/xds/server/v3/snapshot_generator_test.go
lobkovilya added a commit that referenced this pull request Dec 2, 2022
@lobkovilya
fix: sort resources when building MeshContext (#5391)
bb28fe2 
Signed-off-by: Ilya Lobkov <[email protected]>
# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/plugins/runtime/gateway/generator.go
#	pkg/test/resources/builders/dataplane_builder.go
#	pkg/xds/server/v3/reconcile.go
#	pkg/xds/server/v3/snapshot_generator_test.go
lobkovilya added a commit that referenced this pull request Dec 2, 2022
@mergify @lobkovilya
fix: sort resources when building MeshContext (backport #5391) (#5409)
d004fbc 
fix: sort resources when building MeshContext (#5391)

Signed-off-by: Ilya Lobkov <[email protected]>
# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/xds/server/v3/snapshot_generator_test.go

Co-authored-by: Ilya Lobkov <[email protected]>
lobkovilya added a commit that referenced this pull request Dec 2, 2022
@lobkovilya
fix: sort resources when building MeshContext (#5391)
f7199e3 
Signed-off-by: Ilya Lobkov <[email protected]>
# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/plugins/runtime/gateway/generator.go
#	pkg/test/resources/builders/dataplane_builder.go
#	pkg/xds/server/v3/reconcile.go
#	pkg/xds/server/v3/snapshot_generator_test.go
lobkovilya added a commit that referenced this pull request Dec 2, 2022
@mergify @lobkovilya
fix: sort resources when building MeshContext (backport #5391) (#5410)
1b3a49e 
fix: sort resources when building MeshContext (#5391)

Signed-off-by: Ilya Lobkov <[email protected]>
# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/plugins/runtime/gateway/generator.go
#	pkg/test/resources/builders/dataplane_builder.go
#	pkg/xds/server/v3/reconcile.go
#	pkg/xds/server/v3/snapshot_generator_test.go

Co-authored-by: Ilya Lobkov <[email protected]>
lobkovilya added a commit that referenced this pull request Dec 2, 2022
@lobkovilya
fix: sort resources when building MeshContext (#5391)
5b363ad 
Signed-off-by: Ilya Lobkov <[email protected]>
# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/plugins/runtime/gateway/generator.go
#	pkg/test/resources/builders/dataplane_builder.go
#	pkg/xds/server/v3/reconcile.go
#	pkg/xds/server/v3/snapshot_generator_test.go
lobkovilya added a commit that referenced this pull request Dec 2, 2022
@mergify @lobkovilya
fix: sort resources when building MeshContext (backport #5391) (#5411)
d592168 
fix: sort resources when building MeshContext (#5391)

Signed-off-by: Ilya Lobkov <[email protected]>
# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/plugins/runtime/gateway/generator.go
#	pkg/test/resources/builders/dataplane_builder.go
#	pkg/xds/server/v3/reconcile.go
#	pkg/xds/server/v3/snapshot_generator_test.go

Co-authored-by: Ilya Lobkov <[email protected]>
lobkovilya added a commit that referenced this pull request Dec 2, 2022
@lobkovilya
fix: sort resources when building MeshContext (#5391)
a515380 
Signed-off-by: Ilya Lobkov <[email protected]>
# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/plugins/runtime/gateway/generator.go
#	pkg/test/resources/builders/dataplane_builder.go
#	pkg/xds/server/v3/reconcile.go
#	pkg/xds/server/v3/snapshot_generator_test.go
lobkovilya added a commit that referenced this pull request Dec 2, 2022
@mergify @lobkovilya
fix: sort resources when building MeshContext (backport #5391) (#5412)
d8c9512 
fix: sort resources when building MeshContext (#5391)

Signed-off-by: Ilya Lobkov <[email protected]>
# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/plugins/runtime/gateway/generator.go
#	pkg/test/resources/builders/dataplane_builder.go
#	pkg/xds/server/v3/reconcile.go
#	pkg/xds/server/v3/snapshot_generator_test.go

Co-authored-by: Ilya Lobkov <[email protected]>
lobkovilya added a commit that referenced this pull request Dec 2, 2022
@mergify @lobkovilya
fix: sort resources when building MeshContext (backport #5391) (#5413)
670b195 
fix: sort resources when building MeshContext (#5391)

Signed-off-by: Ilya Lobkov <[email protected]>
# Conflicts:
#	pkg/core/resources/store/options.go
#	pkg/plugins/runtime/gateway/generator.go
#	pkg/test/resources/builders/dataplane_builder.go
#	pkg/xds/server/v3/reconcile.go
#	pkg/xds/server/v3/snapshot_generator_test.go

Co-authored-by: Ilya Lobkov <[email protected]>
@lahabana lahabana mentioned this pull request Dec 13, 2022
Closed
@shaneutt shaneutt mentioned this pull request Feb 14, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lahabana lahabana lahabana approved these changes

@jakubdyszkiewicz jakubdyszkiewicz jakubdyszkiewicz approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@lobkovilya @michaelbeaumont @lahabana @jakubdyszkiewicz