Skip to content
This repository has been archived by the owner on Jul 11, 2023. It is now read-only.

ads: fix wrong accountability of sent resources #3803

Merged
merged 1 commit into from
Jul 29, 2021

Conversation

eduser25
Copy link
Contributor

@eduser25 eduser25 commented Jul 20, 2021

This commit introduces 2 changes:

  • Knowledge of what resources a certain proxy has requested/subscribed
    to, up to a current point in time and,
  • the use of this new facility to avoid accounting for resources
    sent that might have not been requested by envoy yet, as well as using them
    to generate stub requests from ctl-driven proxy updates.

There's a potential bug in how we treated resources sent, where a
non-wildcard vertical sending more resources than the requested/subscribed ones
could make a later request be understood as an ACK.

Consider the following case:

req: [N:-, A]
resp: [N:1, A, B]  <- let's say vertical code generated more resources than requested
                      OSM would record that we last sent [A, B]
// Envoy will silently drop [B]
req [N:1, A, B]   <- Envoy is actually requesting B apart from A now
resp: // No response, OSM thinks this is an ACK

Related to #3775

Signed-off-by: Eduard Serra [email protected]
Please answer the following questions with yes/no.

  1. Does this change contain code from or inspired by another project?
    • Did you notify the maintainers and provide attribution?
      No
  2. Is this a breaking change?
    No

@eduser25 eduser25 requested a review from a team as a code owner July 20, 2021 15:54
@eduser25 eduser25 marked this pull request as draft July 20, 2021 15:54
@eduser25 eduser25 added the wip Work-in-Progress label Jul 20, 2021
@eduser25 eduser25 force-pushed the requestedResources branch from 046e6a6 to d758e9f Compare July 21, 2021 08:36
@codecov-commenter
Copy link

codecov-commenter commented Jul 21, 2021

Codecov Report

Merging #3803 (0cf7acf) into main (8a80bd2) will decrease coverage by 0.14%.
The diff coverage is 70.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3803      +/-   ##
==========================================
- Coverage   67.76%   67.61%   -0.15%     
==========================================
  Files         199      201       +2     
  Lines       11096    11337     +241     
==========================================
+ Hits         7519     7666     +147     
- Misses       3528     3620      +92     
- Partials       49       51       +2     
Flag Coverage Δ
unittests 67.61% <70.00%> (-0.15%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
pkg/envoy/types.go 33.33% <0.00%> (-16.67%) ⬇️
pkg/envoy/ads/stream.go 10.20% <44.44%> (+6.29%) ⬆️
pkg/envoy/ads/response.go 70.51% <100.00%> (+2.51%) ⬆️
pkg/envoy/proxy.go 86.27% <100.00%> (+1.16%) ⬆️
pkg/smi/health.go 63.63% <0.00%> (-7.80%) ⬇️
pkg/k8s/announcement_handlers.go 72.97% <0.00%> (-5.41%) ⬇️
pkg/validator/server.go 21.59% <0.00%> (-4.44%) ⬇️
pkg/envoy/rds/response.go 78.18% <0.00%> (-2.96%) ⬇️
pkg/envoy/sds/response.go 72.09% <0.00%> (-2.71%) ⬇️
pkg/health/health.go 63.26% <0.00%> (-2.70%) ⬇️
... and 39 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8a80bd2...0cf7acf. Read the comment docs.

@eduser25 eduser25 force-pushed the requestedResources branch from d758e9f to 63ca157 Compare July 21, 2021 09:04
@eduser25 eduser25 marked this pull request as ready for review July 21, 2021 16:03
@eduser25 eduser25 requested review from shashankram and allenlsy July 21, 2021 16:36
@eduser25
Copy link
Contributor Author

I will add a couple unit tests for the newly added functions

pkg/envoy/ads/response.go Outdated Show resolved Hide resolved
pkg/envoy/ads/stream.go Outdated Show resolved Hide resolved
pkg/envoy/ads/stream.go Outdated Show resolved Hide resolved
pkg/envoy/ads/stream.go Outdated Show resolved Hide resolved
pkg/envoy/proxy.go Outdated Show resolved Hide resolved
@eduser25 eduser25 force-pushed the requestedResources branch from 63ca157 to a460516 Compare July 23, 2021 09:44
@eduser25
Copy link
Contributor Author

eduser25 commented Jul 23, 2021

This commit unblocks #3799 (needs rebase), which will fix #3775. We couldn't do a proper Equal diff of resources before without accounting properly the sent resources.

Istio's resource diff is not a superset (I took that from go-control-plane) and is in fact an Equal diff of resources:
https://github.com/istio/istio/blob/da6178604559bdf2c707a57f452d16bee0de90c8/pilot/pkg/xds/ads.go#L466

pkg/envoy/ads/response.go Outdated Show resolved Hide resolved
This commit introduces 2 related changes:
- Knowledge of what resources a certain proxy has requested/subscribed
to, up to a current point in time and,
- the use of this new data structure to avoid accounting for resources
sent that might have not been requested by envoy.

There's a potential bug in how we treated resources sent, where a
non-wildcard vertical sending more resources than the requested/subscribed ones
could make a later request be understood as an ACK.

Related to openservicemesh#3775

Signed-off-by: Eduard Serra <[email protected]>
@eduser25 eduser25 force-pushed the requestedResources branch from 1297704 to 0cf7acf Compare July 29, 2021 23:08
@eduser25 eduser25 merged commit 0fc7e5e into openservicemesh:main Jul 29, 2021
@eduser25 eduser25 deleted the requestedResources branch July 29, 2021 23:37
eduser25 added a commit to eduser25/osm that referenced this pull request Jul 31, 2021
PR openservicemesh#3803 introduced a mapset (backed by golang map) to slice conversion,
which does not guarantee same output ordering for the same input,
thus was making a unit test fail from time to time.

This commit forces output of the slice to be alphabetically ordered,
ensuring determinism of the output slice for the same input mapset.

Signed-off-by: Eduard Serra <[email protected]>
@shashankram shashankram added the bug-fix/backport-to=v0.9 Backport to release v0.9 label Aug 16, 2021
eduser25 added a commit to eduser25/osm that referenced this pull request Aug 16, 2021
PR openservicemesh#3803 introduced a mapset (backed by golang map) to slice conversion,
which does not guarantee same output ordering for the same input,
thus was making a unit test fail from time to time.

This commit forces output of the slice to be alphabetically ordered,
ensuring determinism of the output slice for the same input mapset.

Signed-off-by: Eduard Serra <[email protected]>
eduser25 added a commit to eduser25/osm that referenced this pull request Aug 16, 2021
PR openservicemesh#3803 introduced a mapset (backed by golang map) to slice conversion,
which does not guarantee same output ordering for the same input,
thus was making a unit test fail from time to time.

This commit forces output of the slice to be alphabetically ordered,
ensuring determinism of the output slice for the same input mapset.

Signed-off-by: Eduard Serra <[email protected]>
@snehachhabria snehachhabria removed the bug-fix/backport-to=v0.9 Backport to release v0.9 label Aug 17, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants