Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

udpa: filesystem list collection support for inline entries. #13028

Merged
merged 13 commits into from
Sep 11, 2020

Conversation

htuch
Copy link
Member

@htuch htuch commented Sep 9, 2020

This is the first PR towards filesystem support for file:/// URLs
compatible with #11264. Currently it plumbs in only LDS filesystem
support for list collections with only inline entries.

Risk level: Low (opt in)
Testing: Unit and integration tests added.

Signed-off-by: Harvey Tuch [email protected]

This is the first PR towards filesystem support for file:/// URLs
compatible with envoyproxy#11264. Currently it plumbs in only LDS filesystem
support for list collections with only inline entries.

Risk level: Low (opt in)
Testing: WiP, will finalize PR when these are ready.

Signed-off-by: Harvey Tuch <[email protected]>
@repokitteh-read-only
Copy link

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to api/envoy/.
CC @envoyproxy/api-watchers: FYI only for changes made to api/envoy/.

🐱

Caused by: #13028 was opened by htuch.

see: more, trace.

@htuch
Copy link
Member Author

htuch commented Sep 9, 2020

@chaoqin-li1123 FYI re: common plumbing from LDS to subscription factory.

@stevenzzzz
Copy link
Contributor

/cc stevenzzzz

@htuch htuch changed the title [WiP] udpa: filesystem list collection support for inline entries. udpa: filesystem list collection support for inline entries. Sep 9, 2020
@htuch htuch marked this pull request as ready for review September 9, 2020 20:33
@htuch
Copy link
Member Author

htuch commented Sep 10, 2020

The file:// encoding for Windows is a bit broken, since it needs to be file:///c:/foo/bar. If we do this, when we do generic URI decoding, we get /c:/foo/bar. @wrowe @sunjayBhatia do you folks know how this is usually handled on Windows? Is there a need to special case URI decoding, etc.

Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice to see this coming together. Some small comments but generally LGTM. Thank you!

/wait

source/common/config/decoded_resource_impl.h Outdated Show resolved Hide resolved
source/common/config/filesystem_subscription_impl.cc Outdated Show resolved Hide resolved
source/common/config/filesystem_subscription_impl.cc Outdated Show resolved Hide resolved
return std::make_unique<Config::FilesystemCollectionSubscriptionImpl>(
dispatcher_, path, callbacks, resource_decoder, stats, validation_visitor_, api_);
}
default:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this checked by schema? If not, maybe better to not use default and specify all of the options so we get a compile error if something is added?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an enum, so you have weird values like ResourceLocator_Scheme_ResourceLocator_Scheme_INT_MIN_SENTINEL_DO_NOT_USE_

source/server/lds_api.cc Outdated Show resolved Hide resolved
@@ -79,6 +81,222 @@ TEST_F(FilesystemSubscriptionImplTest, UpdateTimeChangedOnUpdateSuccess) {
EXPECT_TRUE(statsAre(3, 2, 0, 0, 0, TEST_TIME_MILLIS + 1, 7148434200721666028, "0"));
}

// TODO(htuch): Add generic test harness support for collection subscriptions so that we can test
// gRPC/HTTP transports similar to below.
class FilesystemCollectionSubscriptionImplTest : public testing::Test,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering, per my comment above in the prof code about shared code, could the tests here actually be a TEST_P that somehow creates a collection or not and just runs all the same tests for the most part to verify the same stats, etc.?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I plan on a followup PR to address this as per the TODO.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, seems like a lot of duplication but up to you.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The more time I spent looking at these tests and what coverage they provide, the more I realized they don't actually do a whole lot for some of the unhappy path cases that I've been adding in this PR. I definitely want to reform this entire set of tests before moving forward with any further UDPA PRs, since they are key to being able to validate implementations for both filesystem and gRPC. This is probably going to be a series of PRs that don't have a ton to do with the filesystem list collection support per se, but they would benefit from having it landed as a forcing function.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure sounds fine to do as followup cleanups if you are tracking.

Signed-off-by: Harvey Tuch <[email protected]>
Signed-off-by: Harvey Tuch <[email protected]>
Signed-off-by: Harvey Tuch <[email protected]>
Signed-off-by: Harvey Tuch <[email protected]>
mattklein123
mattklein123 previously approved these changes Sep 11, 2020
Copy link
Member

@mattklein123 mattklein123 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Signed-off-by: Harvey Tuch <[email protected]>
@htuch htuch merged commit 108d2bc into envoyproxy:master Sep 11, 2020
@htuch htuch deleted the udpa-file-collections branch September 11, 2020 22:48
lhluo pushed a commit to lhluo/envoy that referenced this pull request Sep 11, 2020
…code

* upstream/master:
  lint: add more linters for using absl:: over std:: (envoyproxy#13043)
  udpa: filesystem list collection support for inline entries. (envoyproxy#13028)
  filter: http: jwt: implement matching for HTTP CONNECT (envoyproxy#13064)
  [fuzz] split http filter logic into a fuzzing class (envoyproxy#13016)
  xds: allow empty delta update (envoyproxy#12699)
  CacheFilter: parses the allowed_vary_headers from the cache config. (envoyproxy#12928)
  router: extend HTTP CONNECT route matching criteria (envoyproxy#13056)
  docs: clarify use of Extended CONNECT for h/2 (envoyproxy#13051)
  build: shellcheck tools/ (envoyproxy#13007)
  [fuzz] Refactored Health Checker Impl Tests (envoyproxy#13017)

Signed-off-by: Lihao Luo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants