Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sync from master #237

Merged
merged 51 commits into from
Dec 3, 2021
Merged

sync from master #237

merged 51 commits into from
Dec 3, 2021

Conversation

wangfakang
Copy link
Owner

Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]

ME-ON1 and others added 30 commits November 19, 2021 09:35
Additional Description: See #11318 for details.
Risk Level: N/A
Testing: N/A
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

Signed-off-by: Tarun Sharma <[email protected]>
https://github.com/google/quiche/compare/4f552f349..81314ecbd

$ git log 4f552f349..81314ecbd --date=short --no-merges --format="%ad %al %s"

2021-11-17 haoyuewang Internal change
2021-11-16 haoyuewang Deprecate --gfe2_restart_flag_quic_dispatcher_support_multiple_cid_per_connection_v2
2021-11-16 vasilvv Use raw hashes in WebTransportFingerprintProofVerifier.
2021-11-16 wub In TlsServerHandshaker, do not call ProofSourceHandle::SelectCertificate if QUIC connection has disconnected.
2021-11-16 quiche-dev Check for the data_deferred state before writing data for a stream.
2021-11-16 quiche-dev Consolidates references to the third-party nghttp2.h header in a single place.
2021-11-16 quiche-dev Renames Http2ErrorCode::NO_ERROR to Http2ErrorCode::HTTP2_NO_ERROR to avoid conflicts with the builtin Windows macro NO_ERROR.
2021-11-16 dschinazi Deprecate QUIC version T051
2021-11-16 quiche-dev Adds optional RST_STREAM NO_ERROR behavior after sending a fin to an incomplete request as a server.
2021-11-16 wub Deprecate --gfe2_reloadable_flag_quic_tls_restore_connection_context_in_callbacks.
2021-11-15 quiche-dev Let OgHttp2Session avoid visitor callbacks for data on unknown streams.
2021-11-15 quiche-dev Introduce NoOpHeadersHandler in OgHttp2Session.
2021-11-15 quiche-dev Extract NoOpHeadersHandler and HeaderByteListenerInterface into separate header files.
2021-11-15 quiche-dev Add testing to demonstrate nghttp2 and oghttp2 handling of data on a closed stream.
2021-11-15 quiche-dev Consolidates stream close behavior in a smaller number of places.

Signed-off-by: David Schinazi <[email protected]>
If, for a request, the host/authority header is changed when the
request is proxied, set the x-forwarded-host header as:
x-forwarded-host = append(x-forwarded-host, host)

Risk Level: Low

Testing: unit test and manual testing

Docs Changes:
Added x-forwarded-host header in HTTP header manipulation
(configuration/http/http_conn_man/headers).
Under the host_rewrite options in route_components.proto
(in v3), added that when host is rewritten, the XFH header is
appended with the original value of host header if append_x_forwarded_host
option is set.

Release Notes: router: added support for x-forwarded-host header.

Platform Specific Features: N/A

Fixes #5940

Signed-off-by: Arun Olappamanna Vasudevan <[email protected]>
* Update QUICHE from 81314ecbd to c2ddf95dc
https://github.com/google/quiche/compare/81314ecbd..c2ddf95dc

$ git log 81314ecbd..c2ddf95dc --date=short --no-merges --format="%ad %al %s"

2021-11-22 quiche-dev Adds an `emplace()` method to RunOnExit, and deletes the move and copy constructors.
2021-11-22 quiche-dev Makes a safety mechanism related to trailers configurable, and disabled by default.
2021-11-22 haoyuewang Internal change
2021-11-22 haoyuewang Internal change
2021-11-22 fayang Do not reuse tokens received in NEW_TOKEN frames for different connection attempts by: 1) add source address token to QuicClientSessionCache, 2) Clear token after use.
2021-11-19 fayang Add mutable_session_cache() to QuicCryptoClientConfig.
2021-11-19 fayang Move QuicClientSessionCache in chromium to shared code by making following changes: 1) Rename FlushInvalidEntries() and Flush() to RemoveExpiredEntries() and Clear(), respectively. 2) Remove clock_ and SetClockForTesting(), instead, pass in QuicWallTime (use ToUNIXSeconds to get seconds from UNIX epoch because SSL_SESSION_get_time returns seconds from UNIX epoch) to Lookup and RemoveExpiredEntries. 3) Remove memory_pressure_listener_ and OnMemoryPressure(). In chromium, memory_pressure_listener_ and OnMemoryPressure() will be moved to QuicStreamFactory::QuicCryptoClientConfigOwner. 4) Replace base::LRUCache with QuicLRUCache (and add hasher for QuicServerId).
2021-11-19 quiche-dev Move the logic of submitting SETTINGS from OgHttp2Adapter to OgHttp2Session.
2021-11-19 haoyuewang Add QUIC_EXPORT_PRIVATE to RawSha256 method since it is used directly in third_party/quic/quic_transport/web_transport_fingerprint_proof_verifier_test.cc
2021-11-19 fayang Let QuicLRUCache take explicit hasher. Also added various iterators, change Lookup to return iterator and add Erase function.
2021-11-19 quiche-dev Validates that a HEADERS frame with a 100-199 status code does not contain a fin.

Signed-off-by: Ryan Hamilton <[email protected]>
Signed-off-by: Dan Zhang [email protected]

Commit Message: hard code prefer_gro to false. The performance of GRO hasn't been evaluated yet, so it shouldn't be default on.

Risk Level: low
Testing: existing tests pass
Since the requested_server_name_ was moved to ConnectionInfoProvider, this field is useless now.

Risk Level: low
Testing: n/a
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: no

Signed-off-by: He Jie Xu <[email protected]>
An internal 3pp scan reports that six version is older and a newer version is available. However as six is no longer used, attempting to remove it.

Risk Level: low
Testing: local build and pre checks done

Signed-off-by: Faseela K <[email protected]>
Bumps [slack-sdk](https://github.com/slackapi/python-slack-sdk) from 3.11.2 to 3.12.0.
- [Release notes](https://github.com/slackapi/python-slack-sdk/releases)
- [Commits](slackapi/python-slack-sdk@v3.11.2...v3.12.0)

---
updated-dependencies:
- dependency-name: slack-sdk
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [charset-normalizer](https://github.com/ousret/charset_normalizer) from 2.0.7 to 2.0.8.
- [Release notes](https://github.com/ousret/charset_normalizer/releases)
- [Changelog](https://github.com/Ousret/charset_normalizer/blob/master/CHANGELOG.md)
- [Commits](jawah/charset_normalizer@2.0.7...2.0.8)

---
updated-dependencies:
- dependency-name: charset-normalizer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* kafka: upgrade librdkafka (used by mesh-filter)
* kafka: upgrade dpkp (used in broker- & mesh-filter integration tests)
* kafka: upgrade kafka server binary (user in broker- & mesh-filter integration tests)
* kafka: upgrade kafka dependency (used to generated protocol code)

Signed-off-by: Adam Kotwasinski <[email protected]>
Bumps [frozendict](https://github.com/Marco-Sulla/python-frozendict) from 2.0.7 to 2.1.0.
- [Release notes](https://github.com/Marco-Sulla/python-frozendict/releases)
- [Commits](Marco-Sulla/python-frozendict@v2.0.7...v2.1.0)

---
updated-dependencies:
- dependency-name: frozendict
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Risk Level: Low
Testing: new integration test
Docs Changes: made API more clear when requests count as connections :-/
Release Notes: inline
Fixes #19033

Signed-off-by: Alyssa Wilk <[email protected]>
https://github.com/google/quiche/compare/c2ddf95dc..7f2d442e3

$ git log c2ddf95dc..7f2d442e3 --date=short --no-merges --format="%ad %al %s"

2021-11-24 wub Replace --gfe2_reloadable_flag_quic_add_cached_network_parameters_to_address_token by --gfe2_reloadable_flag_quic_add_cached_network_parameters_to_address_token2.
2021-11-23 fayang Internal change

Risk Level: Low
Testing: Unit Tests
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

Signed-off-by: Ryan Hamilton <[email protected]>
Commit Message: drain only once when reached max_requests_per_connection
Additional Description: fixes #19045
Risk Level: low
Testing: unit test
Docs Changes: n/a

Signed-off-by: YaoZengzeng <[email protected]>
pims and others added 21 commits November 30, 2021 09:45
* ext-authz: fix missing UAEX flag on Denied CheckResponse

This fixes a bug in which the UAEX flag is not set prior to calling
`callback->sendLocalReply(...)`.

Signed-off-by: Tim Bart <[email protected]>
Bumps [sphinx](https://github.com/sphinx-doc/sphinx) from 4.3.0 to 4.3.1.
- [Release notes](https://github.com/sphinx-doc/sphinx/releases)
- [Changelog](https://github.com/sphinx-doc/sphinx/blob/4.x/CHANGES)
- [Commits](sphinx-doc/sphinx@v4.3.0...v4.3.1)

---
updated-dependencies:
- dependency-name: sphinx
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Provide the ability to filter stats to be flushed to sinks to reduce CPU usage for the periodic stats aggregation process.

Commit Message:

Additional Description:

Envoy stats are periodically flushed to stat sinks (default cadence of 5s) on the main thread. The number of stats scales linearly with the number of clusters, as approximately 100 stats are replicated for each cluster. For high counts of clusters (order of 10k), the flushing of stats dominates CPU usage on the main thread. Being tied up in stats flushing can prevent the main thread from processing xDS updates in a timely manner, or even starve worker threads of CPU if the CPU is overcommitted.

Usually, the number of stats of interest can be an order of magnitude lower than the number of stats. There is a mechanism to reject unwanted stats, but doing so will also make them unavailable for viewing in the admin console, which could hinder debuggability. Further, Envoy actually needs some of its stats to run (see for eg. #14610) which is currently an open bug.

See the design doc below for more details:
https://docs.google.com/document/d/1lzMvRlU5xY0yezpqA75N6kU747GY7I_WeGpBXiPaP5M/edit#heading=h.xgjl2srtytjt

Risk Level: Low
Testing: Added tests
Docs Changes: NA
Release Notes: NA
Platform Specific Features: NA

See below benchmark results from //test/server:server_stats_flush_benchmark
```
----------------------------------------------------------------------------------
Benchmark                                        Time             CPU   Iterations
----------------------------------------------------------------------------------
bmFlushToSinks/10                            0.003 ms        0.003 ms       247626
bmFlushToSinks/100                           0.019 ms        0.019 ms        36474
bmFlushToSinks/1000                          0.193 ms        0.193 ms         3622
bmFlushToSinks/10000                          2.25 ms         2.25 ms          299
bmFlushToSinks/100000                         61.8 ms         61.8 ms           10
bmFlushToSinks/1000000                        1212 ms         1212 ms            1
bmFlushToSinksWithPredicatesSet/10           0.001 ms        0.001 ms       496056
bmFlushToSinksWithPredicatesSet/100          0.007 ms        0.007 ms       104775
bmFlushToSinksWithPredicatesSet/1000         0.067 ms        0.067 ms        10411
bmFlushToSinksWithPredicatesSet/10000        0.704 ms        0.704 ms          938
bmFlushToSinksWithPredicatesSet/100000        28.0 ms         28.0 ms           25
bmFlushToSinksWithPredicatesSet/1000000        484 ms          484 ms            2
```

Signed-off-by: Pradeep Rao <[email protected]>
Commit Message: remove negative_capacity_ to make the code easier to understand
Additional Description: fixes part I of #18880
Risk Level: high
Testing: unit
Docs Changes: n/a

Signed-off-by: YaoZengzeng <[email protected]>
Risk Level: n/a (test only)
Testing: new integration test
Docs Changes: n/a
Release Notes: n/a
Signed-off-by: Alyssa Wilk <[email protected]>
This bug existed previously, but with the recent reuse port
as the default change is now more obvious. Previously, we would
allow multiple listeners to listen on the same port, which is
obviously very wrong. This change blocks that at config load time.

Fixes #19099

Signed-off-by: Matt Klein <[email protected]>
…TTEMPT_COUNT% (#19150)

Adds a new command operator called %UPSTREAM_REQUEST_ATTEMPT_COUNT% in the access logs which can be used to retrieve the number of times given request got attempted upstream. 

Fixes #18870

Signed-off-by: Rohit Agrawal <[email protected]>
This PR establishes the ability to filter access log production via CEL expressions over the set of Envoy attributes. This can simply the creation of Envoy access log filters, allowing complex tailoring.

Risk Level: low
Testing: unit
Docs Changes: included
Release Notes: updated

Signed-off-by: Douglas Reid <[email protected]>
* Added missing check outside lambda

Signed-off-by: tangsaidi <[email protected]>
When we fail to get an upstream connection (e.g.: PoolFailureReason::Overflow)
there's no need to close the downstream connection, since the request never
made it through. So we keep it open and avoid an issue that happens when
closing remote connections after a local response - see below.

Risk Level: low
Testing: updated unit tests
Docs Changes: n/a
Release Notes: added

Signed-off-by: Raul Gutierrez Segales <[email protected]>
Other properties check for non-null `info` before attempting
to read since it can be null.

Signed-off-by: LaVigne, Scott <[email protected]>
…19169)

Also, benchmark to cover buffer account use.

Signed-off-by: Antonio Vicente <[email protected]>
@wangfakang wangfakang merged commit 6bb0267 into wangfakang:master Dec 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.