Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tls: support BoringSSL private key async functionality #6326

Merged
merged 63 commits into from
Aug 22, 2019

Conversation

ipuustin
Copy link
Member

@ipuustin ipuustin commented Mar 20, 2019

This PR adds BoringSSL private key API abstraction, as discussed in #6248. All comments and discussion is welcomed to get the API sufficient for most private key API tasks.

The PR contains the proposed API and the way how it can be used from ssl_socket.h. Also there is some code showing how the PrivateKeyMethodProvider is coming from TLS certificate config. Two example private key method providers are included in the tests.

Description: tls: support BoringSSL private key async functionality
Risk Level: medium
Testing: two basic private key provider implementation
Docs Changes: TLS arch doc, cert.proto doc

@ipuustin ipuustin force-pushed the private-key branch 2 times, most recently from 8e2035b to 86a859f Compare March 20, 2019 18:17
@ipuustin ipuustin changed the title tls: support BoringSSL private key async functionality RFC: tls: support BoringSSL private key async functionality Mar 20, 2019
ops_ = provider_->getPrivateKeyOperations(*this, callbacks_->connection().dispatcher());
Ssl::PrivateKeyMethodSharedPtr private_key_methods = ops_->getPrivateKeyMethods(ssl_.get());
if (private_key_methods) {
SSL_set_private_key_method(ssl_.get(), private_key_methods.get());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this function should be incapsulated somewhere inside source/extensions/transport_socket_tls/private_key to avoid messing with OpenSSL's API if we add conditional OpenSSL builds.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right -- the setup could also be in a PrivateKeyImpl class or similar. On the other hand the whole concept of BoringSSL private key calls doesn't make sense if OpenSSL is used.

Opens:

  1. Validate the thread model. Now there is an unique PrivateKeyOperations
     instance for each connection, but the PrivateKeyOperationsProvider is shared.
     This makes the dispatcher model easier and the PrivateKeyOperations lifecycle
     can be tied with the caller.
  2. How to get the private key to the provider? Just let the provider use
     SSL_get_privatekey()?
  3. Does SDS require any special handling?
  4. Should we expose BoringSSL primitives (such as 'SSL *ssl') in the API?
  5. Automatic registration of PrivateKeyOperationsProvider extensions to the
     PrivateKeyOperationsManager. We need a
     NamedPrivateKeyOperationsProviderConfigFactory?
  6. Is the API sufficient for all private key users?

Signed-off-by: Ismo Puustinen <[email protected]>
The private key operations manager allows extensions to register
private key operations provider factories. These factories in turn
create providers for individual SSL contexts.

Signed-off-by: Ismo Puustinen <[email protected]>
@ipuustin
Copy link
Member Author

@htuch @PiotrSikora I added a second patch which adds support for registering private key provider factories. Any comments whether this is going the right way would be helpful. I was considering between this approach and extending Connection API to have network read filters to be able to access SSL internal structures; WDYT?

@htuch htuch self-assigned this Mar 27, 2019
@@ -147,6 +147,11 @@ message TlsSessionTicketKeys {
repeated core.DataSource keys = 1 [(validate.rules).repeated .min_items = 1];
}

message PrivateKeyOperations {
string private_key_provider = 1;
core.ConfigSource private_key_provider_config = 2;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Starting from first principles on this review (just noticed it when you tagged me). How come we need to use ConfigSource here? Wouldn't the correct thing be a typed config, i.e. an Any, similar to how we configure other extensions? I think the best example for what you're trying to integrate here is something like the gRPC credential extension, see https://github.com/envoyproxy/envoy/blob/master/source/common/grpc/google_grpc_creds_impl.h

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. I added a patch which does just that. I was bit unsure if the plan is to use the combined Struct/Any approach for new extension types, or just go with Any.

I'm currently thinking that the configuration conversion would be done in the private key provider extensions something like this:

Ssl::PrivateKeyOperationsProviderSharedPtr TestPrivateKeyOperationsFactory::createPrivateKeyOperationsProviderInstance(
    const envoy::api::v2::auth::PrivateKeyOperations& message,
    Server::Configuration::TransportSocketFactoryContext& private_key_provider_context) {
  ProtobufTypes::MessagePtr proto_config = std::make_unique<test::TestPrivateKeyOperationsProto>();

  Config::Utility::translateOpaqueConfig(message.typed_config(), ProtobufWkt::Struct(), *proto_config);
  const test::TestPrivateKeyOperationsProto config = MessageUtil::downcastAndValidate<const test::TestPrivateKeyOperationsProto&>(*proto_config);

  return std::make_shared<TestPrivateKeyOperationsProvider>(config);
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you can factor some of this out into an extension factory base, you can see what was done for L7 filters at https://github.com/envoyproxy/envoy/blob/master/source/extensions/filters/http/common/factory_base.h.

@htuch
Copy link
Member

htuch commented Apr 2, 2019

Some answers to questions above:

Validate the thread model. Now there is an unique PrivateKeyOperations instance for each connection, but the PrivateKeyOperationsProvider is shared. This makes the dispatcher model easier and the PrivateKeyOperations lifecycle can be tied with the caller.

Yep, this is how to do things in Envoy. It's fine to share the provider, assuming it's essentially a stateless (or immutable) factory, and all the private key operation state lives thread-local.

How to get the private key to the provider? Just let the provider use SSL_get_privatekey()?

Isn't this highly dependent on the extension? Won't some have their private key material living in an HSM?

Does SDS require any special handling?

SDS is orthogonal. It's inherently about fetching keys from a remote server and then using them for crypto, rather than delegating to dedicated crypto hardware. Maybe in theory you could imagine SDS being implemented behind this new interface, but I think what we have today works well in parallel to offload.

Should we expose BoringSSL primitives (such as SSL *ssl) in the API?

Ideally what is being built works for both OpenSSL and BoringSSL, or some abstraction exists that can support this. I have no idea of the varying offload implementations, but just throwing this out there.

Copy link
Member

@htuch htuch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I've taken a further look and have a few more suggestions and questions.

string provider_name = 1 [(validate.rules).string.min_bytes = 1];

// Private key operations provider specific configuration.
google.protobuf.Any typed_config = 2;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lizan should we continue to require Struct untyped option, or is it fine to have new future looking extensions go Any-only at this point. My intuition on the protobuf hashing issue is that this is still problematic, but maybe we have confluence between that fix and when a feature like this goes prod ready?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the answer to this thread is we should support both, since Any is not production ready yet and Struct has been undeprecated.

@@ -69,6 +70,12 @@ class ContextConfig {
* @param callback callback that is executed by context config.
*/
virtual void setSecretUpdateCallback(std::function<void()> callback) PURE;

/**
* @return the configured BoringSSL private key operations provider.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any thoughts on how this can work for OpenSSL implementations?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't - private key operations are a BoringSSL concept. OpenSSL uses a bit similar mechanism called asynchronous mode, which makes almost all operations potentially asynchronous. It's up to the backend (an OpenSSL engine) to implement the operations in an asynchronous way. OpenSSL provides a file descriptor to tell when the operation is ready.

The benefit of OpenSSL model is that many more operations can be performed asynchronously and also the possibility to have dynamic shared objects (engines) with a specified ABI to run the asynchronous operations. Also, if an asynchronous engine isn't used, the library falls back to synchronous operation. The drawback is that the OpenSSL library needs to be dynamically linked to Envoy in order to load external engines. Also, the ASYNC API is not present in BoringSSL.

I wrote a patch on top of @bdecoste's OpenSSL Envoy patches some time back to enable the OpenSSL asynchronous mode:
ipuustin@41bc5be

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any hope of unifying these behind a common Envoy abstraction? That would be ideal, but obviously not if it needs to be a mega-project to make happen. Thoughts?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's my understanding that the BoringSSL and OpenSLL support will be implemented as separate TLS extensions. This means that the way the library is used will stay "hidden" in the implementations. I think the abstraction could be done in two different possible scopes:

  1. Write support to loading OpenSSL Engines to OpenSSL TLS extension. Abstract the configuration and callback interface.
  2. Find functions corresponding to the BoringSSL private key ops from OpenSSL ASYNC API, and provide a way to do "OpenSSL mini-engines" which would have the same functionality as the BoringSSL private key operations.

The problem with the first item is that it's already pretty well abstracted: there is a bunch of OpenSSL engines out there, and the benefit of Envoy knowing about them would be mainly to pass them configuration values. Envoy can use the dynamic engines quite easily (the only requirement is linking the OpenSSL library dynamically into Envoy), and there is Dispatcher access from the Connection interface to the TLS extension to facilitate the async event channel. The OpenSSL engines can also be configured from outside Envoy, by using OpenSSL configuration files (see https://www.openssl.org/docs/man1.1.1/man5/config.html). Writing the OpenSSL TLS extension to use asynchronous mode from day one would be a good idea.

The second item would mean in a sense that there would be feature parity between OpenSSL and BoringSSL implementations regarding private key operations. The OpenSSL would be pretty handicapped in this case, since the async engine support is more complete than BoringSSL private key operations. Using the QAT HW acceleration as an example, OpenSSL QAT engine (https://github.com/intel/QAT_Engine) can accelerate also EDCHE and some crypto operations, while the BoringSSL private key support is limited to RSA decryption and RSA+ECDSA signing. I think people would gravitate using the full OpenSSL engines rather than writing the Envoy-specific OpenSSL "mini-engines".

TL;DR: it feels to me that having a common abstraction for OpenSSL engines and BoringSSL private key operations might not have enough benefits compared to letting the TLS extensions set up the asynchronous operations as they please. However, I've had only limited exposure to OpenSSL engines, so it could be there are other considerations that I'm not aware of. Maybe there would be a special Envoy configuration Protobuf type for the TLS extensions, so that they could be configured separately? This could mean OpenSSL protobuf configuration message having a submessage for setting up the requested OpenSSL engine(s).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ipuustin thanks for the analysis. My thinking was that if X wanted to write a new private key offload extension Y for X's HSM, then it would be nice to have Y work no matter whether you go with BoringSSL or OpenSSL TLS. If there are already a number of OpenSSL engines out there that only require a light wrapping for Envoy, and OpenSSL is strictly more expressive than BoringSSL, I think there's not much hope or point of reuse.

So, I think this is resolved, we go with your current approach. Thanks again for taking the time to provide such a detailed discussion.

@@ -124,6 +133,21 @@ Network::IoResult SslSocket::doRead(Buffer::Instance& read_buffer) {
return {action, bytes_read, end_stream};
}

void SslSocket::complete(Envoy::Ssl::PrivateKeyOperationStatus status) {
if (status == Envoy::Ssl::PrivateKeyOperationStatus::Success) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What kind of threading guarantees does BoringSSL or OpenSSL provide? Do we know that the thread that private key operations are run on is the same as the invoking thread? Should complete post to the dispatcher to continue, or is that the job of the extension? This is one place that I'm not 100% sure of the thread correctness.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The BoringSSL operations are driven by the caller: BoringSSL's private key complete function will be called when the SSL_do_handshake() is called the second time. However, the complete() function (indicating that the asynchronous operation is ready and ssl_socket.cc needs to call the handshake again) might in principle be called from whatever thread the private key operations extension happens to use. I believe the correct semantics is to define this to be an error, and expect the extension to call the complete() function from the worker thread context.

Should we add ASSERT(isThreadSafe()); to SslSocket::complete() the way it is done in dispatcher_impl.cc?

I just wrote a (rough) implementation of a QAT private key extension to see if it fits the private key API. There I'm using a pipe and dispatcher to make sure that the callback thread is correct, because QAT device polling is done from a different thread. See the code here: https://github.com/ipuustin/envoy/blob/qat-provider/source/extensions/private_key_operations_providers/qat/qat_private_key_provider.cc#L20

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think it should be an error to SslSocket::complete() on a different thread. The Envoy pattern for handling this is to do a post to the original thread's dispatcher with the completion as the body. The only cross-thread safe dispatcher operation is post().

Also nicer handling of Protobuf message existence.

Signed-off-by: Ismo Puustinen <[email protected]>
@@ -147,6 +148,15 @@ message TlsSessionTicketKeys {
repeated core.DataSource keys = 1 [(validate.rules).repeated .min_items = 1];
}

message PrivateKeyOperations {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these going to be BoringSSL specific or an opaque placeholder for both (incompatible) types of private key operations (OpenSSL engines and BoringSSL)?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OpenSSL engines are much more than just private key operations, and I don't think treating them as alternatives is going to result in a clean API.

@@ -147,6 +148,15 @@ message TlsSessionTicketKeys {
repeated core.DataSource keys = 1 [(validate.rules).repeated .min_items = 1];
}

message PrivateKeyOperations {
// Private key operations provider name. The name must match a
// supported private key operations provider type.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typically we are making these names reverse DNS, e.g. com.acme.foo.bar FWIW. Inbuilt extensions can live in the envoy.foo.bar namespace.

@@ -315,6 +325,8 @@ message CommonTlsContext {
repeated string alpn_protocols = 4;

reserved 5;

PrivateKeyOperations private_key_operations = 9;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems effectively a oneof with TlsCertificate.private_key. We can't promote it to a oneof without breaking the API, but should this field live next to that in this file and have a comment indicating the exclusive nature of these fields?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this should definitely be a part of TlsCertificate, so that it could be configured on a per-certificate basis.

Ideally, this would be a oneof between private_key and private_key_method, which works well for a lot of use cases (remote oracle, etc.) and is more generic, but it makes this use case (crypto in separate thread) a bit awkward, since then we would need to pass private_key to private_key_method via different means.

Thoughts?

(Using private_key_method instead of private_key_operations to align with the BoringSSL name of this feature, though I don't feel too strongly about it).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't do oneof for wire compat reasons anyway, so you have flexibility here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was under the impression that you can add oneof to non-repeatable fields and stay wire-compatible?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wire compatible yes, but it breaks Go language bindings. We've had a request from the gRPC team to avoid doing this gratuitously, this seems a reasonable accommodation to make. See #6271.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I wasn't aware of that, thanks! Yes, it makes sense to accommodate them (though, it's unfortunate that language bindings are source of this restriction).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, I'll move the private key operations provider to TlsCertificate. I'll have to set up the SSL socket bit differentlythen (because I'll need to initialize the private key methods for each SSL_CTX).

include/envoy/ssl/context_manager.h Show resolved Hide resolved
namespace Envoy {
namespace Server {
namespace Configuration {
class TransportSocketFactoryContext;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this to break an include dependency loop?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's why this is needed -- Bazel will complain if this is done with a real dependency.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Arguably we could do some header refactoring to fix this, but I think this is OK if this is a one off.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Just for the record, the dep loop would be this:

    //include/envoy/ssl:context_manager_interface
    //include/envoy/ssl:context_config_interface
.-> //include/envoy/ssl:tls_certificate_config_interface
|   //include/envoy/ssl/private_key:private_key_interface
|   //include/envoy/server:transport_socket_config_interface
|   //include/envoy/secret:secret_manager_interface
|   //include/envoy/secret:secret_provider_interface
`-- //include/envoy/ssl:tls_certificate_config_interface

if (status == Envoy::Ssl::PrivateKeyOperationStatus::Success) {
ENVOY_CONN_LOG(debug, "async handshake complete", callbacks_->connection());
if (!handshake_complete_) {
// It's possible that the async call comes in later, but the handshare has been retried from
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: handshake

@htuch
Copy link
Member

htuch commented Apr 10, 2019

@PiotrSikora do you want to take a pass and do some design level feedback? I'm good with the high-level now, would be good to allow @ipuustin to commit on it for implementation.

@@ -147,6 +148,15 @@ message TlsSessionTicketKeys {
repeated core.DataSource keys = 1 [(validate.rules).repeated .min_items = 1];
}

message PrivateKeyOperations {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OpenSSL engines are much more than just private key operations, and I don't think treating them as alternatives is going to result in a clean API.

@@ -315,6 +325,8 @@ message CommonTlsContext {
repeated string alpn_protocols = 4;

reserved 5;

PrivateKeyOperations private_key_operations = 9;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this should definitely be a part of TlsCertificate, so that it could be configured on a per-certificate basis.

Ideally, this would be a oneof between private_key and private_key_method, which works well for a lot of use cases (remote oracle, etc.) and is more generic, but it makes this use case (crypto in separate thread) a bit awkward, since then we would need to pass private_key to private_key_method via different means.

Thoughts?

(Using private_key_method instead of private_key_operations to align with the BoringSSL name of this feature, though I don't feel too strongly about it).

include/envoy/ssl/context_manager.h Show resolved Hide resolved
provider_ = ctx_->getPrivateKeyOperationsProvider();
if (provider_) {
ops_ = provider_->getPrivateKeyOperations(*this, callbacks_->connection().dispatcher());
if (ops_) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be fatal, not ignored (same for private_key_methods below).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the logic a bit. Originally the idea was that there might be a fallback to software processing, but now with the potential replacing of private_key with private_key_method this should be fatal.


if (status == Envoy::Ssl::PrivateKeyOperationStatus::Success) {
ENVOY_CONN_LOG(debug, "async handshake complete", callbacks_->connection());
if (!handshake_complete_) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this method be ever called with handshake_complete_ == true? Perhaps this should be an ASSERT() instead?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, changed to ASSERT().

Copy link
Member Author

@ipuustin ipuustin May 15, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, changed async_handshake_in_progress_ check to ASSERT(). I think it's possible that (depending on how the private key method provider is implemented) there's a race between the async callback and a doHandshake() call from doWrite() or doRead(). In that case it could be that the handshake is already completed and an ASSERT for !handshake_complete_ would be wrong.

@@ -89,6 +94,9 @@ class SslSocket : public Network::TransportSocket,
mutable std::string cached_sha_256_peer_certificate_digest_;
mutable std::string cached_url_encoded_pem_encoded_peer_certificate_;
mutable std::string cached_url_encoded_pem_encoded_peer_cert_chain_;
Envoy::Ssl::PrivateKeyOperationsProviderSharedPtr provider_;
Envoy::Ssl::PrivateKeyOperationsPtr ops_;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to store this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The provider is no longer needed. The ops_ (or actually a vector now) is meant for connecting the operations lifecycle with the SslSocket lifecycle.

* @param status is "Success" or "Failure" depending on whether the private key operation was
* successful or not.
*/
virtual void complete(PrivateKeyOperationStatus status) PURE;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit worried about the case where the connection (and SslSocket) goes away while waiting for the async handshake to complete, and then the complete() callback is called, resulting in crash.

Do we have any protections against that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Originally the idea was that the PrivateKeyOperations lifecycle would be tied to the SslSocket lifecycle. I'll have to do this bit differently now when the operations are tied to certificates (which may change on-the-fly if multiple certificate support is used).

@htuch htuch added the waiting label Apr 15, 2019
@ipuustin
Copy link
Member Author

I moved the private key method support into the TlsCertificate message (from TlsContext). Some things are not yet very well tested, such as if the multiple certificate support works as expected. I changed the naming conventions a bit, but I'll need to think about it still some more.

Copy link
Member

@htuch htuch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The move to TlsCertificate LGTM, will wait for the other feedback and more progress to review again.
/wait

* function parameter, so this enables the private key method provider
* to use SSL connection custom data fields instead.
*/
virtual bool associateWithSsl(SSL* ssl) PURE;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why isn't this part of PrivateKeyOperationsCallbacks? I.e. someone implementing the lifecycle callbacks around an async method should also be able to take an initial SSL binding call as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the associateWithSsl() function because the SSL object pointer was already available in the provider call. I also did a bunch of renaming effort which should hopefully move the naming conventions closer to the BoringSSL.

Signed-off-by: Ismo Puustinen <[email protected]>
@repokitteh-read-only
Copy link

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to api/.

🐱

Caused by: #6326 was synchronize by ipuustin.

see: more, trace.

@repokitteh-read-only
Copy link

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to api/.

🐱

Caused by: #6326 was synchronize by ipuustin.

see: more, trace.

Signed-off-by: Ismo Puustinen <[email protected]>
@repokitteh-read-only
Copy link

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to api/.

🐱

Caused by: #6326 was synchronize by ipuustin.

see: more, trace.

@ipuustin
Copy link
Member Author

@rojkov @lizan : I replaced the bools with an enum, please take a look. I also merged the master to get the tests working again.

@PiotrSikora : ping!

@rojkov
Copy link
Member

rojkov commented Aug 13, 2019

Thanks @ipuustin ! I'll update openssl tls accordingly.

lizan
lizan previously approved these changes Aug 16, 2019
Copy link
Member

@lizan lizan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Appreciate the patience. Let's ship this and see how it goes? @PiotrSikora @htuch

@htuch
Copy link
Member

htuch commented Aug 19, 2019

@lizan yeah, I think we can ship as soon as we have sign-off from SSL owners on this, which is @PiotrSikora (and optionally @agl / @davidben).

PiotrSikora
PiotrSikora previously approved these changes Aug 22, 2019
Copy link
Contributor

@PiotrSikora PiotrSikora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, thanks! Sorry about the delay.

* no provider can be used with the context configuration.
*/
virtual PrivateKeyMethodProviderSharedPtr
createPrivateKeyMethodProvider(const envoy::api::v2::auth::PrivateKeyProvider& message,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: message -> config (here and in other places, but feel free to ignore it)

@@ -38,9 +39,11 @@ struct SslSocketFactoryStats {
};

enum class InitialState { Client, Server };
enum class SocketState { PreHandshake, HandshakeInProgress, HandShakeComplete, ShutdownSent };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: HandShakeComplete -> HandshakeComplete (here and in other places)

virtual PrivateKeyMethodProviderSharedPtr
createPrivateKeyMethodProvider(const envoy::api::v2::auth::PrivateKeyProvider& message,
Envoy::Server::Configuration::TransportSocketFactoryContext&
private_key_method_provider_context) PURE;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: private_key_method_provider_context -> transport_socket_factory_context or simply factory_context? (here and in other places)

* Finds and returns a private key operations provider for BoringSSL.
*
* @param message a protobuf message object containing a
* PrivateKeyProvider message.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: you can pull this into previous line.

* @param message a protobuf message object containing a
* PrivateKeyProvider message.
* @param private_key_method_provider_context context that provides components for creating and
* initializing connections for keyless TLS etc.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: ...connections using asynchronous private key operations.

@ipuustin ipuustin dismissed stale reviews from PiotrSikora and lizan via abf8758 August 22, 2019 12:10
@repokitteh-read-only
Copy link

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to api/.

🐱

Caused by: #6326 was synchronize by ipuustin.

see: more, trace.

Copy link
Contributor

@PiotrSikora PiotrSikora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@lizan
Copy link
Member

lizan commented Aug 22, 2019

Needs @htuch's approval (for requested changes and api)

Copy link
Member

@htuch htuch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

API LGTM, I will approve/merge based on my previous reviews and @lizan + @PiotrSikora recent reviews with approval. Thanks a ton @ipuustin, this is a very valuable contribution to Envoy.

@htuch htuch merged commit 9a3a234 into envoyproxy:master Aug 22, 2019
@ipuustin
Copy link
Member Author

A big thanks to all reviewers!

@rgs1
Copy link
Member

rgs1 commented Aug 26, 2019

Hmm any chance this (accidentally) changed the build system so that TLS isn't included in upstream transport sockets anymore?

After this change we get:

 transport_sockets.upstream: envoy.transport_sockets.alts,envoy.transport_sockets.tap,raw_buffer

Note no TLS...

ccc: @ipuustin @htuch

@rgs1
Copy link
Member

rgs1 commented Aug 26, 2019

cc: @fishcakez

@rgs1
Copy link
Member

rgs1 commented Aug 26, 2019

Oops, i think it's just our internal build that accidentally broke. Sorry for the false ping!

@rgs1 rgs1 mentioned this pull request Sep 4, 2019
PiotrSikora pushed a commit to PiotrSikora/envoy that referenced this pull request Oct 10, 2019
* ext_authz: add metadata_context to ext_authz filter (envoyproxy#7818)

This adds the ability to specify dynamic metadata (by namespace) to
send with the ext_authz check request. This allows one filter to
specify information that can be then used in evaluating an
authorization decision.

Risk Level: Medium. Optional feature/extension of existing filter
Testing: Unit testing
Docs Changes: Inline in attribute_context.proto and ext_authz.proto

Fixes envoyproxy#7699

Signed-off-by: Ben Plotnick <[email protected]>

* fuzz: codec impl timeout fix + speed ups (envoyproxy#7963)

Some speed-ups and validations for codec impl fuzz test:

* validate actions aren't empty (another approach would be to scrub / clean these)
* limit actions to 1024
* require oneofs

Fixes OSS-Fuzz Issue:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16481
Testing: local asan/libfuzzer exec/sec go from 25 to 50

Signed-off-by: Asra Ali <[email protected]>

* docs: more detail about tracking down deprecated features (envoyproxy#7972)

Risk Level: n/a (docs only)
Testing: n/a
Docs Changes: yes
Release Notes: no
envoyproxy#7945

Signed-off-by: Alyssa Wilk <[email protected]>

* Fix the alignement in optval of setsockopt when compiled with libc++. (envoyproxy#7958)

Description:
libc++ std::string may inline the data which results the memory is not
aligned to `void*`. Use vector instead to store the optval.

Detected by UBSAN with libc++ config. Preparation for envoyproxy#4251

Risk Level: Low
Testing: unittest locally
Docs Changes: N/A
Release Notes: N/A
Fixes envoyproxy#7968 

Signed-off-by: Lizan Zhou <[email protected]>

* security: some intra-entity and 3rd party embargo clarifications. (envoyproxy#7977)

* security: some intra-entity and 3rd party embargo clarifications.

These came up in the last set of CVEs.

Signed-off-by: Harvey Tuch <[email protected]>

* protobuf: IWYU (envoyproxy#7989)

Include What You Use fix for source/common/protobuf/message_validator_impl.h.

Signed-off-by: Andres Guedez <[email protected]>

* api: add name into filter chain (envoyproxy#7966)

Signed-off-by: Yuchen Dai <[email protected]>

* rds: validate config in depth before update config dump (envoyproxy#7956)

Route config need deep validation for virtual host duplication check, regex check, per filter config validation etc, which PGV wasn't enough.

Risk Level: Low
Testing: regression test
Docs Changes: N/A
Release Notes: N/A

Fixes envoyproxy#7939

Signed-off-by: Lizan Zhou <[email protected]>

* tls: maintain a free slot index set in TLS InstanceImpl to allocate in O(1… (envoyproxy#7979)

Signed-off-by: Xin Zhuang <[email protected]>

* redis: handle invalid ip address from cluster slots and added tests (envoyproxy#7984)

Signed-off-by: Henry Yang <[email protected]>

* protobuf: report field numbers for unknown fields. (envoyproxy#7978)

Since binary proto won't have field names, report at least the field
numbers, as per
https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.unknown_field_set#UnknownField.

Also fix minor typo encountered while doing this work.

Risk level: Low
Testing: Unit tests added/updated.

Fixes envoyproxy#7937

Signed-off-by: Harvey Tuch <[email protected]>

* Content in envoy docs does not cover whole page (envoyproxy#7993)

Signed-off-by: Manish Kumar <[email protected]>

* stats: Add option to switch between fake and real symbol-tables on the command-line. (envoyproxy#7882)

* Add option to switch between fake and real symbol-tables on the command-line.

Signed-off-by: Joshua Marantz <[email protected]>

* api config: add build rules for go protos (envoyproxy#7987)

Some BUILD files are missing build rules to generate go protos. envoyproxy/go-control-plane depends on these protos, so they should be exposed publicly. Added build rules to generate *.pb.go files.

Risk Level: Low
Testing: These rules were copied to google3 and tested internally. Unfortunately, I am having a bit of trouble with bazel build directly on these targets ("Package is considered deleted due to --deleted_packages"). Please let me know if there is a better way to test this change.

Signed-off-by: Teju Nareddy <[email protected]>

* test: don't use <experimental/filesystem> on macOS. (envoyproxy#8000)

Xcode 11 requires at least macOS 10.15 (upcoming) in order to use
either <experimental/filesystem> or C++17 <filesystem>.

Signed-off-by: Piotr Sikora <[email protected]>

*  event: adding the capability of creating an alarm with a given scope (envoyproxy#7920)

Precursor to envoyproxy#7782
Adding scope tracking functionality to the basic alarm functions.

Risk Level: Medium (should be a no-op but is a large enough refactor)
Testing: new unit tests
Docs Changes: n/a
Release Notes: n/a

Signed-off-by: Alyssa Wilk <[email protected]>

* ext authz: add dns san support for ext authz service (envoyproxy#7948)

Adds support for DNS SAN in ext authz peer validation

Risk Level: Low
Testing: Added
Docs Changes: Added
Release Notes: N/A

Signed-off-by: Rama Chavali <[email protected]>

* accesslog: don't open log file with read flag (envoyproxy#7998)

Description:
File access log shouldn't need read access for a file.

Risk Level: Low
Testing: local in mac, CI
Docs Changes:
Release Notes:
Fixes envoyproxy#7997

Signed-off-by: Lizan Zhou <[email protected]>

* protobuf: towards unifying PGV, deprecated and unknown field validation. (envoyproxy#8002)

This is part of envoyproxy#7980; basically, we want to leverage the recursive pass
that already exists for the deprecated check. This PR does not implement
the recursive behavior yet for unknown fields though, because there is a
ton of churn, so this PR just has the mechanical bits. We switch
plumbing of validation visitor into places such as anyConvert() and
instead pass this to MessageUtil::validate.

There are a bunch of future followups planned in additional PRs:
* Combine the recursive pass for unknown/deprecated check in
  MessageUtil::validate().
* Add mitigation for envoyproxy#5965 by copying to a temporary before recursive
  expansion.
* [Future] consider moving deprecated reporting into a message
  validation visitor handler.

Risk level: Low
Testing: Some new //test/common/protobuf::utility_test unit test.

Signed-off-by: Harvey Tuch <[email protected]>

* http: forwarding x-forwarded-proto from trusted proxies (envoyproxy#7995)

Trusting the x-forwarded-proto header from trusted proxies.
If Envoy is operating as an edge proxy but has a trusted hop in front, the trusted proxy should be allowed to set x-forwarded-proto and its x-forwarded-proto should be preserved.
Guarded by envoy.reloadable_features.trusted_forwarded_proto, default on.

Risk Level: Medium (L7 header changes) but guarded
Testing: new unit tests
Docs Changes: n/a
Release Notes: inline
Fixes envoyproxy#4496

Signed-off-by: Alyssa Wilk <[email protected]>

* build: adding an option to hard-fail when deprecated config is used. (envoyproxy#7962)

Adding a build option to default all deprecated protos off, and using it on the debug build.

Risk Level: Low
Testing: new UT
Docs Changes: inline
Release Notes: n/a
Fixes envoyproxy#7548

Signed-off-by: Alyssa Wilk <[email protected]>

* envoy_cc_library: add export of foo_with_external_headers (envoyproxy#8005)

Add a parallel native.cc_library to envoy_cc_library
for external projects that consume Envoy's libraries. This allows the consuming
project to disambiguate overlapping include paths when repository overlaying is used,
as it can now include envoy headers via external/envoy/...

Risk Level: Low
Testing: N/A

Signed-off-by: Otto van der Schaaf <[email protected]>

* ci: add fuzz test targets to ci (envoyproxy#7949)

Builds fuzz targets with asan+libfuzzer and runs them against their corpora. Our native bazel builds work, this PR integrates the asan+libfuzzer builds in to CI. The fuzz target binaries will be in your envoy docker build directory.

Invoke with the following for all fuzz targets, or a specified one.
./ci/run_envoy_docker.sh './ci/do_ci.sh bazel.fuzz'
./ci/run_envoy_docker.sh './ci/do_ci.sh bazel.fuzz //test/common/common:utility_fuzz_test'

Risk level: low
Signed-off-by: Asra Ali [email protected]

Signed-off-by: Asra Ali <[email protected]>

* tls: support BoringSSL private key async functionality (envoyproxy#6326)

This PR adds BoringSSL private key API abstraction, as discussed in envoyproxy#6248. All comments and discussion is welcomed to get the API sufficient for most private key API tasks.

The PR contains the proposed API and the way how it can be used from ssl_socket.h. Also there is some code showing how the PrivateKeyMethodProvider is coming from TLS certificate config. Two example private key method providers are included in the tests.

Description: tls: support BoringSSL private key async functionality
Risk Level: medium
Testing: two basic private key provider implementation
Docs Changes: TLS arch doc, cert.proto doc

Signed-off-by: Ismo Puustinen <[email protected]>

* use SymbolTableCreator rather than fakes in a few stray places. (envoyproxy#8006)

stats: use SymbolTableCreator rather than fakes in a few stray places. (envoyproxy#8006)

Signed-off-by: Joshua Marantz <[email protected]>

* [router] Add SRDS configUpdate impl (envoyproxy#7451)

This PR contains changes on the xRDS side for SRDS impl, cribbed from http://go/gh/stevenzzzz/pull/8/files#diff-2071ab0887162eac1fd177e89d83175a

* Add onConfigUpdate impl for SRDS subscription
* Remove scoped_config_manager as it's not used now.
* Move ScopedConfigInfo to scoped_config_impl.h/cc
* Add a hash to scopeKey and scopeKeyFragment, so we can look up scopekey by hash value in constant time when SRDS has many scopes.
* Add a initManager parameter to RDS createRdsRouteConfigProvider API interface, when creating RouteConfigProvider after listener/server warmed up, we need to specify a different initManager than the one from factoryContext to avoid an assertion failure. see related:envoyproxy#7617

This PR only latches a SRDS provider into the connection manager, the "conn manager using SRDS to make route decision" plus integration tests will be covered in a following PR.

Risk Level: LOW [not fully implemented].
Testing: unit tests

Signed-off-by: Xin Zhuang <[email protected]>

* Fix version history (envoyproxy#8021)

Follow-up for envoyproxy#7995.

Signed-off-by: Raul Gutierrez Segales <[email protected]>

* tools: sync tool for envoyproxy/assignable team. (envoyproxy#8015)

Bulk update of team to match envoyproxy organization. While at it, cleaned up some venv stuff in
shell_utils.sh.

Risk level: Low
Testing: Synced 157 members from envoyproxy to envoyproxy/assignable.

Signed-off-by: Harvey Tuch <[email protected]>

* redis: fix onHostHealthUpdate got called before the cluster is resolved. (envoyproxy#8018)

Signed-off-by: Henry Yang <[email protected]>

* api/build: migrate UDPA proto tree to external cncf/udpa repository. (envoyproxy#8017)

This is a one-time movement of all UDPA content from envoyproxy/envoy to
cncf/udpa. The permanent home of UDPA will be
https://github.com/cncf/udpa.

Risk level: Low
Testing: Added UDPA service entry to build_test.

Signed-off-by: Harvey Tuch <[email protected]>

* http: tracking active session under L7 timers (envoyproxy#7782)

Signed-off-by: Alyssa Wilk <[email protected]>

* upstream: remove thread local cluster after triggering call backs (envoyproxy#8004)

Signed-off-by: Rama Chavali <[email protected]>

* upstream: Introducing close_connections_on_host_set_change property (envoyproxy#7675)

Signed-off-by: Kateryna Nezdolii <[email protected]>

* upstream: delete stale TODO (envoyproxy#8028)

This was fixed in envoyproxy#7877

Signed-off-by: Matt Klein <[email protected]>

* Enhance comment about MonotonicTime (envoyproxy#8011)

Depending on the execution environment in which envoy is being run, it
is possible that some of the assumption on the clock are maybe not
holding as previously commented. With some sandboxing technologies the
clock does not reference the machine boot time but the sandbox boot
time. This invalidates the assumtpion that the first update in the
cluster_manager will most likely fall out of the windows and ends up
showing a non intuitive behavior difficult to catch.
This PR simply adds a comment that will allow the reader to consider
this option while reading to the code.

Signed-off-by: Flavio Crisciani <[email protected]>

* build: some missing dep fixups for Google import. (envoyproxy#8026)

Signed-off-by: Harvey Tuch <[email protected]>

* introduce safe regex matcher based on re2 engine (envoyproxy#7878)

The libstdc++ std::regex implementation is not safe in all cases
for user provided input. This change deprecates the used of std::regex
in all user facing paths and introduces a new safe regex matcher with
an explicitly configurable engine, right now limited to Google's re2
regex engine. This is not a drop in replacement for std::regex as all
language features are not supported. As such we will go through a
deprecation period for the old regex engine.

Fixes envoyproxy#7728

Signed-off-by: Matt Klein <[email protected]>

* docs: reorganize configuration tree (envoyproxy#8027)

This is similar to what I did for the arch overview a while ago as
this section is also getting out of control.

Signed-off-by: Matt Klein <[email protected]>

* build: missing regex include. (envoyproxy#8032)

Signed-off-by: Harvey Tuch <[email protected]>

* [headermap] speedup for appending data (envoyproxy#8029)

For debug builds, performance testing and fuzzers reveal that when appending to a header, we scan both the existing value and the data to append for invalid characters. This PR moves the validation check to just the data that is appended, to avoid hangups on re-scanning long header values multiple times.

Testing: Added corpus entry that reveals time spent in validHeaderString

Signed-off-by: Asra Ali <[email protected]>

* eds: avoid send too many ClusterLoadAssignment requests  (envoyproxy#7976)

During initializing secondary clusters, for each initialized cluster, a ClusterLoadAssignment
request is sent to istio pilot with the cluster's name appended into request's resource_names
list. With a huge number of clusters(e.g 10k clusters), this behavior slows down Envoy's
initialization and consumes ton of memory. This change pauses ADS mux for ClusterLoadAssignment to avoid that.

Risk Level: Medium
Testing: tiny change, no test case added

Fixes envoyproxy#7955

Signed-off-by: lhuang8 <[email protected]>

* Set the bazel verison to 0.28.1 explicitly (envoyproxy#8037)

In theopenlab/openlab-zuul-jobs#622 , the OpenLab add the ability to set the bazel to specific version explicitly. This patch add the bazel role for the envoy job.

Signed-off-by: Yikun Jiang <[email protected]>

* Read_policy is not set correctly. (envoyproxy#8034)

Add more integration test and additional checks in the unit tests.

Signed-off-by: Henry Yang <[email protected]>

* admin: fix /server_info hot restart version (envoyproxy#8022)

Signed-off-by: Matt Klein <[email protected]>

* test: adding debug hints for integration test config failures (envoyproxy#8038)

Risk Level: n/a (test only)
Testing: manual
Docs Changes: n/a
Release Notes: n/a

Signed-off-by: Alyssa Wilk <[email protected]>

* udp_listener: refactor ActiveUdpListener creation (envoyproxy#7884)

Signed-off-by: Dan Zhang <[email protected]>

* accesslog: implement TCP gRPC access logger (envoyproxy#7941)

Description:
Initial implementation for TCP gRPC access logger.

Risk Level: Low (extension only)
Testing: integration test
Docs Changes: Added
Release Notes: Added

Signed-off-by: Lizan Zhou <[email protected]>

* tracing: add OpenCensus agent exporter support to OpenCensus driver. (envoyproxy#8023)

Signed-off-by: Emil Mikulic <[email protected]>

* Exporting platform_impl_lib headers (envoyproxy#8045)

This allows consuming projects using repository overlaying to disambiguate overlapping include paths when it comes to platform_impl.h by going through envoy/external/...

Addendum to envoyproxy#8005

Risk Level: Low
Testing: N/A

Signed-off-by: Otto van der Schaaf <[email protected]>

* access_log: minimal log file error handling (envoyproxy#7938)

Rather than ASSERT for a reasonably common error condition
(e.g. disk full) record a stat that indicates log file writing
failed. Also fixes a test race condition.

Risk Level: low
Testing: added stats checks
Docs Changes: documented new stat
Release Notes: updated

Signed-off-by: Stephan Zuercher <[email protected]>

* tracing: add grpc-status and grpc-message to spans (envoyproxy#7996)

Signed-off-by: Caleb Gilmour <[email protected]>

* fuzz: add bounds to statsh flush interval (envoyproxy#8043)

Add PGV bounds to the stats flush interval (greater than 1ms and less than 5000ms) to prevent Envoy from hanging from too small of a flush time.

Risk Level: Low
Testing: Corpus Entry added
Fixes OSS-Fuzz issue
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=16300

Signed-off-by: Asra Ali <[email protected]>

* Improve tools/stack_decode.py (envoyproxy#8041)

Adjust tools/stack_decode.py to more obviously be Python 2 (not 3), and to work on stack traces that don't include the symbol names.

Risk Level: Low
Testing: Manually tested on a stack trace that one of our users sent us

Signed-off-by: Luke Shumaker <[email protected]>

* build: tell googletest to use absl stacktrace (envoyproxy#8047)

Description:
https://github.com/google/googletest/blob/d7003576dd133856432e2e07340f45926242cc3a/BUILD.bazel#L42

Risk Level: Low (test only)
Testing: CI
Docs Changes:
Release Notes:

Signed-off-by: Lizan Zhou <[email protected]>

* Update references to local scripts to enable using build container for filter repos (envoyproxy#7907)

Description: This change enables using run_envoy_docker.sh to build envoy-filter-example
Risk Level: Low
Testing: Manually tested building envoy-filter-example using: envoy/ci/run_envoy_docker.sh './ci/do_ci.sh build'
Docs Changes: N/A
Release Notes: N/A

Signed-off-by: Santosh Kumar Cheler <[email protected]>

* bazel: patch gRPC to fix Envoy builds with glibc v2.30 (envoyproxy#7971)

Description: the latest glibc (v2.30) declares its own `gettid()` function (see [0]) and this creates a naming conflict in gRPC which has a function with the same name.

Apply to gRPC [a patch](grpc/grpc#18950) which renames `gettid()` to `sys_gettid()`.

[0] https://sourceware.org/git/?p=glibc.git;a=commit;h=1d0fc213824eaa2a8f8c4385daaa698ee8fb7c92

Risk Level: low
Testing: unit tests
Docs Changes: n/a
Release Notes: n/a

Signed-off-by: Dmitry Rozhkov <[email protected]>

* build: link C++ stdlib dynamically in sanitizer runs (envoyproxy#8019)

Description:
Sanitizers doesn't support static link, reverts envoyproxy#7929 and link lib(std)c++ dynamically in sanitizer runs. Addresses test issue for envoyproxy#4251. Added workaround in ASAN for envoyproxy#7647.

Risk Level: Low (test only)
Testing: CI, local libc++ runs
Docs Changes: N/A
Release Notes: N/A
Fixes envoyproxy#7928

* test: cleaning up test runtime (envoyproxy#8012)

Using the new runtime utility to clean up a bunch of test gorp. Yay utils!

Risk Level: n/a (test only)
Testing: tests pass
Docs Changes: n/a
Release Notes: n/a
Signed-off-by: Alyssa Wilk <[email protected]>

*  test: improved coverage and handling of deprecated config (envoyproxy#8057)

Making ENVOY_DISABLE_DEPRECATED_FEATURES work for unit tests without runtime configured.
Fixing up a handful of unit tests to remove legacy code or use the handy
DEPRECATED_FEATURE_TEST macro
Adding back coverage of cors.enabled() and redis.catch_all_route()

Risk Level: Low (test only)
Testing: new unit tests
Docs Changes: n/a
Release Notes: n/a
Fixes envoyproxy#8013
Fixes envoyproxy#7548

Signed-off-by: Alyssa Wilk <[email protected]>

* [Docs typo] Remote Executioon -> Remote Execution (envoyproxy#8061)

Fixes mispelling of `Executioon` -> `Execution`

Signed-off-by: Colin Schoen <[email protected]>

* api: Fix duplicate java_outer_classname declarations (envoyproxy#8059)

The java_outer_classname is unintentionally duplicated in the new
udp_listener_config and regex proto files. This changes them to unique
names that match the predominant naming scheme.

Signed-off-by: Bryce Anderson <[email protected]>

* http: making the behavior of the response Server header configurable (envoyproxy#8014)

Default behavior remains unchanged, but now Envoy can override, override iff there's no server header from upstream, or always leave the server header (or lack thereof) unmodified.

Risk Level: low (config guarded change)
Testing: new unit tests
Docs Changes: n/a
Release Notes: inline
Fixes envoyproxy#6716

Signed-off-by: Alyssa Wilk <[email protected]>

* use bazelversion for filter-example too (envoyproxy#8069)

Signed-off-by: Lizan Zhou <[email protected]>

* grpc-httpjson-transcode: Update for RFC2045 support (envoyproxy#8065)

RFC2045 (MIME) Base64 decoding support has been fixed upstream

Description: The grpc transcoding filter has been updated to support RFC2045 (MIME) based inputs for protobuf type "Bytes". This is important since Base64 is often using the RFC2045 format for inputs.
Also see: grpc-ecosystem/grpc-httpjson-transcoding#34

Risk Level: Low
Testing: Integration / Manual Tests
Docs Changes: N/A
Release Notes: N/A

Signed-off-by: Hans Viken Duedal <[email protected]>

* stats: Clean up all calls to Scope::counter() et al in production code. (envoyproxy#7842)

* Convert a few more counter() references to use the StatName interface.

Signed-off-by: Joshua Marantz <[email protected]>

* tls_inspector: inline the recv in the onAccept (envoyproxy#7951)

Description:
As discussed in envoyproxy#7864 this PR is the attempt to peek the socket at the invoke of onAccept.
Usually client_hello packet should be in the buffer when tls_inspector is peeking, we could save a poll cycle for this connection.

Once we agree on the solution I can apply to http_inspector as well.

The expecting latency improvement especially when poll cycle is large.

Benchmark:
Env:
hardware Intel(R) Xeon(R) CPU @ 2.20GHz
envoy: concurrency = 1, tls_inspector as listener filter. One tls filter chain, and one plain text filter chain.
load background: a [sniper](https://github.com/lubia/sniper) client with concurrency = 5 hitting the server with tls handshake, aiming to hit using the tls_filter chain. The qps is about 170/s
Another load client hitting the plain text filter chain but would go through tls_inspector with concurrency = 1

This PR: 
TransactionTime:              10.3 - 11.0 ms(mean)
Master                
TransactionTime:              12.3 - 12.8 ms(mean)

Risk Level: Med (ActiveSocket code is affected to adopt the side effect of onAccept)
Testing: 
Docs Changes:
Release Notes:
Fixes envoyproxy#7864

Signed-off-by: Yuchen Dai <[email protected]>

* Fixes gcc 8.3.1 build failure due to FilterChainBenchmarkFixture::SetUp hiding base-class virtual functions (envoyproxy#8071)

Description: I'm seeing "bazel-out/k8-fastbuild/bin/external/com_github_google_benchmark/_virtual_includes/benchmark/benchmark/benchmark.h:1071:16: error: 'virtual void benchmark::Fixture::SetUp(benchmark::State&)' was hidden" when running tests. This resolves the issue with hiding of the base-class functions.
Risk Level: low
Testing:
Docs Changes:
Release Notes:

Signed-off-by: Dmitri Dolguikh <[email protected]>

* test: fix ups for various deprecated fields (envoyproxy#8068)

Takeaways: we've lost the ability to do empty regex (which was covered in router tests and is proto constraint validated on the new safe regex) as well as negative lookahead (also covered in tests) along with a host of other things conveniently documented as not supported here: https://github.com/google/re2/wiki/Syntax

Otherwise split up a bunch of tests, duplicated and tagged a bunch of tests, and cleaning up after we finally can remove deprecated fields again will be an order of magnitude easier.

Also fixing a dup relnote from envoyproxy#8014

Risk Level: n/a (test only)
Testing: yes. yes there is.
Docs Changes: no
Release Notes: no

Signed-off-by: Alyssa Wilk <[email protected]>

* include: add log dependency header to connection_handler.h (envoyproxy#8072)

Signed-off-by: Teju Nareddy <[email protected]>

* quiche: Update QUICHE dep (envoyproxy#8044)

Update QUICHE tar ball to 4abb566fbbc63df8fe7c1ac30b21632b9eb18d0c.
Add some new impl's for newly added api.

Risk Level: low
Testing: using quiche build in tests.
Part of envoyproxy#2557

Signed-off-by: Dan Zhang <[email protected]>

* tools: deprecated field check in Route Checker tool (envoyproxy#8058)

We need a way to run the deprecated field check on the RouteConfiguration. Today the schema check tool validates the bootstrap config. This change will help achieve similar functionality on routes served from rds.
Risk Level: Low
Testing: Manual testing
Docs Changes: included
Release Notes: included

Signed-off-by: Jyoti Mahapatra <[email protected]>

* tracing: Add support for sending data in Zipkin v2 format (envoyproxy#6985)

Description: This patch supports sending a list of spans as JSON v2 and protobuf message over HTTP to Zipkin collector. [Sending protobuf](https://github.com/openzipkin/zipkin-api/blob/0.2.1/zipkin.proto) is considered to be more efficient than JSON, even compared to the v2's JSON (openzipkin/zipkin#2589 (comment)). This is an effort to rework envoyproxy#6798.

The approach is by serializing the v1 model to both v2 JSON and protobuf.

Risk Level: Low, since the default is still HTTP-JSON v1 based on https://github.com/openzipkin/zipkin-api/blob/0.2.2/zipkin-api.yaml.
Testing: Unit testing, manual integration test with real Zipkin collector server.
Docs Changes: Added
Release Notes: Added
Fixes: envoyproxy#4839

Signed-off-by: Dhi Aurrahman <[email protected]>
Signed-off-by: José Carlos Chávez <[email protected]>

* Route Checker tool Fix code coverage bug in proto based schema (envoyproxy#8101)

Signed-off-by: Jyoti Mahapatra <[email protected]>

* [hcm] Add scoped RDS routing into HCM (envoyproxy#7762)

Description: add Scoped RDS routing logic into HCM. Changes include:

* in ActiveStream constructor latch a ScopedConfig impl to the activeStream if SRDS is enabled
* in the beginning of ActiveStream::decodeHeaders(headers, end_stream), get routeConfig from latched ScopedConfig impl.

This PR is the 3rd in the srds impl PR chain: [envoyproxy#7704, envoyproxy#7451, this].

Risk Level: Medium
Testing: unit test and integration tests.
Release Notes: Add scoped RDS routing support into HCM.

Signed-off-by: Xin Zhuang <[email protected]>

* owners: add @asraa and @lambdai to OWNERS. (envoyproxy#8110)

* @asraa is joining Envoy OSS security team.

* @lambdai is joining Friends of Envoy as v2 xDS point.

Signed-off-by: Harvey Tuch <[email protected]>

* protobuf: recursively validate unknown fields. (envoyproxy#8094)

This PR unifies the recursive traversal of deprecated fields with that of unknown fields. It doesn't
deal with moving to a validator visitor model for deprecation; this would be a nice cleanup that we
track at envoyproxy#8092.

Risk level: Low
Testing: New nested unknown field test added.

Fixes envoyproxy#7980

Signed-off-by: Harvey Tuch <[email protected]>

* Fuzz reuse (envoyproxy#8119)

This PR allows the envoy_cc_fuzz_test rule to be used when pulling in envoy. which can be useful when you're writing filters for envoy, and want to reuse the fuzzing architecture envoy has already built. other rules already allow for this (see envoy_cc_test in this same file for example).

Risk Level: Low
Testing:

Testing the Old Rule Still Works

It is possible to test the old rules still work (even without specifying a repository), by simply choosing your favorite fuzz test, and choosing to run bazel test on it. For example: bazel test //test/common/router:header_parser_fuzz_test. Any envoy_cc_fuzz_test rule should do.

Testing New Rules Work

I've done testing inside my own repository, but if you want to create your own test rule you can probably do the following in envoy-filter-example:

Checkout envoy-filter-example, and update the envoy submodule to this pr.
Follow the directions in: test/fuzz/README.md to define a envoy_cc_fuzz_test rule. Make sure to add a line for: repository = "@envoy" which is the new argument being added.
You should be able to run the fuzz test.

Signed-off-by: Cynthia Coan <[email protected]>

* Set INCLUDE_DIRECTORIES so libcurl can find local urlapi.h (envoyproxy#8113)

Fixes envoyproxy#8112

Signed-off-by: John Millikin <[email protected]>

* cleanup: move test utility methods in ScopedRdsIntegrationTest to base class HttpIntegrationTest (envoyproxy#8108)

Fixes envoyproxy#8050
Risk Level: LOW [refactor only]

Signed-off-by: Xin Zhuang <[email protected]>

* upstream: fix invalid access of ClusterMap iterator during warming cluster modification (envoyproxy#8106)

Risk Level: Medium
Testing: New unit test added. Fix verified via --config=asan.

Signed-off-by: Andres Guedez <[email protected]>

* api:Add a flag to disable overprovisioning in ClusterLoadAssignment (envoyproxy#8080)

* api:Add a flag to disable overprovisioning in ClusterLoadAssignment

Signed-off-by: Jie Chen <[email protected]>

* api:Add [#next-major-version and [#not-implemented-hide to the comment
for field of disable_overprovisioning in ClusterLoadAssignment
Signed-off-by: Jie Chen <[email protected]>

* api:Refine comments for the new added bool flag as suggested.
Signed-off-by: Jie Chen <[email protected]>

* api: clone v2[alpha] to v3alpha. (envoyproxy#8125)

This patch establishes a v3alpha baseline API, by doing a simple copy of
v2[alpha] dirs and some sed-style heuristic fixups of BUILD dependencies
and proto package namespaces.

The objective is provide a baseline which we can compare the output from
tooling described in envoyproxy#8083 in later PRs, providing smaller visual diffs.

The core philosophy of the API migration is that every step will be
captured in a script (at least until the last manual steps),
api/migration/v3alpha.sh. This script will capture deterministic
migration steps, allowing v2[alpha] to continue to be updated until we
finalize v3.

There is likely to be significant changes, e.g. in addition to the work
scoped for v3, we might want to reduce the amount of API churn by
referring back to v2 protos where it makes sense. This will be done via
tooling in later PRs.

Part of envoyproxy#8083.

Risk level: Low
Testing: build @envoy_api//...

Signed-off-by: Harvey Tuch <[email protected]>

* dubbo: Fix heartbeat packet parsing error (envoyproxy#8103)

Description: 
The heartbeat packet may carry data, and it is treated as null data when processing the heartbeat packet, causing some data to remain in the buffer.

Risk Level: low 
Testing: Existing unit test
Docs Changes: N/A
Release Notes: N/A
Fixes envoyproxy#7970 

Signed-off-by: tianqian.zyf <[email protected]>

* stats: Shared cluster isolated stats (envoyproxy#8118)

* shared the main symbol-table with the isolated stats used for cluster info.

Signed-off-by: Joshua Marantz <[email protected]>

* protodoc: upgrade to Python 3. (envoyproxy#8129)

Risk level: Low
Testing: Rebuilt docs, manual inspection of some example generated files.

Signed-off-by: Harvey Tuch <[email protected]>

* protodoc: single source-of-truth for doc protos. (envoyproxy#8132)

This avoids having to list new docs protos in both docs/build.sh and
api/docs/BUILD. This technical debt cleanup is helpful in v3 proto work
to simplify collecting proto artifacts from a Bazel aspect.

Risk level: Low
Testing: docs/build.sh, visual inspection of docs.

Signed-off-by: Harvey Tuch <[email protected]>

* api: organize go_proto_libraries (envoyproxy#8003)

Fixes envoyproxy#7982

Defines a package level proto library and its associated internal go_proto_library.

Deletes all existing api_go_proto_library, api_go_grpc_library, and go_package annotations in protos (they are not required and pollute the sources).

I deliberately avoided touching anything under udpa since it's being moved to another repository.

Risk Level: low
Testing: build completes

Signed-off-by: Kuat Yessenov <[email protected]>

* api: straggler v2alpha1 -> v3alpha clone. (envoyproxy#8133)

These were missed in envoyproxy#8125.

Signed-off-by: Harvey Tuch <[email protected]>

* docs: remove extraneous escape (envoyproxy#8150)

Old versions of bash (e.g. on macOS) don't handle ${P/:/\/} the same way as modern versions. In particular, the expanded parameter on macOS includes a backslash, causing subsequent use of the string as a filename to include a slash (/) instead of treating the slash as a directory separator. Both versions of bash accept ${P/://} as a way to substitute : with /. Verified that this change does not alter the generated docs when running under Linux.

Risk Level: low
Testing: generated docs under linux & macOS

Signed-off-by: Stephan Zuercher <[email protected]>

* Do not 503 on Upgrade: h2c instead remove the header and ignore. (envoyproxy#7981)

Description: When a request comes in on http1 with "upgrade: h2c", the current behavior is to 503.  Instead we should ignore the upgrade and remove the header and continue with the request as http1.
Risk Level: Medium
Testing: Unit test. Hand test with ithub.com/rdsubhas/java-istio client server locally.
Docs Changes: N/A
Release Notes:  http1: ignore and remove Upgrade: h2c.
Fixes istio/istio#16391

Signed-off-by: John Plevyak <[email protected]>

* docs: add line on installing xcode for macOS build flow (envoyproxy#8139)

Because of rules_foreign_cc in bazelbuild, Envoy will not compile successfully when following the instructions in the build docs due to how the tools are referenced. One fix for this is installing Xcode from the App Store (see bazel-contrib/rules_foreign_cc#185).

Risk Level: Low
Testing: N/A (docs change)
Docs Changes: see Description
Release Notes: N/A

Signed-off-by: Lisa Lu <[email protected]>

* docs: note which header expressions cannot be used for request headers (envoyproxy#8138)

As discussed in envoyproxy#8127, some custom header expressions evaluate as
empty when used in request headers.

Risk Level: low, docs only
Testing: n/a
Docs Changes: updated
Release Notes: n/a

Signed-off-by: Stephan Zuercher <[email protected]>

* api: use traffic_direction over operation_name if specified (envoyproxy#7999)

Use the listener-level field for the tracing direction over the per-filter field. Unfortunately, the per filter did not provide an "unspecified" default, so this appears to be the right approach to deprecate the per-filter field with minimal impact.

Risk Level: low (uses a newly introduce field traffic_direction)
Testing: unit test
Docs Changes: proto docs

Signed-off-by: Kuat Yessenov <[email protected]>

* add more diagnostic logs (envoyproxy#8153)

Istio sets listener filter timeout to 10ms by default but requests fail from time to tome. It's very difficult to debug. Even though downstream_pre_cx_timeout_ is exposed to track the number of timeouts, it would be better to have some debug logs.

Description: add more diagnostic logs
Risk Level: low

Signed-off-by: crazyxy <[email protected]>

* http conn man: add tracing config for path length in tag (envoyproxy#8095)

This PR adds a configuration option for controlling the length of the request path that is included in the HttpUrl span tag. Currently, this length is hard-coded to 256. With this PR, that length will be configurable (defaulting to the old value).

Risk Level: Low
Testing: Unit
Docs Changes: Inline with the API proto. Documented new field.
Release Notes: Added in the PR.

Related issue: istio/istio#14563

Signed-off-by: Douglas Reid <[email protected]>

* cds: Add general-purpose LB policy configuration (envoyproxy#7744)

This PR adds fields to CDS that allow for general-purpose LB policy configuration.

Risk Level: Low
Testing: None (but if anything is needed, please let me know)
Docs Changes: Inline with API protos
Release Notes: N/A

Signed-off-by: Mark D. Roth <[email protected]>

* thrift_proxy: fix crash on invalid transport/protocol (envoyproxy#8143)

Transport/protocol decoder errors that occur before the connection manager
initializes an ActiveRPC to track the request caused a crash. Modifies the
connection manager to handle this case, terminating the downstream the
connection.

Risk Level: low
Testing: test case that triggers crash
Docs Changes: n/a
Release Notes: added

Signed-off-by: Stephan Zuercher <[email protected]>

* api: strip gogoproto annotations (envoyproxy#8163)

Remove gogoproto annotations. They can be replaced with a custom gogoproto compiler (e.g. something like https://github.com/gogo/googleapis/tree/master/protoc-gen-gogogoogleapis). I have an experimental version of it to validate that it's possible to re-apply important annotations in the compiler.

Risk Level: low
Testing: builds

Signed-off-by: Kuat Yessenov <[email protected]>

* hotrestart: remove dynamic_resources from server config used by hotrestart_test (envoyproxy#8162)

In the server config file `test/config/integration/server.yaml` used by
//test/integration:hotrestart_test, `dynamic_resources` includes `lds_config`
and `cds_config` definitions, which use HTTP API to fetch config, but CDS and
LDS service do not exist, so the initial fetch will be failed with a
connection failure, then Envoy server continue startup.

Envoy server shouldn't continue startup because connection failure, see
issue envoyproxy#8046.

For this test, `dynamic_resources` is not needed, this change clean it up.

Signed-off-by: lhuang8 <[email protected]>

* clang-tidy: misc-unused-using-decls (envoyproxy#8159)

Description: clang-tidy check to flag unused using statements. There's a lot in test code that's just copy pasta, and it's hard to manually review whether it's being used, especially for things like using testing::_;
Risk Level: low
Testing: existing
Docs Changes: N/A
Release Notes: N/A

Signed-off-by: Derek Argueta <[email protected]>

* build: curl with c-ares, nghttp2 and zlib (envoyproxy#8154)

Build curl dependency with async DNS resolver c-ares avoiding potential
crashes due to longjmp on modern kernels. Add zlib and nghttp2.
Use Envoy's version of all of the above libraries.

Signed-off-by: Taras Roshko <[email protected]>

* log: add upstream TLS info (envoyproxy#7911)

Description: add upstream TLS info for logging purposes

Refactor SSL connection info to be a shared pointer.
Use read-only interface.
Cache computed values in the SSL info object (this allows transition to remove the underlying SSL object if necessary).

Risk Level: medium due to use of bssl::SSL to back ConnectionInfo
Testing: unit
Docs Changes: none
Release Notes: add upstream TLS info

Signed-off-by: Kuat Yessenov <[email protected]>

* fix windows implementation of PlatformImpl (envoyproxy#8169)

Add missing destructor to class declaration.
Fix copy/paste errors.
These errors were apparently introduced in e1cd4cc.

Risk Level: Low
Testing: Passed Windows testing locally
Docs Changes: n/a
Release Notes: n/a

Signed-off-by: William Rowe [email protected]
Signed-off-by: Yechiel Kalmenson <[email protected]>

* Update Opencensus SHA (envoyproxy#8173)

Signed-off-by: Pengyuan Bian <[email protected]>

* Outlier Detection: use gRPC status code for detecting failures (envoyproxy#7942)

Signed-off-by: ZhouyihaiDing <[email protected]>

* fix build (envoyproxy#8177)

Signed-off-by: Derek Argueta <[email protected]>

* docs: improving websocket docs (envoyproxy#8156)

Making it clear H2 websockets don't work by default

Risk Level: n/a
Testing: n/a
Docs Changes: yes
Release Notes: no
envoyproxy#8147

Signed-off-by: Alyssa Wilk <[email protected]>

* Upstream WebAssembly VM and Null VM from envoyproxy/envoy-wasm. (envoyproxy#8020)

Description: Upstream from envoyproxy/envoy-wasm the WebAssembly VM support along with the Null VM support and tests. This is the first PR dealing with WebAssembly filter support in envoy.  See https://github.com/envoyproxy/envoy-wasm/blob/master/WASM.md and https://github.com/envoyproxy/envoy-wasm/blob/master/docs/root/api-v2/config/wasm/wasm.rst for details.
Risk Level: Medium
Testing: Unit tests.
Docs Changes: N/A
Release Notes: N/A
Part of envoyproxy#4272 

Signed-off-by: John Plevyak <[email protected]>

* quiche: implement Envoy Quic stream and connection (envoyproxy#7721)

Implement QuicStream|Session|Disptacher in Envoy. Weir up QUIC stream and connection with HCM callbacks.

Risk Level: low, not in use
Testing: Added unit tests for all new classes
Part of envoyproxy#2557
Signed-off-by: Dan Zhang <[email protected]>

* protodoc/api_proto_plugin: generic API protoc plugin framework. (envoyproxy#8157)

Split out the generic plugin and FileDescriptorProto traversal bits from
protodoc. This is in aid of the work in envoyproxy#8082 ad envoyproxy#8083, where additional
protoc plugins will be responsible for v2 -> v3alpha API migrations and
translation code generation.

This is only the start really of the api_proto_plugin framework. I
anticipate additional bits of protodoc will move here later, including
field type analysis and oneof handling.

In some respects, this is a re-implementation of some of
https://github.com/lyft/protoc-gen-star in Python. The advantage is that
this is super lightweight, has few dependencies and can be easily
hacked. We also embed various bits of API business logic, e.g.
annotations, in the framework (for now).

Risk level: Low
Testing: diff -ru against previous protodoc.py RST output, identical modulo some
  trivial whitespace that doesn't appear in generated HTML. There are no
  real tests yet, I anticipate adding some golden proto style tests.

Signed-off-by: Harvey Tuch <[email protected]>

* adaptive concurrency: Gradient algorithm implementation (envoyproxy#7908)

Signed-off-by: Tony Allen <[email protected]>

* ext_authz: Check for cluster before sending HTTP request (envoyproxy#8144)

Signed-off-by: Dhi Aurrahman <[email protected]>

* make getters const-ref (envoyproxy#8192)

Description:
Follow-up to envoyproxy#7911 to make cached values be exposed as const-references, saving on a copy of a string during retrieval.

Risk Level: low
Testing: updated mocks to return references
Docs Changes: none
Release Notes: none

Signed-off-by: Kuat Yessenov <[email protected]>

* test: add curl features check (envoyproxy#8194)

Add a test ensuring curl was built with the expected features.

Description: Add a test ensuring curl was built with the expected features.
Risk Level: Low.
Testing: n/a.
Docs Changes: n/a.
Release Notes: n/a.

Signed-off-by: Taras Roshko <[email protected]>

* subset lb: allow ring hash/maglev LB to work with subsets (envoyproxy#8030)

* subset lb: allow ring hash/maglev LB to work with subsets

Skip initializing the thread aware LB for a cluster when the subset
load balancer is enabled. Also adds some extra checks for LB policies
that are incompatible with the subset load balancer.

Risk Level: low
Testing: test additional checks
Docs Changes: updated docs w.r.t subset lb compatibility
Release Notes: n/a
Fixes: envoyproxy#7651

Signed-off-by: Stephan Zuercher <[email protected]>

* redis: add a request time metric to redis upstream (envoyproxy#7890)

Signed-off-by: Nicolas Flacco <[email protected]>

* bazel: update bazel to 0.29.1 (envoyproxy#8198)

Description:
Upgrade bazel to 0.29.1 and bazel-toolchains to corresponding version.

Risk Level: Low
Testing: CI
Docs Changes: N/A
Release Notes: N/A

Signed-off-by: Lizan Zhou <[email protected]>

* upstream: Add ability to disable host selection during panic (envoyproxy#8024)

Previously, when in a panic state, requests would be routed to all
hosts. In some cases it is instead preferable to not route any requests.
Add a configuration option for zone-aware load balancers which switches
from routing to all hosts to no hosts.

Closes envoyproxy#7550.

Signed-off-by: James Forcier [email protected]

Risk Level: Low
Testing: 2 new unit tests written; manual testing
Docs Changes: Note about new configuration option added
Release Notes: added

Signed-off-by: James Forcier <[email protected]>

* metrics service: flush histogram buckets (envoyproxy#8180)

Signed-off-by: Rama Chavali <[email protected]>

* tracing: fix random sample fraction percent (envoyproxy#8205)

Signed-off-by: Pengyuan Bian <[email protected]>

* stats: Add per-host memory usage test case to stats_integration_test (envoyproxy#8189)

Signed-off-by: Antonio Vicente <[email protected]>

* router check tool: add flag for only printing failed tests (envoyproxy#8160)

Signed-off-by: Lisa Lu <[email protected]>

* fix link to runtime docs (envoyproxy#8204)

Description: Looks like the runtime docs moved under operations/. The PR fixes the link.
Risk Level: low
Testing: existing
Docs Changes: this
Release Notes: n/a

Signed-off-by: Derek Argueta <[email protected]>

* config: make SlotImpl detachable from its owner, and add a new runOnAllThreads interface to Slot. (envoyproxy#8135)

See the issue in envoyproxy#7902, this PR is to make the SlotImpl detachable from its owner, by introducing a Booker object wraps around a SlotImpl, which bookkeeps all the on-the-fly update callbacks. And on its destruction, if there are still on-the-fly callbacks, move the SlotImpl to an deferred-delete queue, instead of destructing the SlotImpl which may cause an SEGV error.

More importantly, introduce a new runOnAllThreads(ThreadLocal::UpdateCb cb) API to Slot, which requests a Slot Owner to not assume that the Slot or its owner will out-live (in Main thread) the fired on-the-fly update callbacks, and should not capture the Slot or its owner in the update_cb.

Picked RDS and config-providers-framework as examples to demonstrate that this change works. {i.e., changed from the runOnAllThreads(Event::PostCb) to the new runOnAllThreads(TLS::UpdateCb) interface. }

Risk Level: Medium
Testing: unit test
Docs Changes: N/A
Release Notes: N/A
[Optional Fixes #Issue] envoyproxy#7902

Signed-off-by: Xin Zhuang <[email protected]>

* test: remove static config from subset lb integration test (envoyproxy#8203)

Build the config programmatically to make future API changes less
onerous.

Risk Level: low (test change only)
Testing: n/a
Doc Changes: n/a
Release Notes: n/a

Signed-off-by: Stephan Zuercher <[email protected]>

* cleanup: clarify Cluster.filters and Dispatcher::createClientConnection (envoyproxy#8186)

Signed-off-by: Fred Douglas <[email protected]>

* redis: health check is not sending the auth command on its connection (envoyproxy#8166)

Signed-off-by: Henry Yang <[email protected]>

* redis: mirroring should work when default value is zero, not just greater than zero (envoyproxy#8089)

Signed-off-by: Nicolas Flacco <[email protected]>

* tools: regularize pip/venv for format_python_tools.py. (envoyproxy#8176)

As well as being a nice cleanup, this fixes some issues I had with local
Docker use of fix_format as a non-root user.

Signed-off-by: Harvey Tuch <[email protected]>

* absl: Absl hash hook in a couple of places rather than hash functors (envoyproxy#8179)

Signed-off-by: Joshua Marantz <[email protected]>

* Update dependency: jwt_verify_lib (envoyproxy#8212)

Signed-off-by: Daniel Grimm <[email protected]>

* upstream: add failure percentage-based outlier detection (envoyproxy#8130)

Description: Add a new outlier detection mode which compares each host's rate of request failure to a configured fixed threshold.

Risk Level: Low
Testing: 2 new unit tests added.
Docs Changes: New mode and config options described.
Release Notes: white_check_mark
Fixes envoyproxy#8105

Signed-off-by: James Forcier <[email protected]>

* Replace deprecated thread annotations macros. (envoyproxy#8237)

Abseil thread annotation macros are now prefixed by ABSL_.

There is no semantic change; this is just a rename.

Signed-off-by: Yan Avlasov <[email protected]>

* Update protoc-gen-validate (PGV) (envoyproxy#8234)

This picks up fixes for the Windows build and a C preprocessor defect

Signed-off-by: Yechiel Kalmenson <[email protected]>
Signed-off-by: William Rowe <[email protected]>

* upstream: use named constants for outlier detection config defaults (envoyproxy#8221)

Signed-off-by: James Forcier <[email protected]>

* server: add a post init lifecycle stage (envoyproxy#8217)

Signed-off-by: Jose Nino <[email protected]>

* docs: document access control conditions and attributes (envoyproxy#8230)

Signed-off-by: Kuat Yessenov <[email protected]>

* server: return processContext as optional reference (envoyproxy#8238)

Signed-off-by: Elisha Ziskind <[email protected]>

* Update envoy.yaml in Redis proxy example (envoyproxy#8220)

Description: Make Redis example use catch_all_route.
Risk Level: Low.
Testing: Done. docker-compose up --build brings up envoy proxy and I was able to run Redis commands using redis-cli.

Signed-off-by: Raju Kadam <[email protected]>

* quiche: implement ActiveQuicListener (envoyproxy#7896)

Signed-off-by: Dan Zhang <[email protected]>

* srds: allow SRDS pass on scope-not-found queries to filter-chain (issue envoyproxy#8236).  (envoyproxy#8239)

Description: Allow a no-scope request to pass through the filter chain, so that some special queries (e.g., data plane health-check ) can be processed by the customized filter-chain. By default, the behavior is the same (404).
Risk Level: LOW
Testing: unit test and integration test.
Docs Changes: N/A
Release Notes: N/A
Fixes envoyproxy#8236
Signed-off-by: Xin Zhuang <[email protected]>

* Updated to new envoyproxy master branch.

Signed-off-by: John Plevyak <[email protected]>

* Remove offending go proto option.

Signed-off-by: John Plevyak <[email protected]>

* Fix format/tidy issues.

Signed-off-by: John Plevyak <[email protected]>
@incfly incfly mentioned this pull request Jan 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants