Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: unhiding HTTP/3 and adding docs #15926

Merged
merged 27 commits into from
May 7, 2021
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions api/envoy/config/core/v3/protocol.proto
Original file line number Diff line number Diff line change
Expand Up @@ -416,8 +416,6 @@ message GrpcProtocolOptions {
Http2ProtocolOptions http2_protocol_options = 1;
}

// [#not-implemented-hide:]
phlax marked this conversation as resolved.
Show resolved Hide resolved
//
// A message which allows using HTTP/3.
message Http3ProtocolOptions {
QuicProtocolOptions quic_protocol_options = 1;
Expand Down
2 changes: 0 additions & 2 deletions api/envoy/config/core/v4alpha/protocol.proto

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 12 additions & 2 deletions api/envoy/config/listener/v3/udp_listener_config.proto
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,18 @@ message UdpListenerConfig {

// Configuration for QUIC protocol. If empty, QUIC will not be enabled on this listener. Set
// to the default object to enable QUIC without modifying any additional options.
// [#not-implemented-hide:]
// [#comment:Unhide when QUIC alpha is announced with other docs.]
//
// .. note::
// QUIC support is currently alpha and should be used with caution.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im wondering if this note should be a warning - and also thinking it might be better placed in docs/root/intro/arch_overview/http/http3.rst as its linked to from various places - feels a bit arbitrary to add it to one proto

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I lean towards the API because I think folks will actually read it here, where I'm less confident of folks reading the docs. Happy to make it a warning through.

Copy link
Member

@phlax phlax May 6, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

im wondering if this should make use of the experimental annotation that has just been added

agreed that there needs to be a warning here - but i would put the fuller explanation in the ref docs and point to that - as is done in most of the the api protos - mho

//
// For known outstanding issues before Envoy QUIC support is GA, you can
// track https://github.com/envoyproxy/envoy/labels/quic-mvp for example
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that "For example" is the start of a new sentence? If so, perhaps a period and then Capitalize "for". If it's not the start of a new sentence, perhaps it can be rephrased for clarity?

// QUIC does not currently support in place filter chain updates, so users
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I feel like 'in-place' is how I am accustomed to seeing this phrase in the context of software. (As opposed to, say, "having everything in place"). Maybe? WDYT?

// requiring dynamic config reload for QUIC should wait until
// https://github.com/envoyproxy/envoy/issues/13115 has been addressed.
//
// For general feature requets beyond production readiness, you can track
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "requets" => "requests"

// https://github.com/envoyproxy/envoy/labels/area%2Fquic
QuicProtocolOptions quic_options = 7;
}

Expand Down
14 changes: 12 additions & 2 deletions api/envoy/config/listener/v4alpha/udp_listener_config.proto

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

30 changes: 23 additions & 7 deletions api/envoy/extensions/upstreams/http/v3/http_protocol_options.proto
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ option (udpa.annotations.file_status).package_version_status = ACTIVE;
// [#next-free-field: 6]
message HttpProtocolOptions {
// If this is used, the cluster will only operate on one of the possible upstream protocols.
// Note that HTTP/2 should generally be used for upstream clusters doing gRPC.
// Note that HTTP/2 or above should generally be used for upstream clusters doing gRPC.
phlax marked this conversation as resolved.
Show resolved Hide resolved
message ExplicitHttpConfig {
oneof protocol_config {
option (validate.required) = true;
Expand All @@ -68,7 +68,11 @@ message HttpProtocolOptions {

config.core.v3.Http2ProtocolOptions http2_protocol_options = 2;

// [#not-implemented-hide:]
// .. note::
// QUIC support is currently alpha and should be used with caution.
//
// See comments :ref:`here <envoy_v3_api_field_config.listener.v3.UdpListenerConfig.quic_options>`
// for details.
config.core.v3.Http3ProtocolOptions http3_protocol_options = 3;
}
}
Expand All @@ -80,7 +84,11 @@ message HttpProtocolOptions {

config.core.v3.Http2ProtocolOptions http2_protocol_options = 2;

// [#not-implemented-hide:]
// .. note::
// QUIC support is currently alpha and should be used with caution.
//
// See comments :ref:`here <envoy_v3_api_field_config.listener.v3.UdpListenerConfig.quic_options>`
// for details.
config.core.v3.Http3ProtocolOptions http3_protocol_options = 3;
}

Expand All @@ -98,11 +106,19 @@ message HttpProtocolOptions {

config.core.v3.Http2ProtocolOptions http2_protocol_options = 2;

// [#not-implemented-hide:]
// Unlike HTTP/1 and HTTP/2, HTTP/3 will not be configured unless it is
// present. If HTTP/3 is present, attempts to connect will first be made
// via HTTP/3, and if the HTTP/3 connection fails, TCP (HTTP/1 or HTTP/2
// based on ALPN) will be used instead.
// present, and will eventually rely on server-side alt-svc advertisement to try HTTP/3.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For Alt-Svc, I'd be inclined to say "HTTP Alternative Services" and I might also reference the spec https://tools.ietf.org/html/rfc7838

Is it worth mention the HTTPS DNS RR here as well, since we intend to implement this? (and the spec https://datatracker.ietf.org/doc/html/draft-ietf-dnsop-svcb-https-04)

// If HTTP/3 is configured via HTTP/3, and if the HTTP/3 connection fails (currently controlled by a
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I wonder if we can improve on the phrase "If HTTP/3 is configured via HTTP/3". How bout something like:

When HTTP/3 is configured, HTTP/3 connections will be attempted. However, the connection may fail over to TCP (HTTP/1 or HTTP/2 based on ALPN) instead. If the HTTP/3 connection fails explicitly because of a protocol error, a TCP connection will be attempted instead. In addition, if the HTTP/3 connection has not succeeded after a (currently) fixed timeout, a TCP connection will be started in a parallel. The connection the succeeds first will be used.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, having written all that... I think that's only true when H1, H2 and H3 are all enabled. If only H3 is enabled then there's no fallback and not alt-svc checking, right? Should we mention this? (Wow this gets complex in a hurry!)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, for auto, remember that H2 and H1 are on by default, so you can't not configure it.

// hard-coded timeout) TCP (HTTP/1 or HTTP/2 based on ALPN) will be
// attempted as well.
// See :ref:`here <arch_overview_http3_upstream>` for more information.
//
// .. note::
// QUIC support is currently alpha and should be used with caution, and
// the auto config is undergoing especially rapid change.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for my curiosity, what is "auto config"?

//
// See comments :ref:`here <envoy_v3_api_field_config.listener.v3.UdpListenerConfig.quic_options>`
// for details.
config.core.v3.Http3ProtocolOptions http3_protocol_options = 3;
}

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

50 changes: 50 additions & 0 deletions configs/envoyproxy_io_proxy_http3_downstream.template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,56 @@ admin:
address: 0.0.0.0
port_value: 9901
static_resources:
listeners:
phlax marked this conversation as resolved.
Show resolved Hide resolved
- name: listener_tcp
address:
socket_address:
protocol: TCP
address: 0.0.0.0
port_value: 10000
reuse_port: true
udp_listener_config:
quic_options: {}
downstream_socket_config:
prefer_gro: true
alyssawilk marked this conversation as resolved.
Show resolved Hide resolved
filter_chains:
transport_socket:
name: envoy.transport_sockets.tls
typed_config:
"@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
downstream_tls_context:
common_tls_context:
alpn_protocols: h3
tls_certificates:
certificate_chain:
filename: test/config/integration/certs/servercert.pem
private_key:
filename: test/config/integration/certs/serverkey.pem
filters:
- name: envoy.filters.network.http_connection_manager
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
codec_type: HTTP2
stat_prefix: ingress_http
route_config:
name: local_route
virtual_hosts:
- name: local_service
response_headers_to_add:
- header:
key: alt-svc
value: h3=":10000"; ma=86400, h3-29=":10000"; ma=86400
domains: ["*"]
routes:
- match:
prefix: "/"
route:
host_rewrite_literal: www.envoyproxy.io
cluster: service_envoyproxy_io
http3_protocol_options:
http_filters:
- name: envoy.filters.http.router

listeners:
- name: listener_udp
address:
Expand Down
2 changes: 1 addition & 1 deletion docs/root/configuration/best_practices/edge.rst
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ HTTP proxies should additionally configure:
to true (to avoid consuming HTTP headers from external clients, see :ref:`HTTP header sanitizing <config_http_conn_man_header_sanitizing>`
for details),
* :ref:`connection and stream timeouts <faq_configuration_timeouts>`,
* :ref:`HTTP/2 maximum concurrent streams limit <envoy_v3_api_field_config.core.v3.Http2ProtocolOptions.max_concurrent_streams>` to 100,
* :ref:`HTTP/2 maximum concurrent streams limit <envoy_v3_api_field_config.core.v3.Http2ProtocolOptions.max_concurrent_streams>` and :ref:`HTTP/3 maximum concurrent streams limit <envoy_v3_api_field_config.core.v3.QuicProtocolOptions.max_concurrent_streams>` to 100
* :ref:`HTTP/2 initial stream window size limit <envoy_v3_api_field_config.core.v3.Http2ProtocolOptions.initial_stream_window_size>` to 64 KiB,
* :ref:`HTTP/2 initial connection window size limit <envoy_v3_api_field_config.core.v3.Http2ProtocolOptions.initial_connection_window_size>` to 1 MiB.
* :ref:`headers_with_underscores_action setting <envoy_v3_api_field_config.core.v3.HttpProtocolOptions.headers_with_underscores_action>` to REJECT_REQUEST, to protect upstream services that treat '_' and '-' as interchangeable.
Expand Down
12 changes: 6 additions & 6 deletions docs/root/configuration/best_practices/level_two.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,20 +10,20 @@ edge use case may need to be adjusted when using Envoy in a multi-level deployme
.. image:: /_static/multilevel_deployment.svg

**In summary, if you run level two Envoy version 1.11.1 or greater which terminates
HTTP/2, we strongly advise you to change the HttpConnectionManager configuration of your level
HTTP/2 or above, we strongly advise you to change the HttpConnectionManager configuration of your level
two Envoy, by setting its downstream**
:ref:`validation of HTTP messaging option <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.HttpConnectionManager.stream_error_on_invalid_http_message>`
**to true.**

If there is an invalid HTTP/2 request and this option is not set, the Envoy in
If there is an invalid request and this option is not set, the Envoy in
question will reset the entire connection. This behavior was changed as part of
the 1.11.1 security release, to increase the security of Edge Envoys. Unfortunately,
because there are no guarantees that edge proxies will enforce HTTP/1 or HTTP/2
standards compliance as rigorously as Envoy’s HTTP/2 stack does, this can result
because there are no guarantees that edge proxies will enforce HTTP
standards compliance as rigorously as Envoy’s stack does, this can result
in a problem as follows. If one client sends a request that for example passes
level one proxy's validation checks, and it is forwarded over an upstream multiplexed
HTTP/2 connection (potentially shared with other clients) the strict enforcement on
the level two Envoy HTTP/2 will reset all the streams on that connection, causing
connection (potentially shared with other clients) the strict enforcement on
the level two Envoy will reset all the streams on that connection, causing
alyssawilk marked this conversation as resolved.
Show resolved Hide resolved
a service disruption to the clients sharing that L1-L2 connection. If a malicious
user has insight into what traffic will bypass level one checks, they could spray
“bad” traffic across the level one fleet, causing serious disruption to other users’
Expand Down
7 changes: 5 additions & 2 deletions docs/root/configuration/http/http_conn_man/stats.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ statistics:
downstream_cx_http1_total, Counter, Total HTTP/1.1 connections
downstream_cx_upgrades_total, Counter, Total successfully upgraded connections. These are also counted as total http1/http2 connections.
downstream_cx_http2_total, Counter, Total HTTP/2 connections
downstream_cx_http3_total, Counter, Total HTTP/3 connections
downstream_cx_destroy, Counter, Total connections destroyed
downstream_cx_destroy_remote, Counter, Total connections destroyed due to remote close
downstream_cx_destroy_local, Counter, Total connections destroyed due to local close
Expand All @@ -26,6 +27,7 @@ statistics:
downstream_cx_http1_active, Gauge, Total active HTTP/1.1 connections
downstream_cx_upgrades_active, Gauge, Total active upgraded connections. These are also counted as active http1/http2 connections.
downstream_cx_http2_active, Gauge, Total active HTTP/2 connections
downstream_cx_http3_active, Gauge, Total active HTTP/3 connections
downstream_cx_protocol_error, Counter, Total protocol errors
downstream_cx_length_ms, Histogram, Connection length milliseconds
downstream_cx_rx_bytes_total, Counter, Total bytes received
Expand All @@ -41,6 +43,7 @@ statistics:
downstream_rq_total, Counter, Total requests
downstream_rq_http1_total, Counter, Total HTTP/1.1 requests
downstream_rq_http2_total, Counter, Total HTTP/2 requests
downstream_rq_http3_total, Counter, Total HTTP/3 requests
downstream_rq_active, Gauge, Total active requests
downstream_rq_response_before_rq_complete, Counter, Total responses sent before the request was complete
downstream_rq_rx_reset, Counter, Total request resets received
Expand Down Expand Up @@ -150,8 +153,8 @@ On the upstream side all http2 statistics are rooted at *cluster.<name>.http2.*

.. attention::

The HTTP/2 `streams_active` gauge may be greater than the HTTP connection manager
`downstream_rq_active` gauge due to differences in stream accounting between the codec and the
The HTTP `streams_active` gauges may be greater than the HTTP connection manager
`downstream_rq_active` gauge due to differences in stream accounting between the codecs and the
HTTP connection manager.

Tracing statistics
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ AWS Lambda

The AWS Lambda filter is currently under active development.

The HTTP AWS Lambda filter is used to trigger an AWS Lambda function from a standard HTTP/1.x or HTTP/2 request.
The HTTP AWS Lambda filter is used to trigger an AWS Lambda function from a standard HTTP request.
It supports a few options to control whether to pass through the HTTP request payload as is or to wrap it in a JSON
schema.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ response trailers to a compliant gRPC server. It works by doing the following:
work with unary gRPC APIs.

This filter also collects stats for all gRPC requests that transit, even if those requests are
normal gRPC requests over HTTP/2.
normal gRPC requests over HTTP/2 or above.

More info: wire format in `gRPC over HTTP/2 <https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md>`_.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ gRPC HTTP/1.1 reverse bridge
* This filter should be configured with the name *envoy.filters.http.grpc_http1_reverse_bridge*.

This is a filter that enables converting an incoming gRPC request into a HTTP/1.1 request to allow
a server that does not understand HTTP/2 or gRPC semantics to handle the request.
a server that does not understand HTTP/2 or HTTP/3 or gRPC semantics to handle the request.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Totally your call... I wonder if we should be consistent about saying either "HTTP/2 or HTTP/3" vs "HTTP/2 or above". Probably not worth doing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like in some situations it's awkward to phrase it the other way :-/


The filter works by:

Expand Down
2 changes: 1 addition & 1 deletion docs/root/configuration/http/http_filters/lua_filter.rst
Original file line number Diff line number Diff line change
Expand Up @@ -659,7 +659,7 @@ protocol()
streamInfo:protocol()

Returns the string representation of :repo:`HTTP protocol <include/envoy/http/protocol.h>`
used by the current request. The possible values are: *HTTP/1.0*, *HTTP/1.1*, and *HTTP/2*.
used by the current request. The possible values are: *HTTP/1.0*, *HTTP/1.1*, *HTTP/2* and *HTTP/3*.
phlax marked this conversation as resolved.
Show resolved Hide resolved

downstreamLocalAddress()
^^^^^^^^^^^^^^^^^^^^^^^^
Expand Down
2 changes: 1 addition & 1 deletion docs/root/configuration/observability/access_log/stats.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The gRPC access log has statistics rooted at *access_logs.grpc_access_log.* with
:widths: 1, 1, 2

logs_written, Counter, Total log entries sent to the logger which were not dropped. This does not imply the logs have been flushed to the gRPC endpoint yet.
logs_dropped, Counter, Total log entries dropped due to network or HTTP/2 back up.
logs_dropped, Counter, Total log entries dropped due to network or application level back up.


File access log statistics
Expand Down
2 changes: 1 addition & 1 deletion docs/root/configuration/observability/access_log/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ The following command operators are supported:

%PROTOCOL%
HTTP
Protocol. Currently either *HTTP/1.1* or *HTTP/2*.
Protocol. Currently either *HTTP/1.1* *HTTP/2* or *HTTP/3*.

TCP
Not implemented ("-").
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,11 @@ upstream.use_http2
Whether the cluster utilizes the *http2* if configured in `HttpProtocolOptions <envoy_v3_msg_config.upstreams.http.v3.HttpProtocolOptions>`.
Set to 0 to disable HTTP/2 even if the feature is configured. Defaults to enabled.

upstream.use_http3
Whether the cluster utilizes the *http3* if configured in `HttpProtocolOptions <envoy_v3_msg_config.upstreams.http.v3.HttpProtocolOptions>`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this text is identical to the HTTP/2 text (except 3 instead of 2), but I wonder if we should remove "the" after "utilizes"? "Whether the cluster utilizes http3 if configured..." WDYT?

Set to 0 to disable HTTP/3 even if the feature is configured. Defaults to enabled.


.. _config_cluster_manager_cluster_runtime_zone_routing:

Zone aware load balancing
Expand Down
Loading