From def8d346191a4807c2611284caf6f0eb107f361a Mon Sep 17 00:00:00 2001 From: Sean Sullivan Date: Thu, 18 Jan 2024 15:47:24 -0800 Subject: [PATCH] PortForward over WebSockets update to KEP 4006 --- .../sig-api-machinery/4006.yaml | 2 + .../README.md | 482 ++++++++++++++++-- .../kep.yaml | 18 +- .../portforward-stream-translator-proxy.png | Bin 0 -> 35700 bytes 4 files changed, 453 insertions(+), 49 deletions(-) create mode 100644 keps/sig-api-machinery/4006-transition-spdy-to-websockets/portforward-stream-translator-proxy.png diff --git a/keps/prod-readiness/sig-api-machinery/4006.yaml b/keps/prod-readiness/sig-api-machinery/4006.yaml index d432a704af2..0f36ae82b81 100644 --- a/keps/prod-readiness/sig-api-machinery/4006.yaml +++ b/keps/prod-readiness/sig-api-machinery/4006.yaml @@ -1,3 +1,5 @@ kep-number: 4006 alpha: approver: "@deads2k" +beta: + approver: "@jpbetz" diff --git a/keps/sig-api-machinery/4006-transition-spdy-to-websockets/README.md b/keps/sig-api-machinery/4006-transition-spdy-to-websockets/README.md index 86f6887b5d2..b4de052bc60 100644 --- a/keps/sig-api-machinery/4006-transition-spdy-to-websockets/README.md +++ b/keps/sig-api-machinery/4006-transition-spdy-to-websockets/README.md @@ -92,7 +92,10 @@ tags, and then generate with `hack/update-toc.sh`. - [Background: API Server and Kubelet UpgradeAwareProxy](#background-api-server-and-kubelet-upgradeawareproxy) - [Proposal: kubectl WebSocket Executor and Fallback Executor](#proposal-kubectl-websocket-executor-and-fallback-executor) - [Proposal: New RemoteCommand Sub-Protocol Version - v5.channel.k8s.io](#proposal-new-remotecommand-sub-protocol-version---v5channelk8sio) - - [Proposal: API Server StreamTranslatorProxy](#proposal-api-server-streamtranslatorproxy) + - [Proposal: API Server RemoteCommand StreamTranslatorProxy](#proposal-api-server-remotecommand-streamtranslatorproxy) + - [Background: PortForward Subprotocol](#background-portforward-subprotocol) + - [Proposal: New PortForward Subprotocol Version - v2.portforward.k8s.io](#proposal-new-portforward-subprotocol-version---v2portforwardk8sio) + - [Proposal: API Server PortForward StreamTranslatorProxy](#proposal-api-server-portforward-streamtranslatorproxy) - [Pre-GA: Kubelet StreamTranslatorProxy](#pre-ga-kubelet-streamtranslatorproxy) - [Test Plan](#test-plan) - [Prerequisite testing updates](#prerequisite-testing-updates) @@ -101,10 +104,17 @@ tags, and then generate with `hack/update-toc.sh`. - [e2e tests](#e2e-tests) - [Graduation Criteria](#graduation-criteria) - [Alpha](#alpha) + - [v1.29 RemoteCommand Subprotocol (exec, cp, and attach)](#v129-remotecommand-subprotocol-exec-cp-and-attach) + - [v1.30 PortForward Subprotocol (port-forward)](#v130-portforward-subprotocol-port-forward) - [Beta](#beta) + - [v1.30 RemoteCommand Subprotocol (exec, cp, and attach)](#v130-remotecommand-subprotocol-exec-cp-and-attach) + - [v1.31 PortForward Subprotocol (port-forward)](#v131-portforward-subprotocol-port-forward) - [GA](#ga) - [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy) - [Version Skew Strategy](#version-skew-strategy) + - [RemoteCommand Subprotocol](#remotecommand-subprotocol) + - [PortForward Subprotocol](#portforward-subprotocol) + - [Version Skew within the Control Plane and Nodes](#version-skew-within-the-control-plane-and-nodes) - [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire) - [Feature Enablement and Rollback](#feature-enablement-and-rollback) - [Rollout, Upgrade and Rollback Planning](#rollout-upgrade-and-rollback-planning) @@ -147,7 +157,7 @@ Items marked with (R) are required *prior to targeting to a milestone / release* - [ ] (R) [all GA Endpoints](https://github.com/kubernetes/community/pull/1806) must be hit by [Conformance Tests](https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md) - [X] (R) Production readiness review completed - [X] (R) Production readiness review approved -- [ ] "Implementation History" section is up-to-date for milestone +- [X] "Implementation History" section is up-to-date for milestone - [ ] User-facing documentation has been created in [kubernetes/website], for publication to [kubernetes.io] - [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes @@ -185,7 +195,7 @@ Some Kubernetes clients need to communicate with the API Server using a bi-direc streaming protocol, instead of the standard HTTP request/response mechanism. A streaming protocol provides the ability to read and write arbitrary data messages between the client and server, instead of providing a single response to a client request. -For example, the commands `kubectl exec` and `kubectl attach` +For example, the commands `kubectl exec`, `kubectl attach`, and `kubectl port-forward` both benefit from a bi-directional streaming protocol (`kubectl cp` is build on top of `kubectl exec` primitives so it utilizes streaming as well). Currently, the bi-directional streaming solution for these `kubectl` commands is SPDY/3.1. For @@ -221,8 +231,8 @@ know that this has succeeded? --> 1. Transition the bi-directional streaming protocol from SPDY/3.1 to WebSockets for -`kubectl exec`, `kubectl attach`, and `kubectl cp` for the communication leg -between `kubectl` and the API Server. +`kubectl exec`, `kubectl attach`, `kubectl cp`, and `kubectl port-forward` for the +communication leg between `kubectl` and the API Server. 2. Extend the WebSockets communication leg from the API Server to Kubelet. After this extension, WebSockets streaming will occur between `kubectl` and Kubelet (proxied @@ -300,7 +310,38 @@ How will UX be reviewed, and by whom? Consider including folks who also work outside the SIG or subproject. --> -- TBD +- Risk: Security + +A possible security vulnerability might occur when a potential upgraded connection +is redirected to other API endpoints. + + - Mitigation: Upgraded connections are disallowed from redirecting. + +- Risk: Overloaded Concurrency + +PortForward subrequests (e.g. `curl http://localhost:8080/index.html` after the connection +upgrade) can occur concurrently over the the upgraded streaming connection, and these +subrequests can be long-lasting. Each of these subrequests creates two streams (an +error stream and a data stream) over the connection, and there are four goroutines spawned +to service this subrequest and its associated streams. After the completion of the +subrequest, the associated resources are reclaimed. + + - Mitigation: Throttling the number of concurrent subrequests will limit the + number of concurrent streams and the number of concurrent goroutines on the + API Server. This throttling will ensure the server does not get overloaded. + If we need to the reduce number of concurrent goroutines even further we can + explore goroutine pools so that the number of goroutines will grow sublinearly + with the number of subrequests and streams. + +- Risk: Performance + +When transitioning from the SPDY streaming protocol to WebSockets, there may be a +performance degradation. In order to implement the WebSocket streaming functionality +that SPDY already implements, it is necessary for additional headers to be prepended +to the WebSocket data package. + + - Mitigation: Performance testing to ensure the WebSockets implementation is not + noticeably slower than the current SPDY streaming implementation. ## Design Details @@ -326,7 +367,7 @@ could look like: **HTTP Request** ``` -POST /api/v1/…/pods/nginx/exec?command=... HTTP/1.1 +GET /api/v1/…/pods/nginx/exec?command=... HTTP/1.1 Connection: Upgrade Upgrade: SPDY/3.1 X-Stream-Protocol-Version: v4.channel.k8s.io @@ -412,7 +453,7 @@ currently arises when sending data over the STDIN stream, and it is more fully d in the following issue: [exec over web sockets protocol is flawed](https://github.com/kubernetes/kubernetes/issues/89899). A new RemoteCommand version (`v5.channel.k8s.io`) adds this `CLOSE` signal. -### Proposal: API Server `StreamTranslatorProxy` +### Proposal: API Server RemoteCommand `StreamTranslatorProxy` ![Stream Translator Proxy](./stream-translator-proxy-2.png) @@ -426,6 +467,122 @@ translation proxy terminates the WebSocket connection, and it de-multiplexes the various streams in order to pass the data on to a SPDY connection, which continues upstream (to Kubelet and eventually the container runtime). +### Background: `PortForward` Subprotocol + +The following steps articulate the difference betweeen a **request** for an +upgraded streaming connection, and the following **subrequests** which are made +over the upgraded connection. + +1. `kubectl port-forward` makes a **request** to the server upgrading to a SPDY streaming +connection. +2. An arbitrary number of subsequent (and possibly concurrent) **subrequests** can be made over +this previously established connection. Example: `curl http://localhost:8080/index.html`. +3. Each of these **subrequests** creates two streams over the connection (a uni-directional +error stream and a bi-directional data stream) between the client and the container runtime. +4. The resources associated with the **subrequest** are reclaimed once the **subrequest** +is completed. + +The `PortForward` subprotocol is used to implement `kubectl port-forward`, and it differs +from the `RemoteCommand` subprotocol in how the multiple streams within the single upgraded +connection are created. The `RemoteCommand` subprotocol statically creates streams (e.g +STDIN, STDOUT, etc.) when the connection is created. But the `PortForward` subprotocol +dynamically creates two streams (a bi-directional data stream and a unidirectional error +stream) for each subsequent portforward subrequest. These streams are removed when the subrequest +is complete. The connection, however, continues to exist until it is manually stopped +(with a signal). For example, the following portforward command creates the upgraded +connection, but without any streams, listening on the local port 8080: +``` +$ kubectl port-forward nginx 8080:80 +``` +Once the upgraded, streaming connection is created, portforward subrequests are handled +by dynamically creating a data and error stream in another goroutine, forwarding the +data to the remote port on the target. In this example, the HTTP subrequest is forwarded +over the data stream from the local port 8080 to the nginx container listening on port +80: +``` +$ curl http://localhost:8080/index.html +``` +The nginx HTTP response is returned over the same data stream. Once the subrequest is +complete, both streams are closed and removed. + +### Proposal: New `PortForward` Subprotocol Version - `v2.portforward.k8s.io` + +In order to implement `PortForward` over WebSockets, we propose the client attempt to +upgrade the connection to WebSockets with a new `v2.portforward.k8s.io` subprotocol +version. This new version will implement stream functionality over WebSockets +which currently exists in SPDY but not WebSockets. Specifically, the WebSockets streams +will implement: + +1. StreamCreate signal: A signal to the other WebSockets endpoint that a +stream has been created. This communication will also contain the headers used to +create the stream. +2. StreamClose signal: A signal to the other WebSockets endpoint that the stream +has been half-closed, and will not allow any further writing on the stream from +the closed end. +3. Larger stream identifier space: In order to accommodate numerous concurrent +streams from many posssible portforward subrequests, we will need a large stream +identifier space. It appears that SPDY implements 2^31 possible stream identifiers +and we will provide four bytes for the WebSockets stream identifier to match SPDY. +The size of these four bytes should not be material, since the size of the +WebSockets buffer is much larger (32KB). But if we need to reduce the size of +the stream identifier, we can use a `varint` instead of a `uint32`. + +The WebSockets stream functionality will be implemented by encoding and decoding +stream headers within the WebSockets data message. The stream header struct will +look like: +``` +// wsStreamHeader contains the data at the beginning of a websocket binary message. +type wsStreamHeader struct { + MessageType byte // Create, Close, or Data + StreamID uint32 // or varint if we need to optimize + // Headers are only included in a Create message type. Well-known keys are: + // 1. StreamType: DataStream or ErrorStream. + // 2. RequestID: A unique identifier for the subrequest. + // 3. Port: The remote port the stream is forwarding data to. + Headers http.Header +} +``` + +### Proposal: API Server PortForward `StreamTranslatorProxy` + +![PortForward Stream Translator Proxy](./portforward-stream-translator-proxy.png) + +Updated steps detailing **requests** and **subrequests** for the proposed `StreamTranslatorProxy`. + +1. `kubectl port-forward` makes a **request** to the API Server to upgrade to a WebSockets +streaming connection. At the API Server, the WebSockets connection is upgraded and terminated, +while a legacy SPDY connection is created upstream to the container runtime. +2. An arbitrary number of subsequent (and possibly concurrent) client **subrequests** can be made over +this previously established WebSockets connection. Example: `curl http://localhost:8080/index.html`. +3. Each of these **subrequests** creates two streams over the connection (a uni-directional +error stream and a bi-directional data stream) between the client and the API Server. +4. The API Server in turn creates a one-to-one correspondence between the WebSockets streams +and upstream SPDY streams. The WebSocket streams are connected to SPDY streams by goroutines +copying data between them. So each **subrequest** spawns four goroutines to service the two +streams (one for the **subrequest** itself, as well as three to copy stream data in each +direction). +5. All the resources associated with the **subrequest** are reclaimed once the **subrequest** +is completed. + +Similar to `RemoteCommand`, `PortForward` will also have a `StreamTranslatorProxy` +within the API server to route data from WebSocket streams onto upstream, legacy +SPDY streams. The new portforward `StreamTranslatorProxy` will handle requests +for WebSockets connection upgrades with a `v2.portforward.k8s.io` header. The +portforward `StreamTranslatorProxy` will initially attempt to create an upstream +SPDY connection to the Kubelet using the legacy `v1.portforward.k8s.io` header. +If successful, the `StreamTranslatorProxy` will create a server-side WebSockets +connection, returning `101 Switching Protocols` to the WebSockets client. The +server-side WebSockets connection will handle the `StreamCreate` and `StreamClose` +signals, as well as de-multiplexing WebSocket streams using the passed stream +identifier. Upon receiving a `StreamCreate` signal on the server-side of the +WebSockets connection, a WebSocket stream will be created and queued onto a +stream create channel. On the other end of the channel, the `StreamTranslatorProxy` +will create a SPDY stream to associate with the WebSockets stream, using headers +from the WebSockets stream. And if both portforward request streams have been +created (data and error), then streaming will commence between the WebSockets +and SPDY streams. Upon completion, the streams will be closed and removed. The +`StreamClose` signal will be used to determine if the streaming has completed. + ### Pre-GA: Kubelet `StreamTranslatorProxy` The eventual plan is to incrementally transition all SPDY communication legs to WebSockets. @@ -621,6 +778,8 @@ in back-to-back releases. #### Alpha +##### v1.29 RemoteCommand Subprotocol (exec, cp, and attach) + - Implement the alpha version of the `RemoteCommand` subprotocol, and surface the new `kubectl exec`, `kubectl cp`, and `kubectl attach` behind a `kubectl` environment variable which is **OFF** by default. @@ -632,17 +791,39 @@ in back-to-back releases. - Existing `exec`, `cp`, and `attach` integration tests continue to work. - Existing `exec`, `cp`, and `attach` e2e tests continue to work. +##### v1.30 PortForward Subprotocol (port-forward) + +- Implement the alpha version of the `PortForward` subprotocol, and surface the new + `kubectl port-forward` behind the `kubectl` environment variable KUBECTL_PORT_FORWARD_WEBSOCKETS + which is **OFF** by default. +- FallbackDialer is completed and functional behind the `kubectl` environment variable + KUBECTL_PORT_FORWARD which if **OFF** by default. The FallbackDialer executes legacy + SPDY `port-forward` if the server does not support the new WebSockets functionality. +- PortForward `StreamTranslatorProxy` successfully added and integrated, living + behind the API Server feature flag `PortForwardWebsockets` which is **OFF** by default. + #### Beta +##### v1.30 RemoteCommand Subprotocol (exec, cp, and attach) + - Additional `exec`, `cp`, and `attach` unit tests completed and enabled. - Additional `exec`, `cp`, and `attach` integration tests completed and enabled. - Additional `exec`, `cp`, and `attach` e2e tests completed and enabled. +##### v1.31 PortForward Subprotocol (port-forward) + +- Additional `port-forward` unit tests completed and enabled. +- Additional `port-forward` integration tests completed and enabled. +- Additional `port-forward` e2e tests completed and enabled. + #### GA - Conformance tests for `RemoteCommand` completed and enabled. - Conformance tests for `RemoteCommand` have been stable and non-flaky for two weeks. +- Conformance tests for `PortForward` completed and enabled. +- Conformance tests for `PortForward` have been stable and + non-flaky for two weeks. - Extend the WebSockets communication leg from the API Server to Kubelet. ### Upgrade / Downgrade Strategy @@ -671,15 +852,17 @@ components? What are the guarantees? Make sure this is in the test plan. Consider the following in developing a version skew strategy for this enhancement: -- Does this enhancement involve coordinating behavior in the control plane and - in the kubelet? How does an n-2 kubelet without this feature available behave - when this feature is used? +- Does this enhancement involve coordinating behavior in the control plane and nodes? +- How does an n-3 kubelet or kube-proxy without this feature available behave when this feature is used? +- How does an n-1 kube-controller-manager or kube-scheduler without this feature available behave when this feature is used? - Will any other components on the node change? For example, changes to CSI, CRI or CNI may require updating that component before the kubelet. --> This feature needs to take into account the following version skew scenarios: +#### RemoteCommand Subprotocol + 1. A newer WebSockets enabled `kubectl` communicating with an older API Server that does not support the newer `StreamTranslator` proxy. @@ -696,6 +879,33 @@ supports the newer `StreamTranslator` proxy. The legacy `kubectl` will successfully request an upgrade for `SPDY/RemoteCommand - V4`, just as it has for the last several years. +#### PortForward Subprotocol + +1. A newer WebSockets enabled `kubectl` communicating with an older API Server that +does not support the newer PortForward `StreamTranslator` proxy. + +In this case, the initial upgrade request for `PortForward` WebSockets will +fail, because the `WebSockets` upgrade request `v2.portforward.k8s.io` will be proxied +to the current container runtime which only supports version `v1.portforward.k8s.io`. +Upon receiving this upgrade failure, the portforward client will fallback to the +legacy SPDY `v1.portforward.k8s.io`. In this fallback case, the PortForward streaming +functionality in this case will work exactly as it has for the last several years. + +2. A legacy non-WebSockets enabled `kubectl` communicating with a newer API Server that +supports the newer PortForward `StreamTranslator` proxy. + +The `kubectl port-forward` will successfully request an upgrade for legacy +`SPDY/PortForward - V1`, just as it has for the last several years. + +#### Version Skew within the Control Plane and Nodes + +These proposals do not modify intra-cluster version skew behavior. The entire reason +for the current `StreamTranslatorProxy` design is to ensure no modifications +to communication within the Control Plane. The `StreamTranslatorProxy` can update +streaming between the client and the API Server, but it is designed to provide legacy +SPDY streaming from the API Server to the other components within the ControlPlane. +Once this `StreamTranslatorProxy` is moved to the kubelet, we will have to address +the possibility of intra-cluster version skew. ## Production Readiness Review Questionnaire @@ -740,8 +950,11 @@ well as the [existing list] of feature gates. --> - [X] Feature gate (also fill in values in `kep.yaml`) - - Feature gate name(s): KUBECTL_REMOTE_COMMAND_WEBSOCKETS, ClientRemoteCommandWebsockets - - Components depending on the feature gate: kubectl, API Server + - Feature gate name(s) for RemoteCommand Subprotocol: + KUBECTL_REMOTE_COMMAND_WEBSOCKETS, TranslateStreamCloseWebsocketRequests + - Feature gate name(s) for PortForward Subprotocol: + KUBECTL_PORT_FORWARD_WEBSOCKETS, PortForwardWebsockets +- Components depending on the feature gate: kubectl, API Server ###### Does enabling the feature change any default behavior? @@ -750,13 +963,20 @@ Any change of default behavior may be surprising to users or break existing automations, so be extremely careful here. --> -Enabling the feature gate on the API Server will allow the streaming mechanism -to be WebSockets instead of SPDY for communication between `kubectl` and the API -Server. The `kubectl` client must also have the KUBECTL_REMOTE_COMMAND_WEBSOCKETS -environment variable set to **ON**, so it will request the newer WebSockets streaming -feature. These modifications, however, will be transparent to the user unless the -`kubectl`/API Server communication is communicating through an intermediary such -as a proxy (which is the whole reason for the feature). +For each of the two streaming subprotocols: `RemoteCommand` (such as `/exec` and +`/attach` APIs) and `PortForward` (for `/portforward`), enabling the respective +feature gate on the API Server will allow the streaming mechanism to be WebSockets +instead of SPDY for communication between `kubectl` and the API Server. Additionally, +the `kubectl` client must also have the KUBECTL_REMOTE_COMMAND_WEBSOCKETS environment +variable set to **ON** for `exec`, `cp`, and `attach` commands. While the +KUBECTL_PORT_FORWARD_WEBSOCKETS environment variable must be set to **ON** for +`port-forward` command. These modifications, however, will be transparent to the +user unless the `kubectl`/API Server communication is communicating through an +intermediary such as a proxy (which is the whole reason for the feature). + +**NOTE** These two sets of feature flags are currently at different maturity levels. +As of v1.30, `RemoteCommand` feature flags are **enabled** by default (Beta), while +`PortFoward` features flags are **disabled** by default (Alpha). ###### Can the feature be disabled once it has been enabled (i.e. can we roll back the enablement)? @@ -771,9 +991,9 @@ feature. NOTE: Also set `disable-supported` to `true` or `false` in `kep.yaml`. --> -The feature can be disabled for a single user by setting the `kubectl` environment -variable associated with the feature to **OFF**. Or the feature can be turned off -for all `kubectl` users communicating with a cluster by turning off the feature flag +The features can be disabled for a single user by setting the `kubectl` environment +variable associated with the feature to **OFF**. Or the features can be turned off +for all `kubectl` users communicating with a cluster by turning off the feature flags for the API Server. ###### What happens if we reenable the feature if it was previously rolled back? @@ -806,8 +1026,6 @@ https://github.com/kubernetes/kubernetes/pull/97058/files#diff-7826f7adbc1996a05 This section must be completed when targeting beta to a release. --> -- TBD (complete for beta) - ###### How can a rollout or rollback fail? Can it impact already running workloads? The most straightforward signal indicating a problem for the feature is failures -for `kubectl exec, cp, attach` commands. +for `kubectl exec, cp, attach, and port-forward` commands. ###### Were upgrade and rollback tested? Was the upgrade->downgrade->upgrade path tested? @@ -843,7 +1063,12 @@ Longer term, we may want to require automated upgrade/rollback tests, but we are missing a bunch of machinery and tooling and can't do that now. --> -- TBD (complete for beta) +- For upgrade and rollback, both version skew scenarios were successfully tested. So + if, during upgrade or rollback, one of the API servers does not support the new WebSockets + functionality, the client will successfully fallback to the legacy SPDY streaming. + +- The specified upgrade->downgrade->upgrade path was **not** tested, since there is no + persisted state. For that reason, this particular scenario is **not applicable**. ###### Is the rollout accompanied by any deprecations and/or removals of features, APIs, fields of API types, flags, etc.? @@ -862,8 +1087,6 @@ For GA, this section is required: approvers should be able to confirm the previous answers based on experience in the field. --> -- TBD (complete for beta) - ###### How can an operator determine if the feature is in use by workloads? -- TBD (complete for beta) - +- An operator can detect if the WebSocket functionality is enabled by checking + either the `num_ws_remote_command_v5_total[success]` metric for `RemoteCommand` or + the `num_ws_port_forward_v2_total[success]` metric for `PortForward`. ###### How can someone using this feature know that it is working for their instance? @@ -888,9 +1112,14 @@ Recall that end users cannot usually observe component logs or access metrics. - [X] Other (treat as last resort) - Details: - - `kubectl exec -v=7 -- date` - - One of the request headers will be `K8s-Websocket-Protocol: stream-translate` + - RemoteCommand: `kubectl exec -v=7 -- date` + - RemoteCommand: One of the output log lines will be + `...websocket.go:137] The subprotocol is v5.channel.k8s.io` if using the new WebSockets streaming. + - PortForward: `kubectl port-forward -v=7 ` + - PortForward: One of the output log lines will be + `...websocket-dialer.go:91] negotiated protocol: v2.portforward.k8s.io` + if websockets is enabled for port forwarding. ###### What are the reasonable SLOs (Service Level Objectives) for the enhancement? @@ -909,6 +1138,20 @@ These goals will help you determine what you need to measure (SLIs) in the next question. --> +At a high level, we are aiming for an SLO which is the same as the current SPDY +streaming SLO. Turning on WebSockets streaming between the `kubectl` client and +the API Server should not noticeably degrade streaming performance compared to +the current SPDY implementation. But since we do not have numbers for our current +SPDY streaming implementation, these SLO's are necessarily educated estimates. + +- 99.9% of initial HTTP connections eventually succeed in upgrading to a streaming + connection (i.e. `101 Switching Protocols` as the HTTP response). + +- 99.9% of streaming connections complete without writing to the error channel. + Each of the streaming subprotocols, `RemoteCommand` and `PortForward`, create + a separate error channel to communicate problems while streaming. But a normally + successful streaming command should **NOT** need to use these channels. + ###### What are the SLIs (Service Level Indicators) an operator can use to determine the health of the service? - [X] Metrics - - Metric name: TBD (complete for beta) - - Components exposing the metric: kube-apiserver + - Metric name: `num_ws_remote_command_v5_total` with `type` dimension containing + enum values `success` and `failure`. Counts the total number of times the API + Server witnessed a WebSocket/V5 RemoteCommand connection upgrade attempt (either + `success` or `failure`). + - Components exposing the metric: API Server + + - Metric name: `num_ws_port_forward_v2_total` with `type` dimension containing + enum values `success` and `failure`. Counts the total number of times the API + Server witnessed a WebSocket/V2 PortForward connection upgrade attempt (either + `success` or `failure`). + - Components exposing the metric: API Server ###### Are there any missing metrics that would be useful to have to improve observability of this feature? @@ -926,7 +1178,7 @@ Describe the metrics themselves and the reasons why they weren't added (e.g., co implementation difficulties, etc.). --> -- TBD (complete for beta) +N/A ### Dependencies @@ -934,7 +1186,8 @@ implementation difficulties, etc.). This section must be completed when targeting beta to a release. --> -- TBD (complete for beta) +- Gorilla/WebSockets library: The new WebSockets streaming functionality imports the + Gorilla/WebSockets library. ###### Does this feature depend on any specific services running in the cluster? @@ -1066,9 +1319,9 @@ Are there any tests that were run/should be run to understand performance charac and validate the declared limits? --> -`kubectl exec, cp, attach` commands spawn container runtime processes, so there is -the danger of node resource exhaustion. This feature, however, does not change the -current legacy mechanism for how these container runtime processes execute or +`kubectl exec, cp, attach, and port-forward` commands spawn container runtime processes, +so there is the danger of node resource exhaustion. This feature, however, does not +change the current legacy mechanism for how these container runtime processes execute or communicate, except for the communication leg between `kubectl` and the API Server. There should be no more risk of node resource exhaustion than already exists. @@ -1087,7 +1340,10 @@ details). For now, we leave it here. ###### How does this feature react if the API server and/or etcd is unavailable? -- TBD (complete for beta) +- This feature fails when the API server is unavailable. The WebSocket streaming + is proxied through the API server. If the API server is unavailable, streaming + functionality does not work (this applies to the legacy SPDY streaming as well, + so there is no detectable difference in this scenario). ###### What are other known failure modes? @@ -1104,10 +1360,144 @@ For each of them, fill in the following information by copying the below templat - Testing: Are there any tests for failure mode? If not, describe why. --> -- TBD (complete for beta) +- Failure Mode: Proxy or Gateway that does not support SPDY or WebSockets. + SPDY had been deprecated for since 2015, and not all proxies support + WebSockets. If the intermediary does not support either streaming protocol + it will not be possible to run the `kubectl` streaming commands. + - Detection: The `kubectl` streaming command will return a connection error. + The error returned to the client will be from the proxy or gateway; not from + the API Server (since the communication never reaches the API Server). + - Mitigations: None + - Diagnostics: N/A + - Testing: N/A + +- Failure Mode: Overloaded API Server from too many concurrent streaming commands. + - Detection: The streaming commands will show increased latency and timeout errors. + The streaming commands could also hang after initiation on the client. + - Mitigations: The mitigations for this phenomenon are described in detail in + the [Risks and Mitigations](#risks-and-mitigations) section. But a simple + way for a user to mitigate this problem would be to reduce the number of concurrent + streaming command from clients. + - Diagnostics: API Server `/healthz` and `/metrics` monitoring. + - Testing: Manual stress tests of `kubectl` streaming commands through shell + scripts which initiate numerous concurrent streaming commmands. + +- Failure Mode: Increased network latency between the client and the API Server causes + problems with streaming commands. + - Detection: The client streaming commands will lag, timeout, and possibly block. + - Mitigations: Mitigating this problem would require fixing the problematic network + connection. + - Diagnostics: Adding extra logging from the `kubectl` streaming commands will + show the timings for individual streaming request/responses. For example: + `$ kubectl exec -v=7 ...` + will produce logging which shows response times, like: + `... Response Status: 101 Switching Protocols in 20 milliseconds` + Also, API Server `/healthz` and `/metrics` monitoring will produces output + demonstrating increased response times and increased timeouts. + - Testing: Manual stress tests which simulate a bad network connection by + randomly dropping streamed packets. + +- Failure Mode: API Server that does not support the new WebSockets functionality + or the the feature flags are not enabled. If `kubectl` supports new WebSockets + functionality, but the API Server does not, we automatically fall back to + legacy SPDY streaming. + - Detection: The fallback functionality is automatic. But if the user wanted + to see if this fallback was occurring, the user could add the `-v=7` flag + to the `kubectl` command. The output logging would display the initial + WebSockets connection upgrade attempt and failure, and the subsequent SPDY + completion of the command. + - Mitigations: The mitigation (fallback to legacy SPDY) is automatic. + - Diagnostics: We have suggested metrics to measure the number of fallbacks. + the metrics `num_ws_remote_command_v5_total[failure]` as well as + `num_ws_port_forward_v2_total[failure]` will measure the number of fallbacks. + - Testing: We have implemented tests for both the `FallbackWebSocketExecutor` + (for `RemoteCommand`), and the `FallbackDialer` (for `PortForward`). ###### What steps should be taken if SLOs are not being met to determine the problem? +- Step 1: Turn off the `kubectl` feature gate, and check the SLO afterwards. + +For `kubectl exec, kubectl cp, and kubectl attach`: + +``` +$ unset KUBECTL_REMOTE_COMMAND_WEBSOCKETS +# Run simple exec command with higher verbosity to see relevant logging. +# Look for successful connection upgrade with response 101 Switching Protocols. +# Example: $ kubectl exec -v=7 -- date +# +$ kubectl exec -v=7 nginx -- date +... +I0206 02:22:40.500200 3666799 round_trippers.go:463] POST https://127.0.0.1:37243/api/v1/namespaces/default/pods/nginx/exec?command=date&container=nginx&stderr=true&stdout=true +... +I0206 02:22:40.521184 3666799 round_trippers.go:574] Response Status: 101 Switching Protocols in 20 milliseconds +... +Tue Feb 6 02:22:40 UTC 2024 +``` + +For `kubectl port-forward`: + +``` +$ unset KUBECTL_PORT_FORWARD_WEBSOCKETS +# Run simple port-forward command against webserving pod/deployment/service +# with higher verbosity to see relevant logging. Look for successful connection +# upgrade with response 101 Switching Protocols. +# Example: $ kubectl port-forward -v=7 : +# +$ kubectl port-forward -v=7 nginx 8080:80 +... +I0206 02:28:04.352912 3669354 round_trippers.go:463] POST https://127.0.0.1:37243/api/v1/namespaces/default/pods/nginx/portforward +... +I0206 02:28:04.372175 3669354 round_trippers.go:574] Response Status: 101 Switching Protocols in 19 milliseconds +Forwarding from 127.0.0.1:8080 -> 80 +Forwarding from [::1]:8080 -> 80 +I0206 02:28:04.372540 3669354 portforward.go:309] Before listener.Accept()... +I0206 02:28:04.372586 3669354 portforward.go:309] Before listener.Accept()... +... + +# In another terminal, print out index.html served from TARGET after request forwarded. +# Example: $ curl http://localhost:/index.html +# +$ curl http://localhost:8080/index.html + + + +Welcome to nginx! +... +``` + +- Step 2: For a more thorough possible fix, restart the API server with the + cluster feature gate turned off. Check the SLO after restart. + +While there are many ways to start a Kubernetes cluster, I will detail how to modify +the cluster feature flags for this WebSockets functionality using `kubeadm` or `kind` +and cluster configuration files. + +Example: + +``` +$ kind create cluster --config cluster-config.yaml --name --image +``` + +or + +``` +$ kubeadm init --config cluster-config.yaml +``` + +where the `cluster-config.yaml` looks like: + +``` +kind: Cluster +apiVersion: kind.x-k8s.io/v1alpha4 +featureGates: + # Enables/Disables WebSockets for RemoteCommand (e.g. exec, cp, attach) + "TranslateStreamCloseWebsocketRequests": false + # Enables/Disables WebSockets for PortForwar (e.g. port-forward) + "PortForwardWebsockets": true +... +``` + + ## Implementation History -- First Kubernetes release where initial version of KEP available: v1.29 +- First Kubernetes release with initial version of KEP available: v1.29 +- RemoteCommand over WebSockets shipped as alpha: v1.29 +- RemoteCommand over WebSockets shipped as beta: v1.30 +- First Kubernetes release where PortForward over WebSockets described in KEP: v1.30 +- PortForward over WebSockets shipped as alpha: v1.30 ## Drawbacks diff --git a/keps/sig-api-machinery/4006-transition-spdy-to-websockets/kep.yaml b/keps/sig-api-machinery/4006-transition-spdy-to-websockets/kep.yaml index dd90be58dc8..e35fba01e3e 100644 --- a/keps/sig-api-machinery/4006-transition-spdy-to-websockets/kep.yaml +++ b/keps/sig-api-machinery/4006-transition-spdy-to-websockets/kep.yaml @@ -14,14 +14,15 @@ reviewers: approvers: - "@deads2k" - "@jpbetz" + - "@liggitt" # The target maturity stage in the current dev cycle for this KEP. -stage: alpha +stage: beta # The most recent milestone for which work toward delivery of this KEP has been # done. This can be the current (upcoming) milestone, if it is being actively # worked on. -latest-milestone: "v1.29" +latest-milestone: "v1.30" # The milestone at which this feature was, or is targeted to be, at each stage. milestone: @@ -35,11 +36,18 @@ feature-gates: - name: KUBECTL_REMOTE_COMMAND_WEBSOCKETS components: - kubectl - - name: ClientRemoteCommandWebsockets + - name: KUBECTL_PORT_FORWARD_WEBSOCKETS + components: + - kubectl + - name: TranslateStreamCloseWebsocketRequests + components: + - kube-apiserver + - name: PortForwardWebsockets components: - kube-apiserver disable-supported: true -# The following PRR answers are required at beta release +# The following PRR answers are required at beta release. metrics: - - TBD (complete for beta) + - num_ws_remote_command_v5_total + - num_ws_port_forward_v2_total diff --git a/keps/sig-api-machinery/4006-transition-spdy-to-websockets/portforward-stream-translator-proxy.png b/keps/sig-api-machinery/4006-transition-spdy-to-websockets/portforward-stream-translator-proxy.png new file mode 100644 index 0000000000000000000000000000000000000000..f23b2bb8203b5bbf9e1d245db47633e9132c3ba0 GIT binary patch literal 35700 zcmeFYRYRLyv@RMvI23nii#sju#idB0SkMB6;x56RwiGE^3c-sz1oz?;DDJ_7I|S$C zTi;&m?6c0g*gs(3GaRNEL4TK;(0sm_nU3!`(chFGS&tDkuprzeJ-xmzDXBgTuj5 z#FpL36Ak==PK-_*6O5w37d(GHvU?BP&72pHxJ7^@1s8P$Yg;-|U;VlBgCOP)_tGIp z70xB?m7I0f#BYF1;Y9!U2+YtLv%j;0S^95b z_BFXp$8%RULdi1-%E?;$mh4YHVMB*n#bD2HCAm6I*BZ@fHa0ks_V zyq!dlj#OVzbJY$r@oH#FK6Lty_)XT#r2Ivovmz8hsXN72ZKa`h0WZCy19%cIrbt*TKbIeBRAtK$S`@jB4cA%`MxaVk`(in~ zAJx>^G(AG+a5^(uSRhXOHLz#{uT1S|s|(Ha61n!>F*jgB?dt}wYbX!`2PAnV(1={j zhN`g`eZ_H)sl>H$?j)bR0b)}eM8Z_ZComWHDhhl<_<%C81tIbPR2$BWP7-yIuLyEi zHX29_MenXP!0k-*6Uu<#mFw874)?r!Na2c;l?-+MEKt&FXL)>Z(Mr5!9$Z4^kwzNo zDe+@Ap(F1RT%vMhAP&Prs=U?lstne@gbNG% zB3L`cF4D;}UA{xd$G?uT@%vVLayW6^ie_jOXiDBE0njyzTpxU)l7oH$9oFRzMerdd zg8o6zvkz&SD~?`|99J-@kQ+WFv>w$HD=7qq*-Ddw_;EKaJz4OksjUi@I%N1+e+X`* zKaO;P!_=J>cr~ouhLqGa8k*<2G286pDai5;htk- zxs`rAZhUBfvpIFVB+J-d@o1aTJTu?hl{~qSPXPDtdd9yB7e2Ke1T^dyhYT>eS4Kf5 z3yyN0ZMUGKg*1)4>i!WI-!SsMqkpMC^BmawNN|%GREroJdkXZ-y1yB{%GB3X zBUnsAd`Q`vdU5W*D0?vS)}liRxwEgK(jpF_2a4SsqlG#ER}ngulrM?JN2zNyj;2~ z>ij1nml=)7^IMufH(a?fl8fEVMr^BMTW(;?^07Ce)AB`U=%I4q5k$uC%?U1CoYWU) z5c>_?^_ol|xJU5H3u+zgl6Y<+*7Q)8tsy8gmLBVoc1q(u?|G!iS#*vE!kt+RXEID< z&?4HQ3w^{g!HQ3~m?a8mJm5`{`#WqM<=(sf1OAJ*ZH5W2LsVx)KmiQ3SV zx(o)Nd7YaMn(tY|CCt?*c~}m2J>g=@1F-(P6g<`GH$b%wx|Tsd&L;>xVbxYEVWz!2 z9lCi`)7X|XCsF>uU1Td9bHR*f&za#Y?dn+kSXk|m-j&zd#d48;8`z~`EjcUt^h)f1 zr7%go{g|VOm)<6cKR-LyhC;pIZ3+U%{cci$Ci43UkxjU!WuX zo1oFU-MYapNjqmwtA=Jc#bn?+3#ut(nw#_7^@yF&p|f)CX1IcJ-$GV9eM8Z-NLK+p zYiws)<=v=6cyMfi5v*7;y*W25g$d$@%z1N?>ou325N;eGrY13Juk%tpR;trK=un$CxG_YMPcCEXtmPO2mj( z^W;wJEDZFxbWgPwkmkc@Tphie!U4)`;+Qy#+<90zZ=ZupoC#ta`+-&{2Mi%v+Cix~ z@m<}i+dU-nxOXz<>A7rjRaV(&=8r;Pxszap>h;j^E3`NBM972Qr7)kng{k4=zIZGO zIu(W>x@;s{#E6*|;0)^N&?ba5C-w~G(3*suPg-B~>-c`s;BcV8_-dt(k*1{78Pu>( zXGUvP#b)J$q~L1|3`V!_ZdDLIh9(LSirlNs$%EpXMJFqB(wF8D=j_r^Obq%4qgBKr z5A5Jk85~YcbjB{o(8i~h#m~vZ6VI;+Q5-u>3;H$DlzCXHvt!7(J7U1TKVX z=L4)7Aa;Xmw&1F`!HXORb%J9Lq>P`|7R!SRN|(EP zpKup1bfa=H8QIJ*h9)J2m9MVCs~CtzB9XW#oMz{Y@M?;%C+}+Sb3Xu5VNo<)O;obR z#sQuo6dl! zLIk?A)0bBOY_C5!H+3+wVDnhqZMI@&n=oui?XGyQho8)7`|H6d|KPq2xBzJQLY_v% zx&K6d93b}{$JQY1S;?9?;2j%KCKUf1g>w<ILSZ&QUh0x( zwOs{1w2bCf&CL2hfg#$z+Pliz5N25--Xw;RNx^0Zc+R>&K0X|#G6W`6u2MO3)9B3W zI^533V7zX}lbm*2ISmo(%lrt(719MlBTgU1a*lXey!SaBEC^JqU?>huQ+|qkcs$Nw z5F@@`s^Dy_ms2imb9LeXSi$oMwAE8Jb ziwep3AR&+ucoCZc+;{YxA_?ep9nYK(TM)9f-shJ5+gmLpWI+^celwo?-Mi4bES?Nc z6n&pYEjE=vIjS~_Ni1!H1YJb?Z1?#6Ekvd&RD&;ygVg7{*;J0v7o>-SZ-~3VzQ`Ct z;9l`<44lQ-%eVpc4H^u^B8}5u3=V4&+*smmMcvu#=HDPNp-w;Tb0Y;%Bvx_SyH@0QUhs1X-vzF8vCJ)=FGrhLqD?4&lj z%Z6hx&60$+jyX$UBZtkYv&LQ@MAMavYg}pk-22^Q{ji0KANNhTz7m1j!XaJHn?yoHSc&O%bm3nSLi8^NTUc^>ONpPg$x+}B70}F-EQ;82fAMn<&dyIG zjvohw{a$FfXg==`_inoVyLoS)bPS!_Fh<(d{oG6qQ*)>((*3i_fq@{K!RpJ*9=V$< z4%YoF%Rz6OaPV3|h86*huO=)pPJ&=N*mwsQl@D$nLgqon6)Z!-DZ1A1Jk#^eyHb1` z=YEt(-qKBU2oHhlRe>mD+jMDuJ6gX^KcI3kyWd2+`TNiWZ0Ew4G-B3gpIM``cm$Ky ztNDKJy5e$x??qrWk5@%YGZQM%gp)!SzKX!#d~aYYtg?ZSem62AdPmq6cR2*1~BE4rsG=77cmA;nlxXev?BKhD@JDDIr1 z$7MDNtx-O#)gifa-4AmA=j79}1 zDc!<@79_(k{UeV zcEw1*l|sdPoi^pY?`St&@`f+lZj_av@s=p_=fS<$p-IA<1d-#Q4VZU8dl)aweO-;v zOXCa$70lxDZI~_!glQ)7SRc|Meb2Jx45u=ipX;JQ`EZsO&XY~82)oadlw9OiHo^~?fYx9t=)>tK6o^Kr}%jgy$@LUEqB zbLlIbMd2BAjhxR#GaWIJKGji119j{9aX&bj-I8(#fh8)?VjglTsd5;fS6~XS^7mf!%6upS5+N3Xm}WVZR@}brA_5 z3%-D!`3c+2r`zuHg7ojskj1DRbu!kavL;A#?zTsd#!TvhB`;RO9OFtRwf?fce5>Mh zrntj{-M#>4hcNZ1#9o^)cP)a&c;41N*}f%fOM(=cIBuB1g)4C_&|2=owL^PReD=ss z>{3v{<$9AoH^>Zr{Rll867(7LFsDq)?3u%EQYKGc+TQ^pg}dgDEjSeyu_n0T=kb06 z$@IaDMLSj)1axwmy+F{RL-I+1MOv>y%7!goZ75+@uzP4>p4tVS5IZywD!7syY3W^; zacsA)iY1!z$^*Cmz|uaO%!h$!pz}0DE~K!K!l-KuTTXQR#U#uvGI(coCBt6SSJx%{ zta)q(E5+T}<d>Fkj&I!y%ApyzxSsZWTCD5-O1t7* za37#sbK``TbFPzYTO@e=t>mz;%Xhnd)ZyY6OP3C0pnC2?d>P3&31GZO325|mtQvJ6 zh!LrWUzGtiHK7P?kc4^}-N;gnGQc-5?-b|k$_qBqY(S%kH9stnhMx?O>E)ZosToAd zhBB(h=Z#_!$k4;08{`^Aq|7#0dlyaQ$YjOf+vmDC;c{XUauD z!qVxSlm&{l1oUGqIE&7H?yz<7S&vu~&fYr=}fu^n<+L$geJcvVS<|!wwmQOt_ z_=GGk`<~{rT?6<(ZKGGOuUUT>enr!Z8n_T3ze@X-D2awT$D$*SOgFHDeeJ9ti%~h8w*cG<&G(nxUtPBo>+k!d-qQD z88E3m>?sBj^1$<>DAKQ7g^#k5u~Ae(4H?1mA*-s>8^=bY-b^Q5J?wS(u(^f3;eGnh z2rPbHLk*fR_f9j1Qh;YhxiA;})y`#??}k*aDe_EuhEmjt`VBSMJywy}0eVKmbpgyi zl>vklT3s_(W|7fx+pvY=JR=IuUVEom;Nl9?56*k}KzLMFKY@5s9>%7g`t5}N za-XY^DZ&y@2?bve{E0Nq;6@`DM~3KSS6BN)D#5mqhFnRXY95o)#@7{q)*MlpN{V}C z$!tC&6r^FEy*;vO>ii2YitG=aTJRyl(!gfEkGT#fSzxDd59YZgZZkU<5F*-JK?`Jf^ zw~wm{+9!&DB8FjSf4nc}{ z-$$_jSyo2!^TwLdlpMpwn~kOM7x+ZMj5}JMbl`W|N~O1$D8LH*OEwvqXv>wY#XiZ< z$;$1vH9sxQQd9amjH0aO8)z57@a`qxmWT_PGaeEivYMPSW4O@SYbbH>s%b79Ej#dy zWcy&lTV(2@22YCjxlzlB9PF__n^?UFlK|NVJRzzwh8Xa@W7S;l*L2a)8E0=E^|M{I z2zVleWqP=4_h~h!8Kx%qNCB|<-)@l#t1U}OwP#Tj77WzF#$Ux4di_#fwTtyLzXsbz z%UYlSX}>)g-h2bI5dj#pR=$QF-!37fN8ftS>4R>*v7r*tqJ1kxMZpBW@BEhHUPha` zipN+an-{l2C=uWxO{NeQn@vJW8Z6V@gu9cLtony*~kVW9kL$N9~%I=4^Pe?IPeX^oisuz<~vp|nbr&~Jr%c5~lx zWIQaM-THG%D|4sCfl4(a4xm}%d)KfnrLM+!?E-dt~1M&hffHEFYbp4RS zI|h_!`h&8Dv;B`3#uC;TGtxldNn6p^O5_s`j0tW1Fl5y{)z3lnF58<7<=K%Sl7Wlx zbdTD)NcpGXK{P&G-RHJVwfecq&BvXqkUyjchjsOX*{yZTxz)iWV{>mR7D_aD;+KrK zWTYj^>lmLsEG#Nm_1KUZGbVnuSS+e0ES^x4@$dONEx9lxUNkOAbPzQBllot=Ye z+q!nJGmDa=k1i=~c6)}ImkmHi5cNndF>3u5T>D&vBQw-^g(VJK&zE10^c2z^8WlVH z{WXq?fOAVSN6Bh?>CnOm>XclSY^cNCq!!$8kC_ws^BELgSxvBVAhzUC5&No+*1OI;-B>a>^ z;ig1*Z40@FJtViMDI_jMkS`%A8>^)`3SCI8BbnJwHb)+Q<0AtPccj#(&n*^KN%ka;h6 zE)-yF*hU2EB^Etb(;zg1-h+q(AR7?jS48i_esZFC348@W0%gMN47&^Rn7>!t)fv?d zPEiaYz_ShP7F2w?ZAUzW;|svN<(7EIHC`k<2B~7}j%vS-7zVAU$U$++tC1ZfzU#$L zdG(nl_6_SWcCcO3Ro-dWOQm2%ddpgrQLJw%ZnUfD<^~~m#jJfQr55<_xeC|37(iHZ zi9~be$`L<*a=}}TuAzU^EC)hS@mXp8$ig#i0d5sr{#lFcZ}_5KOJFNl#yu^|5=HW_ z7Bg2cd~0hlUPb{Pv1UB?4i0JJOfw7J7b!qdE^ii`VY>D?{wWvfu9`{phUl3-3xOQO ze708sW7yT}v@@@clP*Lo@aJ9w80jPMOReKICW1%Iydo4}CoS%Ua*LmZ>ye25RD#b* zC>Gu|cP`KSe1p;7$MfxnB=|BFwHZZ%GAD#7Jzd<|}Dc6DH{7T0TC+o|Pq_Hm*>d(r;Br@2ZaQ>fTgEl5gZd0#8W-UBTr--8M0rGQT%qy^I7`W}I!~@x=+~qbTATLAu`&W;_vphCqIA7#fP$_F(4nTm`cdYj^U3|bb#s@t`9vuw|b6=J)d3mh(`51 zeouzE2jrpv1Mi9^S6R&sxb|d?;Rdk4Db1qcDh5yPJ@P3gx_~=Ze+TKq;e7%bJJ4tk z0XnFt$FwR#C^yzw8ZBN1HK3uWsIr_f5cBLlU-txMeD$IsLy_uk zxs55iZtKF-oB>ku9)F2jbj&T3AtTH#4bW#a>*=^bzkWAOB8L~(-0e0>)8ALcDco2g zs<);l=2eqnp$-hKfQG{I@P_B3_9-2=IvR;y*NnK1I+opAd$>xpsq^9CAZGB7=#E~X zVvs2}b$Kq?iux{X?5zDZaS+v1Iekfavm_@@q;C3A6fV_+L)#Kd&)udTmU;a79X~(& zT2RKhtZW)d=;vMbPwYA#a-yU1TJIh4dX3=tf5{dz)eZ4$QspwE<7U)gw&#MaWv@ z`4RCR(Nw{Xq3rT5=A}dh!{zI}AL$hVwxB-HXzp_nf;~aJW8GDVIdk<3B`iP-s3L!SN#Xrf}h5gRzq6^R5+i3R0Cm9~k^J(wh?v-R!5NuAhgqH7hS|k(x6;Y7Voc_J zt%;O$Dx2XmS>b>N#+aJCZHA9aKKPjx}|mz++94DcQYd+)rhIN=Ck>Sh+TnfprM*_nUb~ImG7`vb#t%9cVEvt_TWRPhY${MBo<(}u znTPls^M#C0DLDRBMA6RNTmhLtZQ#OVw?&UqcDGJq#ZQF(k87Wiv2LQSR}h8wCA0VI z1&DTjCsMMNX8SF0LYnSX(0ltChe8StCi)j8>i$2ZuOr<#(yFM9jDLu5WfM89kTF4C z@bCoshV^j={yHZ24Lv5;|Aj~1ERq)%udOo34r{+Yn(f2SeJYmj|e`0Ih3hGyS1 zG=WHyn1o@n$tg@MXfKvkf!%X=-f+rUH>ko%JlqE#uVtvnVAtl>DFJB$FQQO=;*5lz zYU)&u_qTV4ytLa@r9g^E%jC-V&&;;VrFBH>RCZGp(7=fi?8r$KZPi-mHoez&UQlo$ zI+JOE>yPkoGy0%Y?jDNl=W6Y11_mp4BWKH13)2^(xa~8}K(Zfu%Qt01HM$-DCtV)YT3WaxE{6))obc4#6`V)A#_9tFpVs4UGYvX&}GF{@~(`W1N zNBdhs+@v30(8wx470m?Y^7RLniYSDb5;Q)%yRYPJ zYg=>?V^`fxMwk()45|d}LFJn;q1x4CiqyW!nJlL>XJsrR}Me@wUYkI4LFD7*%~DQtHN$*=1!zsv=ZfPxpxP{ zg;3U6WKF5=)0`NZf5V@DtR(la&hBhGbI+_~`QgLtZiI$%xTVNX(gIwsTp9u-eMwbJLx$1Aa?f-r0D9eAolp>mw zq4AuVMkHc5XG5?4RnL`{Md+`VTN^P0FPE>33)lbXznlZF_C_oyEICjn9T@ugS zXD5+xAC$WNi{;jT0+Qb^kW6}`o81}Py6nII3O%p9u$4Kf+H}XXr_pu@^oDBxzbQ`N zWxN0H(&kw{p7gU6t-k-Ro)X^z_}`^Rf-zrz{ri8tz4!c<-xV{1jxKPUJ(gD5U3b z>x))eE`pk0+t;)hE*qKYR>VKj1X5K0{ac16S<#qA8B$a6{l_Ux4}b3=PqSsFGFGIa zzs4|630kcor3ljWRb#Vk8*htr{hQPfE--`pAm3;|ls!ir0A87HSLqqsu(XU5Dg}XJ z_JS%J{r2QfM}5#}!NK&e-x#cINFzP ziV`Z>Lf>`@nZNMzW8ZsUel{5oSxy-_oGQ+D(q#?)JEODmY{-Duqte3o^%i$d%;g(t z^6k+~&uWimVsZpE zms`e*jEe`RkCu9KUCXK6=B5UDK(q`n9nQitO$x-(f=)y2MyHDqyDG(jyl{qPzG}o& zQp3Jp`5C;YVaa?>kgVy}XAzUX#@TsZuG(ejTIZQ=cO+k3sce18lea8FxMBt1A6wzTuzv0><{Y*)x6!@$ zfAc3fDL?!ZwySH<65s8F&U(hXLXDP`B_=n4SK-TjnP)d&s+i|=-9B_DRB)$n?Rk0Q z{ZHzLQtDQk?s|QrZq*krtnmr#l~lw~RjA+_^jAjktb3$zw^P7iC9-V|=|YZ1stp~_ zKm6zBsX-jAqM|(a?p4dgF)yrRq{@t=CfRTKb%LIdiU##8B2%>sx~E^tkv?5~$>N|P z-2L3V?(*L>UACvBc>8be%JCP0CiDC1?E)v;-n-8$l-cCQWNekp%L)MnHdG&b z0EK2cbo{jIKlDleoEemwi4gmrjLvskXCH6O%6b>owJbi&x&^jYQK*nwXf}OasSgU* znO~Q1k*crZd?@`iU82$~Ngn!N;fSFn#ly0FOoBytUFlS+yJ>&5l{)ZwJ1W-Qvm=qc zEag3W+kH^YAO9AAvLueR{(9c!J6&qDg}aCsXDyQQUy?8>fBhEwXf;tK z+p#aCs?hF4C9ODmHXr=s6zhyAyZ)rsK!@%R*Fcf68JpSR$;1Ec zr8`Mqz8FmwtK>0smwU&HxzbgI#hwjX_vU$zyt<0WGY|7g1RrJ`NH%9;B2PQb2qDQf zvlz&0oeMgTgfszrh|l98FWOZl?D9J$zt{IA>D`5&y+^{*m&bm)ZO*^v*k=dpe81{_ zIp?Ki;-MFmbpY>?1<@by8rCG)PFDX+^PS9myim##XX|^L$sH$dd(9@gw0AiccyXL1 z{r$tecbLJS6^Y4P?nlxFXQK-V{3%^3uJ1#w?SFTE%cQ7!vii>`X%@ji^I6~WM#J0EwS{vYFZ4Z@Cz$Fs zNyl|GVoGY}1OJJpT$@F`fYa}d%06Q9bU#m6Ok2(Q-VKq%u|W>YRi@f2B>$Dbv3&(( z!>M-^tQmfqF54i%BG&B~s!s>FHeKZVn^p{;wA%Iro2%_89Ulf~mc&!P7JX+{D7`Y- z%=lV1B>MbSY0aGb&Z{Pd(Lf}YlmDr)yya3(x*RJ>lK-m6`+sMON)fe z1fJ*R4zBIH?Cc#2+lF&P29dSclJ4e(lx~AEEq@1yw>Xn1=HGLCeGI$%;^-$t<4YLe zB_n}YNK^!i?6rmj$bW_Y!PRZiaEuwth{$>@j4J+U5va zaCZ)To_Wss%b1g4A{UPORQgaEc%ez9JXPThUD8AHkEF(>8%%qsDcM zrf(Z-ue>*<*cu1}&*VBqGaS}h&{UVgC@D-b&fWmdPDpqMoMa`(K9^{O^?m59Qu}tX zPJYy3x|41FAf`#UKfhdbUogMAq0O0V>Kd6o3EzwV(-( z$r93=5kG#~sH(1H{08y)QM8yHn-JLAExHq#$3{g~?{|BG8#I0x_r>3$M%1+Kc0B+= zefM0UR_R;k?;AqT7Eg=1OI971q0>z9t2UDO$BV9x&u9!mybsJG$Gq)#o}G^%au}<0 zlr}R5psR|-oBsOEPK4GX7=YS#w&s1ti%NlxFL_H)hn9-|^k|MUkg{BPgRmqe0Kcth z*>Y-HZhedOH{TjWCDIfDp&~n2_2I8}eKgo)Hust7oK>QR=?L{a`4>A6^#{8sGj=vTdBuaW|^ zs!y^{Qps+9z10!ERw*VbTJC|=OXMfA-Nf^3nB@zcP?9D-aJ45@zTkk1{ZgJ@l37GC zADyu|9qRBwxbcS~xN*k^EnZ_;Jfj4V?~Ft`$Inl9Mcy~^>`uz5Qwnq{$Gz+lvWM$R zy{mQWU28tQ&9J^d@*Q0i zcMX>St69$D!G|5}5a-BjfL!jld+r%Jt_&n#VHbSCnD%X+q2Uh=Zk%8zc zW#PN_^V8RT3KWGG0Q7{-ozjodzw>JZ=uFHh-Qlt6#BVMAnPPB#%^gy|W$h8lo9ixv z5vjX#gccbvD4g&nPTK$w5)2rX#-q#Xzfo^@G94d-HCrXW_9EjdwwJm1HkJ!|MpC>a zJz^)gA5BBP#F-AYgG(#Lm@WBTyEk=sMpd{iy{I7W=V2QV)4pk!4&E~)>o66uZq?q^ zPN+<|-2bA?3(5#e9>FWaB*E#KFVik<_87VJ*lGT06Mg$_@3X$Nzjo`zl@ou!==7O4 z|4A;&`VpD;fW*Fc85zRSfX7tiJ|oR-`ThE+^?9$jSsN3g4Ot~u-*@?0RSwnd7oyWm zU`VzFMWqQk{~8f~h-6>?TPTcv0K1<8C)|y#qYI6wZuO7Q5Ib^mqW|7w-z=@DO3f^7^J_Vs+};Z_GF%k1<- zpxOd1xBgA@>MQ7|U3(kka?HJ6&TED{5Or?S{IPl|&hK*7vJ~wK3s+|4Zae-UBK@^{ z=sj)ZCngcyyVP=%=XSHD*0WJR&X$(p(Bn+W52K-mXYRtwqDrl2kGLqXsJoc4JRAGH zgTb@8s16@)+hrS_8YKCow7+e_;4x^TZcy=#zel<7xg2k$DUYA@3hX|+n%6klkmv1z zky}(k`(uizLDxRkT$$7F8I`MLk3UTT#rKlMqNJ*nkr7SpBp*i-9y5-Hms+;9Z$wAX zv{42m`-y^AQC|7yQ`hegu8>{?u8LQ*f@iQX7^m&0$>8{{Y~~$2w1oZBlbXpc_$bUr zvN2zIGIXqo8ZKJPUG0wkBElE#6GjMA0YqcjDmeKy)l0aMIBP{4eCG{8w8~ z3~65ombV%!$kH)LYHl(7h1MNi_@;5U@N zNXg#65z89NA!HF*BNLe1I({z7Y>3IC+_BpI=p|=hW~KipOT#~lC9|+!vZQ64)j51%rj+8I>RWRU>?1pZPx$n-Y9Yj{9<1v2!o+v*} zJ*v~_mAUxE-YLUrma5tUB1EL(rv3ztskriKx3%w3MD#q17kDUb#2y%Kc zz25G52d>=YQGj{E(|Kci?DE2}rR5Iw_QWbB@Q4q2aqffy@q&fLsLpm${V`4gezChP z9ZMgat&cNn)a>!;Z_y{Yzf~$|H_MG!Emw}IzI@?aLPCnB;_VC})Dy=;Ehd2!kuz)U zxv98vhi5a~mq9S=4aI?tjfIo8MvWzLnB;g@lkfucJQS!11YaI zD*c|u6D$j)!)0=jjUWHtrIj7;A3Lt9m(N~()t7#D@`J#{Z^+;7&H#-F??!d3h>|Z9 zlQ4l_T4Boqd6SEur&(l+K!%UBShslM!&2Qo+8LKgi^6u3^CVA`I?b7}zOwgjM0z8$~zk*ryc;= z??!F}(v6S@OpHz0;BfvZyE_m3t`Z~u^U@{zLQGvZTc6>As0npYhDM^F=r$a8#F;lgwhiQmbe6MKBeLDiQ{ZxvU}<*JYa1dg5Kfgg^4j$KR7 z7TDNm&q$5Y@osCzw)?Kp{_y@gB))Z(Vk120D6+yq2o_!Nuw{nZt?o)sAXl1o$ifF7 zlxYOl1b#$zqP6iV`)#V&va*}JMk4!( z+-hlj6yRA7TpnLvZY=E1wAfzBIqr6;3{;WuX&+jdfIOE&{l55e9<5L=nW4utvU0p- z2#q+FS)HBcLA$VJ#BY05zR-|_l0@wk1we<*ZnV1ph6qExBT#mgYhSCTu$`_T3mPhC zas94-2&>x`{Nh3P8d;)SAFjThh|KCyM||Lx zDz1g_`$Nn9R)|tfMGbEyfzMVMEIc_6`iV;W&A=d@NnEqN5_{+GS-i-8E7lPAf=3cz zc$V+s3(=-F`{Te5ceqhoh;><^d-)Mj?ekZ$f`G1jWF7&n=vQuX4{~-_;QqeGBX(=aY0re`y}_ zvH@hraN5zNF$2u1##+$glWFMsF?x!taKfI&Q0@-TIwoex0z!FI5c~;*(y!|S#8BfG zk~y>Vxl7lH5py_|_UDJfd9#ck^a4W44}G8WS`mfrSA}hL2n;3TLb)no zDxSC+92t%xk3M#>{zPEWr4Z#J4}HiBc#(NWVV){T5>Ni2?@JAb1ac z*w9BkE6OpYq>ga>XBLA(vhgzTN94Iz5P2I~SQO#wb_xzI0G06Nmtc=FE>HLwhGO7_ zQFgx~vYp(uXK^%Lx^jQ<-gg9hvNCD$A41;AGL1Cy7ZbiD7l%*i1;FYs<^B4jk{%^2 z4ghQuv;z^!jQgRpqV;SqWyw`y$HuMDA%~IVFMqM!48h(uoUsqWcA(n?Qol7v$GIg5 z$sW_iKj!dh?FSKZy^3y|lPeFh9dq3;e!k|+*U?AVCnvA?Z!sU~V@(j;pqk$fObZY(%sR^1dvDAbxz z-cGlG<2@hQVs2m66dFootodgsnSkvM2{vn+S048~MTISXll55hh31G4CE$|&NOw;ymvI_DJ1ZYl8gm%Q~A*y-djrnC!ZQo5D^GsR;es};M3us8_LreV96 zN~ib6Tw30`h7Qv0hH3VgG?ZfXn~yEO^X*v%*6h-pLt4`^D$f`Ju83&1oc8x}gX8Y+ zL<_&)bAbiN_k+#RB}q6s=!0b%5x|LxwAZSEETwJ=KMKcg+U|WX0QsOw9@$EgZX!Xl zRz`rV^OyDm-q=b!%(>aq9d437|6?e>D+PJu>*<#Tb=XZ0pp$blIqkCvuMV%wsza2%N_5#<0fEWAH&UdDP>R@41LQ8q%d z>>Ws^cNBKg97Z^Ql49MXmPy?;le8Rx^TxdNA3|{9X0l8MmG-UrT;CF5z`Ocq8Ginl ziGQoJEzS=TR4lJvkm0-LK9p-qpG7*iTjPjM9yRMt3`r$le``5vaNZ zL!Q@yaM@jXR3ddE??TJ(LfiS~ff(ShlZqr??>}VRi}ie{L?ip-amdy;Bl@dGn1L^z zEU=xVzyIC9TqPD`8uFXIaS?Rq)R-m%0<#{8MC91gh*FmX>4QI`zu8zjeBg;Vm~mk0 ze3dq-YwmYy%$M7(us?)kl>S;%yHgBL{)Tn-u24$hYyXbS?vEQEK4=m4wH(a<+q^j7 zaMt`_?BS}K6(0L}7XK&oM0}5^`GC)YX5(^o(~<|PkGiBfVBl;055G%>gX90;WS-6` z?bG7ZUbgJ$8X+sJ$Bu_A+L{P@++--CoYyLq0jz(4+y#j|uyR4f?xIVHskek}^0BPC zCV>jrGAGg%D$*V6-<5tZz4JAM*1nAw~ zjT#@u(8lG}UiYViP!>}UDCZK^#Ql#$D9NKb^L+Pzt&hX89;cQWt2fPmuv z-;RVbGsj>RWQTNc1qU7AqsEeAjO+y(J=yRPqL->qmo3U{ca3wzzZTL@dNUEKT z)ApqUwNZ)u$zfskoVFr!pK;)^p%qD{kDThZmM5|irJ)#1Rr4cfpKh%#07@jWJdh;I zwB+`eT-)q;41(&91XXxE6XQCej$FGXXBfCLd}TQ2q~k4XreDYtv(_Xm-CjfaQ&9=3 z(j5qLQf11E3bvIpDv(o`1-#lZ*n1*2##ySIssMqVb=*yK7sR_pAQ9h7ZUZDlh@GY(qxM9J#Spv|vDPGL5g{03~U-H=qq_f#U|bge>j zWraszV)A$T&yR!PlcQ4BXfe@bGtc?je|sTw?^huIRP>~g`WB-YD~CA~c%c)JyOFqCc=7s0oycvvyaCZ81`BTN7%^=RVu;t6~v6GZ&#q>6nfy&7uiK z_s7z!c0&v`N*>B5VSZ#x7=ek-Zest~QR?}6w-t#0OtXp=rJd71K4^B^Vjdyx^ka^X zTmdFiXs)0BroZE%^lDefB@dRN{D0be@2{q|t#6#>0W83=A=0Hu7nDu_rFZGQC?L|M zlTZbvNbgEjdhekJq<4@KN&-Y_0YX3$di`$BIrqNbXWaY#1@FU8n~^d0-gE7>)|~TG z_HO&gW`&umrYO9`*-Ss1v!`x48t;us-9VbB*XDLOpSp7aoiPx%hk=Csl*f!oM zinlSMNl6)o5RYVDJoy%?a*vIm-G~w4N zKaGDp54W0+cOdFxDn&f%qu3+L*^X-v z!pX?X#;dPYSwUV=>sKDqrCCLnXXjb#OB97utiF;u!%jtmhwYtCo#X06#Z!6@N%cND zVm0;|{A&s#{c8J#w*0N~?~#Y|s-4XrZCnK^iMcJFlR3{1{djSuxo$%dp^(QJF*x~< z&py$uhS#oxpNP?c$hOB`*%_~Z9S>Q6o`K^%`2I|~$b@zQ#iC(8E5S-|$FWT&QD?>EGe-kF69$Hbm~vU(LPMFcujrQUV^(@#PD zdfyLzN0zt6dJP2+gx!k#G)c`n!)0FV)BOtgW%!vR7|fz1OL?D2`X+VPig)XnoW=N! zo$KWFuODo6+w#x3j*nmJAJn-I(C56c|I~PRKog49(iJ76B;T$2u|R9e*R*9(Wur&B zdBvnPZSs%3{+r`-&>e)>?ADcvo_)IR`;C;O%WD1>zhi^So8HK6h^qHrd>L?l+4Pz6 z^N1RbeL0#Tea&o9(+a6PB(r=b-?bZ3U)_-ne|nZ!my!zn#q3s`4AuTk6D=p@bu%88 z8?$o@{K2f-$YuDOu)}MJo%IZ9=CbiOBYU)EGrEo+6I#4{pV|hsH`KhjjZ(ilIg5Oz z{m1K4UrYHPMx~BOm2Jwv$`6Bo3)&BpXL%xDH_gVS`C{(t)AzJIes%%jZfew`t8cr<4LI zm5P4K7GviC)%IQu-OVdz?AURHORr|76nTH~&1}+bqIq&z(r|=O&FZG?QASqSbqk>i zJ=rb8Ismx4+p&24h7{B6yW~$as3P%~E?0NP4K9b=TD32n?Nu$^O|a+D&%;@+YLl>$ zN*_OWwtB_Lqw*_2D8wLdlb_$}>bbLVu!eL0u_wg!REhG1B_)J`%TLI5xIfN@c00KI zp3qUyyG%QqA@j;j-i5zO=Lf1D8xt07H3%cZQYPNde);`nalMcyow}T`Rv%CGtat{_ zVA#F;elk_R07lwEm%gZBKc25O`xbX(^W<;Hheb4H{G|&7(%xz?!~o%8q6|)uA`X4n z7wJ)aGBotpVA0qWzg|qvjT%%Np#Sil;i^&FtO@YH&Vzh+-?6-?gY*NY%#_(bqy$*i zf02onJozdqXTJarJyiYLWJtsCeK?oFvx(y0dd73Cy9d2O3Xr@aV=w*qO_|wu0OlSD z+iZbu%&K33>4E)l0N5RPWoTd#SUTT}`SabviNlRQECh|)z8g!M=c&Ri%z+B<-3-iu z;hws2T|EEqkm@CR21r)ELqq@W`~HwtFosbCXX-t)P}8EhW;ZANHp{4iFX2y7oQHT} zEL~s`n%vtaMfDElFl@SLcwYNC+17KE;>XJ727B|jbD^b{4)V98m`rW+_uMXa$D04v zmpn;H(9&wZJg?plRo7a4X{r`=1d!4!3l71z`V{tl(gC+M|j*+AV@cL4h zHM^hwYKJP3YHCahKgjgtr~Y`2nXTBd-C=t6L-f7T&dG1KsS8fJZ<>FSJu)1h1Rlgf ziI_YW@|`X)SLG?LqBWkYu_*r6zCzOL21sS-kKNF@SK(5vi3O}0TlbU(-&!_i8TaP? z^c%PFc~(>`!+t9XWBo*%`KhX8D_*`Suro^RQH2`KvilW~;;XXv{|i%!m@mo9X*++e zd4HR~M7MGkrLZzzpmP{B27QZEuEYAp1kxV+9?-6!0?un0<@HzTzQxTj#FAm8@n;8=~*#MD;X?+c^uRo;zfZ!M>r`E`G> zB8Hk~3H$TLHmv99rl%ur!Qdds$8?J@!ta#_Im7j_#$B(#hi9!Z3%;lRQY4RkUX}&k z4={eMZNvha$p33WdWha2d;xz5C7$>}n_vcQJ_|&hUdDk~4P_CTq#)MyW{Kashi{H@uT0q9fLKt99Szv$JIh*hFIL z=%X$4{=QwniMKf;3lK7+m46!COm(^0*Y?G|kS06#6W_e-1Jw=f{0p-wF(%y=&IxqT zUA?MbHRQX7x1KEdHFQ36iGNDE-y|YUuGA;-Jnmu(6G-j7R(>O`@G|aKjzZ>1uK>^Bx(C~KQDn#2)ERZ*)u zkZ{p!;4RFe&PC8Lm=zV zx1a78t{ebeaX9aoQ@WJ2^?*vDY6}p= zZDhZ?APu!~3OERI9sc*`!lU+BlPjIapy|aEqTEvR(hAev>%HNNH%${6g613SQl;sS zpG1`ykBH;sMP_3o)<5F~_9nPjgmbpP683MIJmMqZG%K-=&q}ClagQlVOSa3?`Bkd4 zO;~KqsUzuw*;jx(&Qu05y`0I)1XQl%y9ks;JThM;^>%c6CT~t|t~^8mJRbPY;)|61 znG%?8n3hJ${`1=(jcsMQ6)oVe%QR0%80o6|bLvf9Ydt;unn9Ua5ec3@%b8lXg}9UZmXW`rr{2-hpd9 zscvP`3k1_F=XB|*oV%(^-pvfW8|hnXSO`n$xS5U(Ha#xueZ~m@qh?Rgq_XpESc$v)vid zTkN;wkul}O6OWV|MWlS2Tqiz;1WY=q%$zA($^T&?_rUd`$hx1vDNYs0NPljXKjwdw z0oi>y4zs~?=v|cx=6@3XS!Q{%Qp)(?G>HMcrYBa@L!aTJ{&y)!UxBFJp*LdM-VOY* z*p)rBcg5gR1mHqU#i=_!NQ@;J;a70ZmHs%F%e(?Wf=}5WlJ6BC!}R97B6Z~wsOiCw zJ^IJKCVTETY}tBK$(yt%W?JgQOG7E|LLFAZfhKS?RHXCnK&fTNw559$jGWq>$w_N$ z`F_RIYPByrfA60KmZg?WaDc3+-dTWMoRP^i?{D6Ick|}ti&!!uGVc?7SXOf11MmHN z7j=qReuhardfLyN7nQbKfCh^&o5pd5nCt~b;BMM{0B*Ig}h!Qpr1gi6e5i2Ae+J(m~rxLjgS0Q@f}FT8@#2OXpl7l)Q&MX{Dqya6j>`J z?UIo|Zs)q$q@v7gTW0=|eEETSP{W(ov{obnF)wdrccM3^Tv$U$9doJWEVgfTBA*Gg z-r%-*6@IUm#atApa(Mv?lS*6&hES?Ca3s$oPEJv}!2|q{MqcrhdmCUtfUmXPxhg zuCxFv06225X5J3lQg;V#uP=mq5pi%1ak*%+7YlQf*chwF*_MIy%lg6b&DkaM2e_gMHpEJg){jN)C|0e*Py^nI8RnMUvkrbrv4=&>=b?Du=Y15mwb!lH9L{61T{_+k(vk>asRQ8>4n5yzi|Z`wg6JYKu+IKC(K`M9frRW zK-C&=xN!HS5AAk7Z_Qh)_uUr3T?wF$h1FM{$CCHn*aTBq0&h?3wjKD zp^~S)eF|Ccs39_IW@3XD@z_8ngQ@-s);DU+a&+grU$qFer#7e@T~E4!e80+Lmka16 zX!#84w}?v?Co)HrbX-Z8K6}LMiAc>c#>IeHMTPaNwCxO6uT$#&_P1M|lpFHmjnb7v?yKa}dmLacGr6ZVX)_Lf} zv;F14P2@BZkTko^SeW-*$)Z2;>8Y3se61|&_Tp`j{%pm_UWEXgel zp}#=b56DwulX$dVr~Umjbm<7mQjqeoz4=!ByM@d}qTVttVklX4!7C2FDVhzkfC5=< zPX#T*j;7t+R(Z=}J}P5Q#N^W6MIWnYdfLV6j|A?|gcNfwWgSo6Jk2e77sv3yGQaW_ ztQoaaWBZTjx3m=$@wckkdG2e4qYt;^+oAv*ZBA)dnb4=}pCaLv<(-!u5yb(;>P(*GsEy z@(KqIC%(T)pDNBbmnr&3(e7ZgE%vvf-4!Urcdnp?OtJq1E!mLRyOA|6yPmqw?8_#&fR=$Ls}!fZC1_N&R|XT;$wW(Lz7ee=Z9|4bYGwXA;x|BCYR(OdBPWIvq2ORCQOonoE*Bx_r&Z_(GE#G7-?*A{~Uij!jj z6=`4PVVf8$`m<%u$&sB^^m#m2cKNlqjJS8m9}Ei@H;YK2Osw#Fb<`E-p$a1 ziPURLRe*io&oN){&M5Z1_2ORwIQWK z*~=6??l+(Ky`}88ZWwedT6*jBnXwuR6`=!}xgDPgaqh00JuQ6pBYO7l@!x;(FQNB? zYfc8%j8XtIn~QTNW$rKCDeSaRcbK``F({mI&4il=z$(lJ$jNs|r;KmP6g?vI%9)GK zd!aI@4nx)-90tp5Gj+cPJs7v2DSB`COenD5weiM5{Yt$Z4)$!Dy889TLL?*$AMD&N z^&c|hsNk3wKC$!Qip+~2fCv2A)x~o=E!_K}?gI3IeO_3trNhT-j6Z_-=mU6-a=NF2 zj3Wuj^hi21p0#Lov$J(yHp}>Bul)rlUERU1juhKQu*&Mtxu&zJLnZ1bc+CujJZExC^@TwF|9202pijq$Z0XQ{V?)eqh-EJ-kT8#RwK|EDjdv{k|; zKlO5z8+DyF$-D0OAYDFs(5RCeXkGhxQbQZEz1`+2JO3ryr;Aq@DvNEVH^|zrq(VqbEAQZ3|_~bic zF`{XP?Az)M*ybl0>YpaANsQl8p)EdE98{53d0qR?e1xQ=6|Cpg=)R4KOBiS1q zu@WT)!MZD@Hs{T+eoJH8y9g5u?{ zP`DB+-dSD>_!zwjs9OK^z%qi1EMS{8_D1(&=?q&st5msa_Hmo507Pv+3u7dy4+V3v zoE`+IIwJ1oBdrPlV=n^CD3%i7AyV(1+j}r3U2BKoptjeBW7|N0Z(lZyw)e;@qDqma zq5qzXAIHlyuq|5z7yZkT_aD?N%h#J5f-C)``=kEsit;a%t*YLex+~x%({9j{1!D6w z;rVZdt8JEVYK~jd`eRMgMGuPmr&0A%JYjTl#&Z>IztVDD1q4u?!k$)u+V!6aQ~0>f z@uj!*+=nA+m-KfZU*!TWkgm*!x|Lk0JHZfbRA31SvhxFeE?1#!99tRw-=d`Ey}h&) zAkoiEnl*VR(uI08Vd#?*r%8sbpCeDGIrK;LGqQhAo4d&=w;nz#2i!jvndw3)?qnXm z`Hu_&107*o%%cNEOc-VV%YnO&<%uJOg;D-SPLGeJ9GPLVob)J-wXo z&6Q+HjtgHedI79R@rk2ipo93mxlRd@-*Ib8i|FtMVM)s^evJ+4!6aA<&k8YRZO-QlisX% zf3#ag;X7i6JD zk{;yrUE%ifQ22IvTA(+kambd8`O{ud{(qZAdsN+hz={V46dtr+x#M!RfqB8`WX7L1 zQ|csfnknCy#5bTAZHLsuXC;^U&JsP(heF}C zkb1keHfK23WmIf52mu&-1YY$M!~;&DUDNGHyPIjPlj0p;!~^^rZ894@J0FaQ{PZT&E>60N^HdAVD!h^wyoRVZVP+s&s+PqXH~=t%r!%lj|BE~4;S+9?oy z+Ow17aC*-is-T4s#3J2d-I?76`~PB_3D|RxhqpE`N4Rx$?17B z_O1vF(ZKKhd?rR@57@xBubHOSl+hmDYNQNG_@t<7=%XahwQI^So5NO+{UFWQ{&Sex zmlma@f+cQ{K=RD{#!&*?l6FOf5c5zYJL0%@lGdA3;|sBt@QeRNmu2GX((~>?fD~` z_z-o1^_vM}bi-DnF4hzSa7Qh4=UHL&-m#M!fr!brLOw9}!TxJ2`L^Zb)VA?5pUHoY z1DGqGI2w*g?itUV;aV(CI`5IjzTSzB63^rL$*vl~5IZWff)S$(e1L=+WIr?UA|8?s)uKJI$<&{uZXE_mf{Ap5H&$Xt&^B z$0DnP^pp!V;K^&!9xWz;yH5Om+nR}2Q9_PQZyZYQ=K_oQr|ffe6H;jLmPIsAg-5jfmN1wJm+s+47I)ZeKCppK0oad(R*Avv{SvF`L>E9`ikj zh`q>Wp|CiapXN=M%0vwxA?KDU+7KPtqWjmzOkJtC_}b%9U4JSruIS)V*BzhyUnY;h zYWJ^G$GO(y{q%||)QqzS43l-a?A?N~%|86lx9!sxseuF=bH-%i!DnCv_bfr1W?{u_papUON<(ly4d#8}<>PM*EGgZ{f zgE`ymF+JbJ*CD-ar`W@f%`1)!OP_(kYPFkmXmFl$+Xljy_Wa1bm3k-Ipe%A$RDkuN z#ejF_j^AX!=_=nE%|^cqb9;p92$ZsEO=8=>-Phke;0YIBpz9fN400EyNPm2B>_Q@( z=Gxx&x`jtr*RP(Sakv@x-azuVd%FjMlbb@RXO#nY&giz_5EyU-+LFF}PeAZJ>%jk% zvFdWhe80e;O!Ub;T4@}N(PK5F&9z*%(zE+4jF5mJHOZ>paSjKaG{-rLJZ`1y!JD6C z%;T3aq>O&Q&Ct?h1O$zF3$BxE>J~no{2534?$YPK?va+hUY2V^ab)>TG*o*0NY=g) zSbIzfoQ{uIH66%BKVF3~o@~rydln>4ideS(DtqcZ|GQVD@#lA9%A@|@{ubUB#K8Av zRFMI)XY&E4Sl^D!ZMU#k*a@$#pML-q_d%P_h>7{}NXz{BXHBzLllR(^cD&EB2nl`) zE_2CBI%WGG93P$^q97Lwu@VPeW0Ke;_dv9IVIa-#@>@PbrN>+7E=UH!AG1RXPsxufMQr0Pdv`58kC#_Gd6vOUf3st zA$~b296W)-^?`oj#2i=y`lT~h#1K2S1+yvJ>KA@E$^m?%C$*w0^hh5{vcw6$9fG9X zEre5?wF`H9N8=AjR;0s3UqxlpEHU;TT$c1ame-V76#R0S#4IIL@4~ng1M7md~PYVnkM;?<95d54-PG>)mTUJRt=yK4Cf$um?+=&a^ei|J^hr9!e zD&b%fvwgGK5_Z%piv5*%z$$Hby5pL~gAeSjKh<0_8R0E~jc&+Fx`Z>9_lSv$&@Ab} zQxfq$8V{DltyXr*cC=*A)~I4>mab#%JcpPy%Jn~`#Aj~ws{yNoS@ui;I&i*f%@93S|)Jjx0^i0#zf<2ppLx%j*+IMt{~UL7=n8 zwu;`a{+CpzqqO|NTT^JyU*8uo2sp)wdr*KmRO?ez&>rz_G4ys^dMq|VBG$=)jcHub zAi#T~-Y(p|_n-lu!^6#Su`@_q8vS!`W@op+6~cJ_Ss@`t?&6}5*^PMMEHggiA|r}S zJv(t_HO>`M9}*~)o~6_|g$hjgT6excmHlNT)ve034UCvpT}bd-5yRUA+Ju|L;t8q< zC36vfT-?HwUq(8cM$(V%eHd9tW*9*g*FPPhxL8u+Uq(^vt{szDRvy$saZDGHa1A=m z9sdSn^6A8|quJSj4k`mW^nxBy0$xlI3#vUpPUH6b;e8Y)!hI@p>^6&1HNQc!XBetL zZEPMMsvrX)N>F-{^k-O9T^=1zgxoX>$FMEy;|T3F47rZ@D>vssFkEHY`DJ9@X8y7y zh;SqO`J!Td4-6}2h5WL--0rA}Y@z_4Z8O*FV}b?`e}7|UkQgQY{zp~(ow%LS9ydN` zq6ujZaxzCrPT9c@cMYl?WK{QoaYkTuY2pv}3&jdUIF^f0*-op#-%{$Ev9h+s+OqAW zej@SV@TrSzhHy8`9?WS}=0(x}jL;~pu5g~yiRRZj1ue48c!<9}!^wTrs%etSg&qxF z5H06mngqU7;W!ko!#H8SN&wj!JdK*Iw>KjHtgu!9`*)5QDExI zw*NiQv^?r(sy&$hbt(cgqido+cx?hXT@IKW$`Gf6qb-b7)|qb9Yh^nUB?}MAv!A31 zR!A51xS1ObQBoDugs6eMrdcrfF!-WFAeOOMUen5J&nQ;5G}(S-M^O)BH0n zKhEP2#ELa}T!``9+I4`pr4J`Ae!qapOXc!hGRr#BE=9woCih6TpLSCkn9S6M9&Kcf z87ht0YP=qCyK_A640WXVn(mBoqv++7^{Yi(9!JH=Y05|$U=x}_+EPi_k*SK?agGj8 zd6E51j72@dn1{7?rE~g`w@Vn5L*1Me0ySGyWZbrgbftIw%hz}C9HqRK*@Y8h4#zN8 zh^9;pY%7Kl>2_y`4>?_DGsUibBD~;yc%;OOIwX9@Db|Dt6(voG&nnz{J2CB>S+lz> zk()vFnRq{tjh5vS}*;UrY*e$IeoQ^9#* z?}1(3A3l(|0aj%^`1VFeB$z0{ye5gn+IB|uR*n0}6aTe=y1ly9omdLjm%NPU>+#la zg~VK+PU@s)98RYu8*i;=k3R#v>35bE|eYE2xh2M!_V%;A4pnjNfCc6$z@bz$WO(YIS{&hf@ve9V=6V`CL9nJN4{564I7x>W@n23GuAOu+kSu7ShC$)Ws8e&pdQSXuVtq>FiBo2 zXI>g>ZJiyz!ktME=C@NV)ZQb>t zaU{CCVGmkTjEzn7kb}(Nk6>^wI%TC%8YF|{=pQ+5#V`#3ebNEZSp=%L(FL9#SFH-69MBQqH8#V^3A#2}0ryhNydfO;;_#LOxj-PP1 z2T~AIgPVGUBhh-^wgg55+_N9!p(_qNg6rp`I;O3{r$bJP%okcr{7_H zc5|olW z=BR8*D#ONpj=)I2BeJqL*)%%yHh|w#% zTm8PcIPK7~%U@8@c2?59c;erSDcWUrL<)wv3Fu1;wWY2|VZW(DC=J&Z(spD`2t4Sk~8IKQE{?+YV40G`*OfBDeBQC}XvZ z7c6F_HR@Yuvj)xe<8bB^(~6mV_CyJ+T&t$aADaD8U(m~m!ax`ElE1j;xhKVZZZ=cq ztZ==&0^14sG#bA2RUGZUhkphnFXBll3&!ex@|knX7Dl>4we>CIB%W2ekO0d-HK-gj zmXmVmUEqZVSSul|!orhFfcMVng6oMSVMG$aimO6kLO<)DgUtt?o3aPW8E!;&d3BFi z{R|$aAQ}lj3nA-JVvola)zD@DzU~l8t;o zNl61Y^xRp2!|k^3;Sc+<5#{(}i3kM+!35U5_>cYYN)7tLYy&J42pzb`aKV>LQD{J$ zJb`}CNnTwU)blq! zde~VWcsA9%Y|ei=`;iA57ZdY*L~3R*0&RD!!LJgx=?d->#EH zXF?l4yjy7*+KbY)K6kINQJJf0712M9(l*HKE8g99OnGZD)XSfZfCHYV}(kQMARDC#dHLboIxdSnenue>P%n{VfPWE#&~ z3XABC_(wg8&~!dQxWuISvaQm^KP_2k6ar#_*cvtUy-c-!e5f-hj)vA$fG;N5@hb(T zE=31lRu*-@D5C~eOBk<+>47qVp=6YGn#4fWZ`bMoco?>OnQA5cOq7%vX{^e0n%jGq3z}O z$(2;HYckUheivf>;S@_PSg0hSa>P{Y%BC{hyfRg9i){!1rKt60t-e1Zd zyFAZqF6A(UyYLMMJY~vEYitG_dL{vyDhWZ>-L_o0^!ORo7YTW!1bkC%aQjb_wTHEE zuPQtVE9k>vxqe!5CPp(FFAtf>(!+{fdWR!pu&(LMS*BRAxs63fK_^~s`X%bpa!va$ zI@0uWS_R!q5@UBl8=pgGMo}9PY9Nn6iUUUUn8t}Ke3}JC*8qnZQwOb zJh&5nI`q)VDqTz}d~koLo)3s0DUrOg2Cl6i(?`Y-edn_Y2icI)I7_>S*OtJ^HSff+ zJ`A)dXFrZ|ZA+re+ZKVjHI(UdXQKVS8sEM%2j(#9$7xp9cMZny!lE6NBc3aW;vR08 zsq$NT17YE^7%w5V!@`k*&He_4d5oesypMdSzW8P!#%^Q<1 zGYj=9ds?wcAvWo!3HF(s8_ys<2-(7s!wa`{rXy2~FMXiW0Ff+6yF`SMFt~t5%(mN5-4Cn98$B6zvi90-3NBzVscW9RNj8xN=3vYP6}n z_E?>(Gvp?m6Siwvo)uOvd6YV;=9>`Pf3lCO=wI&_gRYO5mIz;x`-j@E+Te+k85xjn zRfJpq2}(e#k(@&Tv_;z3pRF8B{!=-zKiAHpE$ESpQuRI3x)Qs7#Ry?`B)J$+{&9oqfIgx^!jAZWZ$qY^%#Nbp9gM`nRk(eL62?CzS+H}8RU)24 zROAC|5gmaFMl89Ea}3sPtiY!hQ$ARK$u861{Nm)W)`U6AXkz6PV`RZw&mD`fWs77D zCr(t>6-o;0tI8x8f^rcDyr~aH+c_Q1s|O^d4I&RS1d*)w{qqiIT^as*oRIAcpG8Bg z->0k~N#Ln*!Tb-)yuL1|-q;oyeG2U5X>o2P89KP{_An=)jU4p@ip&gf>*@}FQB|bN(m4b$P>o1c>*6y zaMMs4%m%tR4tVHV;)4|?kkBx)lz!!MMN|`V;6{~I+Mc(!`C)fgQ%_YgVT!d$iSvbm zynZ4>H@@W7AsRZ-m4YjkEKI#A*^0otIo_Z*y&zqE+r#(s4E(#bJklRdf-~Yl#uA^Wrh~cSh)*zeakb#v-UM##M>|~WYy>l z@=v4$wy0?$TXnCBua8rdnwZaMZDPM4t-CMs>Kq`~%J)_5blZuBdXk%0HsBQnZc<9^ zo45X5-(4MSDK|Crjg8S>5eej_3*YE~GpX|EwRC%Jd)KARPreOk*qPt(3*h{1I>3LZ zhk&m(Rf=}2moB{XG`|KDq>NpaLRu&aq zihE#5C+d3qTjxChn){k}Z9v^9u=GB1E-~~7!yMqBkpdoNT=J4_EkoRTJ(-d|%^)B} zN+2Ay*9^-^L4Bk9nbW^6?<{xHYk$Zv`1pHn=+QKlHw{{uC?UOM?Yor7E0{Obmu@vk zVr*n@zEMc|J3jOXM|C)VJY3Qhc2vrI(R82X4`rE}) zbXV6#uY}e%c~59VKdhI+oyF7tjEkaR(;8B@AheVC3fLtU#VFiDN*g`|F zO2Go42eDZ}t(FrhB+Cx68G?3^N!&JzrvUsV#|oZJW@QJD!q}Z=%JtW#MpUb=s;iYC zWh7(~#T(|cWo?`{%vG6v$e=8P^C=#C;&VY|QMsCLs4TT80lrC4+B{B0SvGAYf>O>u z+~R_Ji653u4<$FRZjsdzYamBXhT&dv(vsce@XeG$?X)W0a&PRmtYf-H#(-N)MSaE& zV3I)ROZ?m_Bl&VCIR$mK{B=V$s5SuA$_>SlSp1Q_n9H*#A+yq2Xek z0^H8ABY@Pp<{!`;C;))e4cQE-$RwU~q^KsaWnLfOCs`?EcM@xVXRq=p{JK-1RB4(mjoc#d(}9eF@AoxD0ZK}tEvo@yrcIkhn$0m1_c z7lx%}*^E{dNxO_f0IsKq$^zJyy~du|6sKRN^`Lg~i%jftB*)}*0@hw!w8ja8gaMfv z1Z1X~kqqOp(;=}(bIxcXa|5R-R%FA*sVWgJd3T48y#K%`orEOF459+1SK0i)Lb^5g zKHppvbQ(|cNa7nSeUrF~!b6-lq9#}6MO=K$%^7s}@LtA#d{9)Bb>ZpYuB)bu+t;kD z&H%wxQ^<+i50rNeO?$_S%-G+Cq37aRK?CObN(gzogPX0E@rOT3K2sLRxt0l=gcyHh;( zE1UG)$>rsJ0K=>wZQ!7B`ob=XQI?s;UyMFnVu7A*8ludAl_V&Y0+fl083s;aLdo1M z)}(2yED1q=9s=OPtROYjN(L<_Qi=8zYsiG)>$sZotRQQT*9O35U2HGW@84Hn>TkP= z0Pxn-;{Dck(+(SRt`zeQp7wUH+i?y2>^n*0Nv6j(Ts!UoKM76i^I2Gd&)In_tXgk6 z>bTu2GISxjy)kuU2%Q+|e}72>S1@wNd6jVxihV;s)Ftd0USfr6d7}YpWVvTW?@k^!0c_v7A8cUvVvAV2`)fT7G8~FJInI2t>p+dR(z|R zS7$s4Yrd%tXUpKIH&_1a$q8P|o!>|H^kjhs*soF?`e-I``uo)q8no~JZRUsE&;o4NN4VJ7tFQ8 zjqW7)UQ%x$(d5cSy3H|@!(#^{@7L7l?f~AEs#BY&ItO%|0Aq@%8JA8 z1nLU0UeFegUq!bj#{~hH4MQCB5voYvAS1CYk!MBbqK^Y#CPEm2)R*XcM7qRS>Gdu< zJ~B$Y8*(S^=05rpkC;I&KRRflL5gdJN&BPn2@BC{|7{<1riu9zUBnCFHEds?Jz3o!MO>go4)bkQl4n7kif)^Kl~pB q%m4oKza#L!Bk=$F2m}qmE-C4ry*XGTw@|+t3}tx@xr!H-!T%rUbffzK literal 0 HcmV?d00001