-
Notifications
You must be signed in to change notification settings - Fork 271
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
inbound: Improve policy metrics (#1237)
We recently introduced metrics to help surface inbound policy decisions, but in practice these haven't been as useful as we might hope. Specifically, error metrics do not include the `target_addr` label so these metrics can't be correlated with servers, etc. This change improves error metrics and also introduces new metrics to describe authorization decisions: authorization denials shouldn't be classified as errors, really, anyway. This change also improves TCP forwarding authorization so that policy changes can be honored at runtime: previously authorized connections may dropped if the policy is updated so that the connection is no longer authorized. The gateway is also updated to enforce HTTP policies at runtime as well so that policy changes can be honored after the connection has been established. This change introduces new metrics: * `inbound_http_authz_allow_total` * `inbound_http_authz_deny_total` * `inbound_tcp_authz_allow_total` * `inbound_tcp_authz_deny_total` * `inbound_tcp_authz_terminate_total` _allow_ metrics include `target_addr`, `srv_name`, and `saz_name` labels. _deny_ and _terminate_ metics include only `target_addr` and `srv_name` labels. Authorization denials are no longer reflected in inbound_tcp_error or inbound_http_error metrics. A number of internal changes have been made to support this: * The `inbound::policy::authorize` module includes middlewares for TCP and HTTP authorization, replacing the prior method of enforcing policy in the stack/router. This module ensures that metrics are recorded for policy decisions. * The `error-metrics` crate has been removed. In its place a `monitor` type has been added to the `stack` crate, supporting a general way to observe errors, decoupled from the metrics registry. * Inbound and outbound error metrics are now tracked in the inbound and outbound crates, respectively. Inbound- and outbound-specific error types are also moved into their rspective crates. * The `app_core::errors` module has been updated to only define the types it needs to instrument the error response layer. Error responses are now primarily instrumented via the `HttpError` type so that errors that should be handled can be configured where the error is thrown. The error type now holds an underlying source error so that the error metrics layer can see through this wrapper type to track the underlying error cause. * Server & Authorization labels are no longer handled as a free-form maps. We currently read only the `name` label from each; and this label is required.
- Loading branch information
Showing
66 changed files
with
1,940 additions
and
1,204 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,44 @@ | ||
| pub mod respond; | ||
|
|
||
| use linkerd_error::Error; | ||
| pub use linkerd_timeout::{FailFastError, ResponseTimeout}; | ||
| use thiserror::Error; | ||
|
|
||
| #[derive(Debug, Error)] | ||
| #[error("connect timed out after {0:?}")] | ||
| pub(crate) struct ConnectTimeout(pub std::time::Duration); | ||
|
|
||
| #[derive(Debug, Error)] | ||
| #[error("{source}")] | ||
| pub struct HttpError { | ||
| #[source] | ||
| source: Error, | ||
| http_status: http::StatusCode, | ||
| grpc_status: tonic::Code, | ||
| } | ||
|
|
||
| impl HttpError { | ||
| pub fn bad_request(source: impl Into<Error>) -> Self { | ||
| Self { | ||
| source: source.into(), | ||
| http_status: http::StatusCode::BAD_REQUEST, | ||
| grpc_status: tonic::Code::InvalidArgument, | ||
| } | ||
| } | ||
|
|
||
| pub fn forbidden(source: impl Into<Error>) -> Self { | ||
| Self { | ||
| source: source.into(), | ||
| http_status: http::StatusCode::FORBIDDEN, | ||
| grpc_status: tonic::Code::PermissionDenied, | ||
| } | ||
| } | ||
|
|
||
| pub fn loop_detected(source: impl Into<Error>) -> Self { | ||
| Self { | ||
| source: source.into(), | ||
| http_status: http::StatusCode::LOOP_DETECTED, | ||
| grpc_status: tonic::Code::Aborted, | ||
| } | ||
| } | ||
| } |
Oops, something went wrong.