Skip to content

Commit

Permalink
Add a /config_dump endpoint
Browse files Browse the repository at this point in the history
That returns a JSON of the current config the proxy is running with

This includes a breaking change, as the FilterFactory to now needs to return
the config in addition to the filter it created.

Resolves #394
  • Loading branch information
iffyio committed Sep 14, 2021
1 parent 8adba36 commit 8a62b3d
Show file tree
Hide file tree
Showing 22 changed files with 561 additions and 148 deletions.
5 changes: 5 additions & 0 deletions docs/src/admin.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,8 @@ Will return an HTTP status of 200 when all health checks pass.
Outputs [Prometheus](https://prometheus.io/) formatted metrics for this proxy.
See the [Proxy Metrics](./proxy.md#metrics) documentation for what metrics are available.
## /config_dump
Returns a JSON representation of the cluster and filterchain configuration that the proxy is running
with at the time of invocation.
84 changes: 50 additions & 34 deletions docs/src/filters/writing_custom_filters.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ A [trait][Filter] representing an actual [Filter][built-in-filters] instance in
A [trait][FilterFactory] representing a type that knows how to create instances of a particular type of [Filter].

- An implementation provides a `name` and `create_filter` method.
- `create_filter` takes in [configuration][filter configuration] for the filter to create and returns a new instance of its filter type.
- `create_filter` takes in [configuration][filter configuration] for the filter to create and returns a [CreatedFilter] type containing a new instance of its filter type.
`name` returns the Filter name - a unique identifier of filters of the created type (e.g quilkin.extensions.filters.debug.v1alpha1.Debug).

##### FilterRegistry
Expand Down Expand Up @@ -95,16 +95,17 @@ To extend Quilkin's code with our own custom filter, we need to do the following
# struct Greet;
# impl Filter for Greet {}
# use quilkin::filters::Filter;
use quilkin::filters::{CreateFilterArgs, Error, FilterFactory};
use quilkin::filters::{CreatedFilter, CreateFilterArgs, Error, FilterFactory};

struct GreetFilterFactory;
impl FilterFactory for GreetFilterFactory {
fn name(&self) -> &'static str {
// We provide the name of filter that we defined earlier.
NAME
}
fn create_filter(&self, _: CreateFilterArgs) -> Result<Box<dyn Filter>, Error> {
Ok(Box::new(Greet))
fn create_filter(&self, _: CreateFilterArgs) -> Result<CreatedFilter, Error> {
let filter: Box<dyn Filter> = Box::new(Greet);
Ok((serde_json::Value::Null, filter).into())
}
}
```
Expand All @@ -124,13 +125,13 @@ To extend Quilkin's code with our own custom filter, we need to do the following
Add a main function that starts the proxy.
```rust, no_run
// src/main.rs
# use quilkin::filters::{CreateFilterArgs, Filter, Error, FilterFactory};
# use quilkin::filters::{CreatedFilter, CreateFilterArgs, Filter, Error, FilterFactory};
# struct GreetFilterFactory;
# impl FilterFactory for GreetFilterFactory {
# fn name(&self) -> &'static str {
# "greet.v1"
# }
# fn create_filter(&self, _: CreateFilterArgs) -> Result<Box<dyn Filter>, Error> {
# fn create_filter(&self, _: CreateFilterArgs) -> Result<CreatedFilter, Error> {
# unimplemented!()
# }
# }
Expand Down Expand Up @@ -232,7 +233,7 @@ The [Serde] crate is used to describe static YAML configuration in code while [P
# struct Config {
# greeting: String,
# }
# use quilkin::filters::{CreateFilterArgs, Error, FilterFactory, Filter, ReadContext, ReadResponse, WriteContext, WriteResponse};
# use quilkin::filters::{CreatedFilter, CreateFilterArgs, Error, FilterFactory, Filter, ReadContext, ReadResponse, WriteContext, WriteResponse};
# struct Greet(String);
# impl Filter for Greet { }
use quilkin::config::ConfigType;
Expand All @@ -242,16 +243,16 @@ The [Serde] crate is used to describe static YAML configuration in code while [P
fn name(&self) -> &'static str {
"greet.v1"
}
fn create_filter(&self, args: CreateFilterArgs) -> Result<Box<dyn Filter>, Error> {
let greeting = match args.config.unwrap() {
ConfigType::Static(config) => {
serde_yaml::from_str::<Config>(serde_yaml::to_string(config).unwrap().as_str())
.unwrap()
.greeting
}
ConfigType::Dynamic(_) => unimplemented!("dynamic config is not yet supported for this filter"),
fn create_filter(&self, args: CreateFilterArgs) -> Result<CreatedFilter, Error> {
let config = match args.config.unwrap() {
ConfigType::Static(config) => {
serde_yaml::from_str::<Config>(serde_yaml::to_string(config).unwrap().as_str())
.unwrap()
}
ConfigType::Dynamic(_) => unimplemented!("dynamic config is not yet supported for this filter"),
};
Ok(Box::new(Greet(greeting)))
let filter: Box<dyn Filter> = Box::new(Greet(config.greeting));
Ok((serde_json::Value::Null, filter).into())
}
}
```
Expand All @@ -275,20 +276,20 @@ static:
You might have noticed while adding [static configuration support][anchor-static-config], that the [config][CreateFilterArgs::config] argument passed into our [FilterFactory]
has a [Dynamic][ConfigType::dynamic] variant.
```rust, ignore
let greeting = match args.config.unwrap() {
let config = match args.config.unwrap() {
ConfigType::Static(config) => {
serde_yaml::from_str::<Config>(serde_yaml::to_string(config).unwrap().as_str())
.unwrap()
.greeting
}
ConfigType::Dynamic(_) => unimplemented!("dynamic config is not yet supported for this filter"),
};
```

It contains the serialized [Protobuf] message received from the [management server] for the [Filter] to create.
The [Dynamic][ConfigType::dynamic] contains the serialized [Protobuf] message received from the [management server] for the [Filter] to create.
As a result, its contents are entirely opaque to Quilkin and it is represented with the [Prost Any][prost-any] type so the [FilterFactory]
can interpret its contents anyway it wishes to.
can interpret its contents however it wishes.
However, it usually contains a Protobuf equivalent of the filter's static configuration.

1. Add the proto parsing crates to Cargo.toml:
Expand Down Expand Up @@ -340,9 +341,14 @@ However, it usually contains a Protobuf equivalent of the filter's static config
```
1. Decode the serialized proto message into the generated config:

If the message contains a Protobuf equivalent of the filter's static configuration, we can
leverage the [deserialize][ConfigType::deserialize] method to marshal the input. The function
automatically calls a conversion function from the Protobuf type if the input contains a dynamic
configuration.

```rust
// src/main.rs
# use quilkin::{config::ConfigType, filters::{CreateFilterArgs, Error, Filter, FilterFactory}};
# use quilkin::{config::ConfigType, filters::{CreatedFilter, CreateFilterArgs, Error, Filter, FilterFactory}};
# use serde::{Deserialize, Serialize};
# #[derive(Serialize, Deserialize, Debug)]
# struct Config {
Expand All @@ -363,38 +369,48 @@ However, it usually contains a Protobuf equivalent of the filter's static config
# }
# struct Greet(String);
# impl Filter for Greet { }
use quilkin::filters::ConvertProtoConfigError;
use bytes::Bytes;
use std::convert::TryFrom;

// Implement the conversion from dynamic to static configuration types.
impl TryFrom<greet::Greet> for Config {
type Error = ConvertProtoConfigError;

fn try_from(p: greet::Greet) -> Result<Self, Self::Error> {
Ok(Self {
greeting: p.greeting
})
}
}

struct GreetFilterFactory;
impl FilterFactory for GreetFilterFactory {
fn name(&self) -> &'static str {
"greet.v1"
}
fn create_filter(&self, args: CreateFilterArgs) -> Result<Box<dyn Filter>, Error> {
let greeting = match args.config.unwrap() {
ConfigType::Static(config) => {
serde_yaml::from_str::<Config>(serde_yaml::to_string(config).unwrap().as_str())
.unwrap()
.greeting
}
ConfigType::Dynamic(config) => {
let config: greet::Greet = prost::Message::decode(Bytes::from(config.value)).unwrap();
config.greeting
}
};
Ok(Box::new(Greet(greeting)))
fn create_filter(&self, args: CreateFilterArgs) -> Result<CreatedFilter, Error> {
let (config_json, config) = self
.require_config(args.config)?
.deserialize::<Config, greet::Greet>(self.name())?;
let filter: Box<dyn Filter> = Box::new(Greet(config.greeting));
Ok((config_json, filter).into())
}
}
```

[CreatedFilter]: ../../api/quilkin/filters/prelude/struct.CreatedFilter.html
[Filter]: ../../api/quilkin/filters/trait.Filter.html
[FilterFactory]: ../../api/quilkin/filters/trait.FilterFactory.html
[filter-factory-name]: ../../api/quilkin/filters/trait.FilterFactory.html#tymethod.name
[FilterRegistry]: ../../api/quilkin/filters/struct.FilterRegistry.html
[runner::run]: ../../api/quilkin/runner/fn.run.html
[CreateFilterArgs::config]: ../../api/quilkin/filters/prelude/struct.CreateFilterArgs.html#structfield.config
[ConfigType::dynamic]: ../../api/quilkin/config/enum.ConfigType.html#variant.Dynamic
[ConfigType::static]: ../../api/quilkin/config/enum.ConfigType.html#variant.Static
[ConfigType::deserialize]: ../../api/quilkin/config/enum.ConfigType.html#method.deserialize

[anchor-dynamic-config]: #dynamic-configuration
[anchor-static-config]: #static-configuration
[Filters]: ../filters.md
[filter chain]: ../filters.md#filters-and-filter-chain
Expand Down
7 changes: 4 additions & 3 deletions examples/quilkin-filter-example/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -62,11 +62,12 @@ impl FilterFactory for GreetFilterFactory {
fn name(&self) -> &'static str {
NAME
}
fn create_filter(&self, args: CreateFilterArgs) -> Result<Box<dyn Filter>, Error> {
let greeting = self
fn create_filter(&self, args: CreateFilterArgs) -> Result<CreatedFilter, Error> {
let (config_json, config) = self
.require_config(args.config)?
.deserialize::<Config, ProtoGreet>(self.name())?;
Ok(Box::new(Greet(greeting.greeting)))
let filter: Box<dyn Filter> = Box::new(Greet(config.greeting));
Ok((config_json, filter).into())
}
}

Expand Down
95 changes: 88 additions & 7 deletions src/config/config_type.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,17 +30,42 @@ pub enum ConfigType<'a> {
}

impl ConfigType<'_> {
/// Deserializes the configuration to `T` based on the input type. Errors if
/// the data produces an invalid config.
pub fn deserialize<T, P>(self, filter_name: &str) -> Result<T, Error>
/// Deserializes takes two type arguments `Static` and `Dynamic` representing
/// the types of a static and dynamic configuration respectively.
///
/// If the configuration input is a [ConfigType::Static], then it is directly
/// marshalled into the returned `Static` instance.
///
/// Otherwise if the input is a [ConfigType::Dynamic] then the contained Protobuf
/// data is decoded into a type `Dynamic` instance, after which the decoded
/// value is converted into the returned `Static` instance.
/// As a result [TryFrom] must be implemented from the `Dynamic` to the `Static`
/// type.
///
/// It returns both the marshalled, as well as, a JSON representation
/// of the provided config.
/// It returns an error if any of the serialization or deserialization steps fail.
pub fn deserialize<Static, Dynamic>(
self,
filter_name: &str,
) -> Result<(serde_json::Value, Static), Error>
where
P: prost::Message + Default,
T: for<'de> serde::Deserialize<'de> + TryFrom<P, Error = ConvertProtoConfigError>,
Dynamic: prost::Message + Default,
Static: serde::Serialize
+ for<'de> serde::Deserialize<'de>
+ TryFrom<Dynamic, Error = ConvertProtoConfigError>,
{
match self {
ConfigType::Static(config) => serde_yaml::to_string(config)
.and_then(|raw_config| serde_yaml::from_str(raw_config.as_str()))
.map_err(|err| Error::DeserializeFailed(err.to_string())),
.map_err(|err| {
Error::DeserializeFailed(format!(
"filter `{}`: failed to YAML deserialize config: {}",
filter_name,
err.to_string()
))
})
.and_then(|config| Self::get_json_config(filter_name, config)),
ConfigType::Dynamic(config) => prost::Message::decode(Bytes::from(config.value))
.map_err(|err| {
Error::DeserializeFailed(format!(
Expand All @@ -49,7 +74,63 @@ impl ConfigType<'_> {
err.to_string()
))
})
.and_then(|config| T::try_from(config).map_err(Error::ConvertProtoConfig)),
.and_then(|config| Static::try_from(config).map_err(Error::ConvertProtoConfig))
.and_then(|config| Self::get_json_config(filter_name, config)),
}
}

// Returns an equivalent json value for the passed in config.
fn get_json_config<T>(filter_name: &str, config: T) -> Result<(serde_json::Value, T), Error>
where
T: serde::Serialize + for<'de> serde::Deserialize<'de>,
{
serde_json::to_string(&config)
.map_err(|err| {
Error::DeserializeFailed(format!(
"filter `{}`: failed to serialize config to json: {}",
filter_name,
err.to_string()
))
})
.and_then(|config_json| {
serde_json::from_str::<serde_json::Value>(config_json.as_str()).map_err(|err| {
Error::DeserializeFailed(format!(
"filter `{}`: failed to deserialize into json: {}",
filter_name,
err.to_string()
))
})
})
.map(|config_json| (config_json, config))
}
}

#[cfg(test)]
mod tests {
use super::ConfigType;
use serde::{Deserialize, Serialize};

#[test]
fn get_json_config() {
#[derive(Debug, Eq, PartialEq, Clone, Serialize, Deserialize)]
struct TestConfig {
name: String,
value: usize,
}
let expected_config = TestConfig {
name: "bebop".into(),
value: 98,
};
let (config_json, config) =
ConfigType::get_json_config("my-filter", expected_config.clone()).unwrap();

assert_eq!(expected_config, config);
assert_eq!(
serde_json::json!({
"name": "bebop",
"value": 98,
}),
config_json
)
}
}
6 changes: 3 additions & 3 deletions src/filters.rs
Original file line number Diff line number Diff line change
Expand Up @@ -39,15 +39,15 @@ pub mod token_router;
/// [`FilterFactory`].
pub mod prelude {
pub use super::{
ConvertProtoConfigError, CreateFilterArgs, DynFilterFactory, Error, Filter, FilterFactory,
ReadContext, ReadResponse, WriteContext, WriteResponse,
ConvertProtoConfigError, CreateFilterArgs, CreatedFilter, DynFilterFactory, Error, Filter,
FilterFactory, ReadContext, ReadResponse, WriteContext, WriteResponse,
};
}

// Core Filter types
pub use self::{
error::{ConvertProtoConfigError, Error},
factory::{CreateFilterArgs, DynFilterFactory, FilterFactory},
factory::{CreateFilterArgs, CreatedFilter, DynFilterFactory, FilterFactory},
read::{ReadContext, ReadResponse},
registry::FilterRegistry,
set::{FilterMap, FilterSet},
Expand Down
19 changes: 10 additions & 9 deletions src/filters/capture_bytes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -102,13 +102,12 @@ impl FilterFactory for CaptureBytesFactory {
NAME
}

fn create_filter(&self, args: CreateFilterArgs) -> Result<Box<dyn Filter>, Error> {
Ok(Box::new(CaptureBytes::new(
&self.log,
self.require_config(args.config)?
.deserialize::<Config, ProtoConfig>(self.name())?,
Metrics::new(&args.metrics_registry)?,
)))
fn create_filter(&self, args: CreateFilterArgs) -> Result<CreatedFilter, Error> {
let (config_json, config) = self
.require_config(args.config)?
.deserialize::<Config, ProtoConfig>(self.name())?;
let filter = CaptureBytes::new(&self.log, config, Metrics::new(&args.metrics_registry)?);
Ok((config_json, Box::new(filter) as Box<dyn Filter>).into())
}
}

Expand Down Expand Up @@ -160,7 +159,8 @@ mod tests {
Registry::default(),
Some(&Value::Mapping(map)),
))
.unwrap();
.unwrap()
.filter;
assert_end_strategy(filter.as_ref(), TOKEN_KEY, true);
}

Expand All @@ -174,7 +174,8 @@ mod tests {
Registry::default(),
Some(&Value::Mapping(map)),
))
.unwrap();
.unwrap()
.filter;
assert_end_strategy(filter.as_ref(), CAPTURED_BYTES, false);
}

Expand Down
Loading

0 comments on commit 8a62b3d

Please sign in to comment.