Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http: add headers via local reply mapper #12093

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -605,6 +605,7 @@ message LocalReplyConfig {
}

// The configuration to filter and change local response.
// [#next-free-field: 6]
message ResponseMapper {
// Filter to determine if this mapper should apply.
config.accesslog.v3.AccessLogFilter filter = 1 [(validate.rules).message = {required: true}];
Expand All @@ -619,6 +620,11 @@ message ResponseMapper {
// A per mapper `body_format` to override the :ref:`body_format <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.LocalReplyConfig.body_format>`.
// It will be used when this mapper is matched.
config.core.v3.SubstitutionFormatString body_format_override = 4;
Augustyniak marked this conversation as resolved.
Show resolved Hide resolved

// HTTP headers to add to a local reply. This allows the response mapper to append, to add
// or to override headers of any local reply before it is sent to a downstream client.
repeated config.core.v3.HeaderValueOption headers_to_add = 5
[(validate.rules).repeated = {max_items: 1000}];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would probably remove this. The limit seems somewhat arbitrary.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have these limits in other places for heads to add. I think it was put in for fuzzing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, OK. Sounds good.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this is why I added this requirement - I checked other places where the list of headers was defined and basically all of them had this limitation.

}

message Rds {
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 6 additions & 1 deletion docs/root/configuration/http/http_conn_man/local_reply.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ Features:
Local reply content modification
--------------------------------

The local response content returned by Envoy can be customized. A list of :ref:`mappers <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.LocalReplyConfig.mappers>` can be specified. Each mapper must have a :ref:`filter <envoy_v3_api_field_config.accesslog.v3.AccessLog.filter>`. It may have following rewrite rules; a :ref:`status_code <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.ResponseMapper.status_code>` rule to rewrite response code, a :ref:`body <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.ResponseMapper.body>` rule to rewrite the local reply body and a :ref:`body_format_override <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.ResponseMapper.body_format_override>` to specify the response body format. Envoy checks each `mapper` according to the specified order until the first one is matched. If a `mapper` is matched, all its rewrite rules will apply.
The local response content returned by Envoy can be customized. A list of :ref:`mappers <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.LocalReplyConfig.mappers>` can be specified. Each mapper must have a :ref:`filter <envoy_v3_api_field_config.accesslog.v3.AccessLog.filter>`. It may have following rewrite rules; a :ref:`status_code <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.ResponseMapper.status_code>` rule to rewrite response code, a :ref:`headers_to_add <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.ResponseMapper.headers_to_add>` rule to add/override/append response HTTP headers, a :ref:`body <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.ResponseMapper.body>` rule to rewrite the local reply body and a :ref:`body_format_override <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.ResponseMapper.body_format_override>` to specify the response body format. Envoy checks each `mapper` according to the specified order until the first one is matched. If a `mapper` is matched, all its rewrite rules will apply.

Example of a LocalReplyConfig

Expand All @@ -29,6 +29,11 @@ Example of a LocalReplyConfig
value:
default_value: 400
runtime_key: key_b
headers_to_add:
- header:
key: "foo"
value: "bar"
append: false
status_code: 401
body:
inline_string: "not allowed"
Expand Down
1 change: 1 addition & 0 deletions docs/root/version_history/current.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Minor Behavior Changes
*Changes that may cause incompatibilities for some users, but should not for most*

* compressor: always insert `Vary` headers for compressible resources even if it's decided not to compress a response due to incompatible `Accept-Encoding` value. The `Vary` header needs to be inserted to let a caching proxy in front of Envoy know that the requested resource still can be served with compression applied.
* http: added :ref:`headers_to_add <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.ResponseMapper.headers_to_add>` to :ref:`local reply mapper <config_http_conn_man_local_reply>` to allow its users to add/append/override response HTTP headers to local replies.
* http: added HCM level configuration of :ref:`error handling on invalid messaging <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.HttpConnectionManager.stream_error_on_invalid_http_message>` which substantially changes Envoy's behavior when encountering invalid HTTP/1.1 defaulting to closing the connection instead of allowing reuse. This can temporarily be reverted by setting `envoy.reloadable_features.hcm_stream_error_on_invalid_message` to false, or permanently reverted by setting the :ref:`HCM option <envoy_v3_api_field_extensions.filters.network.http_connection_manager.v3.HttpConnectionManager.stream_error_on_invalid_http_message>` to true to restore prior HTTP/1.1 beavior and setting the *new* HTTP/2 configuration :ref:`override_stream_error_on_invalid_http_message <envoy_v3_api_field_config.core.v3.Http2ProtocolOptions.override_stream_error_on_invalid_http_message>` to false to retain prior HTTP/2 behavior.
* http: the per-stream FilterState maintained by the HTTP connection manager will now provide read/write access to the downstream connection FilterState. As such, code that relies on interacting with this might
see a change in behavior.
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions source/common/local_reply/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ envoy_cc_library(
"//source/common/formatter:substitution_format_string_lib",
"//source/common/formatter:substitution_formatter_lib",
"//source/common/http:header_map_lib",
"//source/common/router:header_parser_lib",
"//source/common/stream_info:stream_info_lib",
"@envoy_api//envoy/extensions/filters/network/http_connection_manager/v3:pkg_cc_proto",
],
Expand Down
7 changes: 7 additions & 0 deletions source/common/local_reply/local_reply.cc
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
#include "common/formatter/substitution_format_string.h"
#include "common/formatter/substitution_formatter.h"
#include "common/http/header_map_impl.h"
#include "common/router/header_parser.h"

namespace Envoy {
namespace LocalReply {
Expand Down Expand Up @@ -43,6 +44,7 @@ class BodyFormatter {
};

using BodyFormatterPtr = std::unique_ptr<BodyFormatter>;
using HeaderParserPtr = std::unique_ptr<Envoy::Router::HeaderParser>;

class ResponseMapper {
public:
Expand All @@ -63,6 +65,8 @@ class ResponseMapper {
if (config.has_body_format_override()) {
body_formatter_ = std::make_unique<BodyFormatter>(config.body_format_override());
}

header_parser_ = Envoy::Router::HeaderParser::configure(config.headers_to_add());
}

bool matchAndRewrite(const Http::RequestHeaderMap& request_headers,
Expand All @@ -79,6 +83,8 @@ class ResponseMapper {
body = body_.value();
}

header_parser_->evaluateHeaders(response_headers, stream_info);

if (status_code_.has_value() && code != status_code_.value()) {
code = status_code_.value();
response_headers.setStatus(std::to_string(enumToInt(code)));
Expand All @@ -95,6 +101,7 @@ class ResponseMapper {
const AccessLog::FilterPtr filter_;
absl::optional<Http::Code> status_code_;
absl::optional<std::string> body_;
HeaderParserPtr header_parser_;
BodyFormatterPtr body_formatter_;
};

Expand Down
44 changes: 44 additions & 0 deletions test/common/local_reply/local_reply_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -291,5 +291,49 @@ TEST_F(LocalReplyTest, TestMapperFormat) {
EXPECT_EQ(content_type_, "text/plain");
}

TEST_F(LocalReplyTest, TestHeaderAddition) {
// Default text formatter without any mappers
const std::string yaml = R"(
mappers:
- filter:
status_code_filter:
comparison:
op: GE
value:
default_value: 0
runtime_key: key_b
headers_to_add:
- header:
key: foo-1
value: bar1
append: true
- header:
key: foo-2
value: override-bar2
append: false
- header:
key: foo-3
value: append-bar3
append: true
)";
TestUtility::loadFromYaml(yaml, config_);
auto local = Factory::create(config_, context_);

response_headers_.addCopy("foo-2", "bar2");
response_headers_.addCopy("foo-3", "bar3");
local->rewrite(nullptr, response_headers_, stream_info_, code_, body_, content_type_);
EXPECT_EQ(code_, TestInitCode);
EXPECT_EQ(stream_info_.response_code_, static_cast<uint32_t>(TestInitCode));
EXPECT_EQ(content_type_, "text/plain");

EXPECT_EQ(response_headers_.get_("foo-1"), "bar1");
EXPECT_EQ(response_headers_.get_("foo-2"), "override-bar2");
std::vector<absl::string_view> out;
Http::HeaderUtility::getAllOfHeader(response_headers_, "foo-3", out);
ASSERT_EQ(out.size(), 2);
ASSERT_EQ(out[0], "bar3");
ASSERT_EQ(out[1], "append-bar3");
}

} // namespace LocalReply
} // namespace Envoy
14 changes: 13 additions & 1 deletion test/integration/local_reply_integration_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,11 @@ TEST_P(LocalReplyIntegrationTest, MapStatusCodeAndFormatToJson) {
name: test-header
exact_match: exact-match-value
status_code: 550
headers_to_add:
- header:
key: foo
value: bar
append: false
body_format:
json_format:
level: TRACE
Expand Down Expand Up @@ -74,6 +79,7 @@ TEST_P(LocalReplyIntegrationTest, MapStatusCodeAndFormatToJson) {
EXPECT_EQ("application/json", response->headers().ContentType()->value().getStringView());
EXPECT_EQ("150", response->headers().ContentLength()->value().getStringView());
EXPECT_EQ("550", response->headers().Status()->value().getStringView());
EXPECT_EQ("bar", response->headers().get(Http::LowerCaseString("foo"))->value().getStringView());
// Check if returned json is same as expected
EXPECT_TRUE(TestUtility::jsonStringEqual(response->body(), expected_body));
}
Expand Down Expand Up @@ -131,7 +137,7 @@ TEST_P(LocalReplyIntegrationTest, MapStatusCodeAndFormatToJson4Grpc) {
expected_grpc_message));
}

// Matched second filter has code and body rewrite and its format
// Matched second filter has code, headers and body rewrite and its format
TEST_P(LocalReplyIntegrationTest, MapStatusCodeAndFormatToJsonForFirstMatchingFilter) {
const std::string yaml = R"EOF(
mappers:
Expand All @@ -147,6 +153,11 @@ TEST_P(LocalReplyIntegrationTest, MapStatusCodeAndFormatToJsonForFirstMatchingFi
name: test-header
exact_match: exact-match-value
status_code: 551
headers_to_add:
- header:
key: foo
value: bar
append: false
body:
inline_string: "customized body text"
body_format_override:
Expand Down Expand Up @@ -199,6 +210,7 @@ TEST_P(LocalReplyIntegrationTest, MapStatusCodeAndFormatToJsonForFirstMatchingFi
EXPECT_EQ("text/plain", response->headers().ContentType()->value().getStringView());
EXPECT_EQ("24", response->headers().ContentLength()->value().getStringView());
EXPECT_EQ("551", response->headers().Status()->value().getStringView());
EXPECT_EQ("bar", response->headers().get(Http::LowerCaseString("foo"))->value().getStringView());
// Check if returned json is same as expected
EXPECT_EQ(response->body(), expected_body);
}
Expand Down