Add headers from the rlp call to the response #31

alexsnaps · 2023-03-24T12:52:15Z

eguzki · 2023-03-27T11:55:11Z

It's panic'ing 😠

how to reproduce

Update docker compose to deploy limitador with rate limit headers

❯ git diff
diff --git a/docker-compose.yaml b/docker-compose.yaml
index e0265d6..ced9f85 100644
--- a/docker-compose.yaml
+++ b/docker-compose.yaml
@@ -27,7 +27,7 @@ services:
       - ./target/wasm32-unknown-unknown/release/wasm_shim.wasm:/opt/kuadrant/wasm/wasm_shim.wasm
   limitador:
     image: quay.io/kuadrant/limitador:latest
-    command: ["limitador-server", "-vvv", "/opt/kuadrant/limits/limits.yaml"]
+    command: ["limitador-server", "--rate-limit-headers", "DRAFT_VERSION_03", "-vvv", "/opt/kuadrant/limits/limits.yaml"]
     expose:
       - "8080"
       - "8081"

Set up development env

make development

Send one request that is supposed to trigger call to limitador

curl -H "Host: test.c.com" -H "x-forwarded-for: 127.0.0.1" -H "My-Custom-Header-01: my-custom-header-value-01" -H "My-Custom-Header-02: my-custom-header-value-02" -H "x-dyn-user-id: bob" http://127.0.0.1:18000/get
curl: (52) Empty reply from server

Check environment logs

wasm-shim-envoy-1      | [2023-03-27 11:53:20.673][49][info][wasm] [source/extensions/common/wasm/context.cc:1167] wasm log kuadrant_wasm kuadrant_wasm vm.sentinel.kuadrant_wasm: Initiated gRPC call (id# 5) to Limitador
wasm-shim-envoy-1      | [2023-03-27 11:53:20.673][49][debug][router] [source/common/router/upstream_request.cc:416] [C0][S12193357470377373743] pool ready
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG h2::codec::framed_write] send frame=Settings { flags: (0x0), initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16777216 }
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG h2::proto::connection] Connection; peer=Server
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG h2::codec::framed_read] received frame=Settings { flags: (0x0), header_table_size: 4096, enable_push: 0, max_concurrent_streams: 2147483647, initial_window_size: 268435456, enable_connect_protocol: 0 }
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG h2::codec::framed_write] send frame=Settings { flags: (0x1: ACK) }
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG h2::codec::framed_read] received frame=WindowUpdate { stream_id: StreamId(0), size_increment: 268369921 }
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG h2::codec::framed_read] received frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) }
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG h2::codec::framed_read] received frame=Data { stream_id: StreamId(1), flags: (0x1: END_STREAM) }
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG h2::codec::framed_write] send frame=WindowUpdate { stream_id: StreamId(0), size_increment: 983041 }
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG limitador_server::envoy_rls::server] Request received: Request { metadata: MetadataMap { headers: {"te": "trailers", "grpc-timeout": "5000m", "content-type": "application/grpc", "x-envoy-internal": "true", "x-forwarded-for": "172.19.0.4", "x-envoy-expected-rq-timeout-ms": "5000"} }, message: RateLimitRequest { domain: "rls-domain-c", descriptors: [RateLimitDescriptor { entries: [Entry { key: "some_generic_key_for_c", value: "some_generic_value_for_c" }], limit: None }, RateLimitDescriptor { entries: [Entry { key: "remote_address", value: "127.0.0.1" }], limit: None }, RateLimitDescriptor { entries: [Entry { key: "header_match", value: "header_value_match_value" }], limit: None }, RateLimitDescriptor { entries: [Entry { key: "my_custom_header_key", value: "my-custom-header-value-02" }], limit: None }, RateLimitDescriptor { entries: [Entry { key: "user_id", value: "bob" }], limit: None }], hits_addend: 1 }, extensions: Extensions }
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG h2::codec::framed_write] send frame=Headers { stream_id: StreamId(1), flags: (0x4: END_HEADERS) }
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG h2::codec::framed_write] send frame=Data { stream_id: StreamId(1) }
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG h2::codec::framed_write] send frame=Headers { stream_id: StreamId(1), flags: (0x5: END_HEADERS | END_STREAM) }
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG h2::codec::framed_read] received frame=Settings { flags: (0x1: ACK) }
wasm-shim-limitador-1  | [2023-03-27T11:53:20Z DEBUG h2::proto::settings] received settings ACK; applying Settings { flags: (0x0), initial_window_size: 1048576, max_frame_size: 16384, max_header_list_size: 16777216 }
wasm-shim-envoy-1      | [2023-03-27 11:53:20.674][49][debug][router] [source/common/router/router.cc:1285] [C0][S12193357470377373743] upstream headers complete: end_stream=false
wasm-shim-envoy-1      | [2023-03-27 11:53:20.674][49][debug][http] [source/common/http/async_client_impl.cc:101] async http request response headers (end_stream=false):
wasm-shim-envoy-1      | ':status', '200'
wasm-shim-envoy-1      | 'content-type', 'application/grpc'
wasm-shim-envoy-1      | 'date', 'Mon, 27 Mar 2023 11:53:20 GMT'
wasm-shim-envoy-1      | 'x-envoy-upstream-service-time', '0'
wasm-shim-envoy-1      | 
wasm-shim-envoy-1      | [2023-03-27 11:53:20.674][49][debug][http] [source/common/http/async_client_impl.cc:128] async http request response trailers:
wasm-shim-envoy-1      | 'grpc-status', '0'
wasm-shim-envoy-1      | 
wasm-shim-envoy-1      | [2023-03-27 11:53:20.675][49][info][wasm] [source/extensions/common/wasm/context.cc:1167] wasm log kuadrant_wasm kuadrant_wasm vm.sentinel.kuadrant_wasm: received gRPC call response: token: 5, status: 0
wasm-shim-envoy-1      | [2023-03-27 11:53:20.675][49][critical][wasm] [source/extensions/common/wasm/context.cc:1176] wasm log kuadrant_wasm kuadrant_wasm vm.sentinel.kuadrant_wasm: panicked at 'unexpected status: 2', /home/eguzki/.cargo/registry/src/github.aaakk.us.kg-1ecc6299db9ec823/proxy-wasm-0.2.0/src/hostcalls.rs:382:23
wasm-shim-envoy-1      | [2023-03-27 11:53:20.675][49][error][wasm] [source/extensions/common/wasm/wasm_vm.cc:38] Function: proxy_on_grpc_receive failed: Uncaught RuntimeError: unreachable
wasm-shim-envoy-1      | Proxy-Wasm plugin in-VM backtrace:
wasm-shim-envoy-1      |   0:  0x121f2b - __rust_start_panic
wasm-shim-envoy-1      |   1:  0x121ead - rust_panic
wasm-shim-envoy-1      |   2:  0x121e7d - _ZN3std9panicking20rust_panic_with_hook17h8d0a63d0479a36c8E
wasm-shim-envoy-1      |   3:  0x1213e4 - _ZN3std9panicking19begin_panic_handler28_$u7b$$u7b$closure$u7d$$u7d$17hac9eebb5245da5ffE
wasm-shim-envoy-1      |   4:  0x12130e - _ZN3std10sys_common9backtrace26__rust_end_short_backtrace17h1def76e070463564E
wasm-shim-envoy-1      |   5:  0x121ad4 - rust_begin_unwind
wasm-shim-envoy-1      |   6:  0x122f2f - _ZN4core9panicking9panic_fmt17h2a7f454a5f5da218E
wasm-shim-envoy-1      |   7:  0x84b59 - _ZN10proxy_wasm9hostcalls19add_map_value_bytes17hbbfded6c97e852a7E
wasm-shim-envoy-1      |   8:  0x3dd94 - _ZN87_$LT$wasm_shim..filter..http_context..Filter$u20$as$u20$proxy_wasm..traits..Context$GT$21on_grpc_call_response17h8f1190c3ef35befbE
wasm-shim-envoy-1      |   9:  0x80911 - _ZN10proxy_wasm10dispatcher10Dispatcher15on_grpc_receive17h2059e4041ee7980cE
wasm-shim-envoy-1      | [2023-03-27 11:53:25.668][49][debug][http] [source/common/http/conn_manager_impl.cc:1517] [C0][S11724252115896496172] stream reset

It is raising uncaught runtime error when adding the response headers. Specifically when calling hostcalls::add_map_value_bytes

alexsnaps · 2023-03-27T12:48:27Z

Thanks @eguzki ! Sorry about this, let me investigate this...

alexsnaps · 2023-03-28T12:41:08Z

With @eguzki steps above, we now get the headers in the response in both cases, 200s & 429s:

< HTTP/1.1 200 OK
< server: envoy
< date: Tue, 28 Mar 2023 12:39:01 GMT
< content-type: application/json
< content-length: 404
< access-control-allow-origin: *
< access-control-allow-credentials: true
< x-envoy-upstream-service-time: 19
< x-ratelimit-limit: 2, 2;w=10
< x-ratelimit-remaining: 1
< 
{
  "args": {}, 
  "headers": {
    "Accept": "*/*", 
    "Host": "test.c.com", 
    "My-Custom-Header-01": "my-custom-header-value-01", 
    "My-Custom-Header-02": "my-custom-header-value-02", 
    "User-Agent": "curl/7.86.0", 
    "X-Dyn-User-Id": "bob", 
    "X-Envoy-Expected-Rq-Timeout-Ms": "15000", 
    "X-Envoy-Internal": "true"
  }, 
  "origin": "127.0.0.1", 
  "url": "http://test.c.com/get"
}

< HTTP/1.1 429 Too Many Requests
< x-ratelimit-limit: 2, 2;w=10
< x-ratelimit-remaining: 0
< content-length: 18
< content-type: text/plain
< date: Tue, 28 Mar 2023 12:39:02 GMT
< server: envoy
< 
Too Many Requests

eguzki

nitpick: instead of headers, which is too generic, why not call it response_headers_to_add? I am sure you can come up with a better name :)

eguzki · 2023-03-28T16:45:03Z

working in my local host. 🎖️

alexsnaps · 2023-03-28T17:24:13Z

Slightly renamed fields where it made sense. The headers, where they are added, are matched from the destructuring of response_headers_to_add, so that's quite a bit of repetition. Otherwise they are the "only" headers

alexsnaps · 2023-03-28T17:40:09Z

Duh, now I understand what you meant: #33

alexsnaps requested a review from eguzki March 24, 2023 12:52

alexsnaps force-pushed the response_headers branch from bb50371 to 4c5a176 Compare March 27, 2023 13:59

Add headers from the rlp call to the response

0962430

alexsnaps force-pushed the response_headers branch from 4c5a176 to 0962430 Compare March 27, 2023 14:08

eguzki approved these changes Mar 28, 2023

View reviewed changes

Renamed some fields, to make it somewhat clearer

2cecf84

alexsnaps merged commit decf58c into main Mar 28, 2023

alexsnaps deleted the response_headers branch March 28, 2023 17:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add headers from the rlp call to the response #31

Add headers from the rlp call to the response #31

alexsnaps commented Mar 24, 2023 •

edited

Loading

eguzki commented Mar 27, 2023

alexsnaps commented Mar 27, 2023

alexsnaps commented Mar 28, 2023 •

edited

Loading

eguzki left a comment

eguzki commented Mar 28, 2023

alexsnaps commented Mar 28, 2023

alexsnaps commented Mar 28, 2023

Add headers from the rlp call to the response #31

Add headers from the rlp call to the response #31

Conversation

alexsnaps commented Mar 24, 2023 • edited Loading

eguzki commented Mar 27, 2023

how to reproduce

alexsnaps commented Mar 27, 2023

alexsnaps commented Mar 28, 2023 • edited Loading

eguzki left a comment

Choose a reason for hiding this comment

eguzki commented Mar 28, 2023

alexsnaps commented Mar 28, 2023

alexsnaps commented Mar 28, 2023

alexsnaps commented Mar 24, 2023 •

edited

Loading

alexsnaps commented Mar 28, 2023 •

edited

Loading