Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect(?) decoding of forwarded URL #688

Closed
yoanisgil opened this issue Nov 5, 2015 · 5 comments
Closed

Incorrect(?) decoding of forwarded URL #688

yoanisgil opened this issue Nov 5, 2015 · 5 comments
Labels

Comments

@yoanisgil
Copy link

I have this API which is defined as follows:

{
  "data": [
    {
      "created_at": 1446738461000,
      "name": "craktracking-track",
      "strip_request_path": true,
      "id": "269db6f2-8b95-43c6-c3e1-227ba5f72826",
      "request_path": "/craktracking-track",
      "upstream_url": "http://vm1.ygil.dev.lan:9005"
    }
  ]
}

when I use Kong to query the upstream API:

curl http://vm1.ygil.dev.lan:9005/v1/track/CR/101.226.89.123247c5a5055f4f1753a6ec25da8a49202/1662321.4049.CN.14465.1662321_3034_SPANK_UPL_CN_3034_10310_ADV27882_12897C_%2F_mobile?apikey=API_KEY

I get this:

{
  "apiVersion": "1.0",
  "method": "unknown",
  "params": {},
  "error": {
    "code": 404,
    "message": "File Not Found",
    "errors": [
      {
        "domain": "General",
        "reason": "ResourceNotFoundException",
        "message": "File Not Found"
      }
    ]
  }
}

which is the response from the upstream server complaining that the provided resource does not exists.

Now, when I bypass Kong and contact directly the upstream API:

http://aws038:8000/craktracking-track/v1/track/CR/107.77.70.6388897fcf61c6a404d41ef51a50379bbe/1.1233.US.4657.TFA_151785_XVID_SQU_US_OTHERS_22805C_NDP_NDP17877_MBAK_GEN_POS2_ALL_MBAK_TPS_POS?apikey=API_KEY 

I get this:

{
  "apiVersion": "1.0",
  "method": "track.get",
  "params": {
    "service": "CR",
    "uid": "107.77.70.6388897fcf61c6a404d41ef51a50379bbe",
    "track": "1.1233.US.4657.TFA_151785_XVID_SQU_US_OTHERS_22805C_NDP_NDP17877_MBAK_GEN_POS2_ALL_MBAK_TPS_POS"
  },
  "data": {
    "items": []
  }
}

If you look close to the request path/parameters you will notice that there is a %2F (/):

/v1/track/CR/101.226.89.123247c5a5055f4f1753a6ec25da8a49202/1662321.4049.CN.14465.1662321_3034_SPANK_UPL_CN_3034_10310_ADV27882_12897C_%2F_mobile?apikey=API_KEY

it looks like either Kong or nginx are performing a URL decode before forwarding the request to the upstream server:

/v1/track/CR/101.226.89.123247c5a5055f4f1753a6ec25da8a49202/1662321.4049.CN.14465.1662321_3034_SPANK_UPL_CN_3034_10310_ADV27882_12897C_/_mobile?apikey=API_KEY

Any ideas?

@yoanisgil yoanisgil changed the title Decoding forwarded URL Incorrect(?) decoding of forwarded URL Nov 5, 2015
@thibaultcha
Copy link
Member

This is the same as #675 if I'm correct

@yoanisgil
Copy link
Author

@thibaultcha not really. As part of our performance tests we're just replaying our access logs in a way that they go through Kong. We then replay the same access log but this time we just contact the upstream API and hence we bypass Kong. On top of this we've Grafana which lets us compare response times and HTTP codes and it came to my attention that we have some 404 when we go through Kong. After further investigation I noticed that we have some requests in our log files which are already encoded, ex:

/v1/track/CR/101.226.89.123247c5a5055f4f1753a6ec25da8a49202/1662321.4049.CN.14465.1662321_3034_SPANK_UPL_CN_3034_10310_ADV27882_12897C_%2F_mobile?apikey=API_KEY

pay close attention to 2F ;). When this request goes through Kong it gets decoded before being sent to the upstream API server, which receives:

/v1/track/CR/101.226.89.123247c5a5055f4f1753a6ec25da8a49202/1662321.4049.CN.14465.1662321_3034_SPANK_UPL_CN_3034_10310_ADV27882_12897C_/_mobile?apikey=API_KEY

Do you see the difference? The %2F was decoded before being sent...

@thibaultcha
Copy link
Member

Sorry was on mobile.

It would seem at first glance that this is due to nginx's proxying. As pointed out in the docs, which were also referred to here and here, the URI part of the proxy_pass directive is URL decoded.

In our case, we have:

location / {
  # ...
  proxy_pass $backend_url;
  # ...
}

in the configuration, and in kong/resolver/access.lua:

-- Setting the backend URL for the proxy_pass directive
ngx.var.backend_url = get_backend_url(api)..uri

It would then seem like the computed proxy_pass is:

proxy_pass http://backend/uri

and as pointed out by those sources, the /uri part seems to be URL decoded by the nginx proxy module. That is my initial thought but we should:

  1. Define a simple setup to reproduce (just smaller URLs, one encoded, one not) and have an upstream server listening so we can monitor.
  2. If this ^ is confirmed, maybe not including the URI part in the ngx.var.backend_url could be a fix. I don't know if it would work.

I will try to tackle this when I'll have some time (which might not be anytime soon due to the Cassandra driver).

Thank you for spotting this btw! Nice find.

@yoanisgil
Copy link
Author

@thibaultcha I will take care of setting up a test scenario for reproducing this. This is kinda blocking for us, given that some of this calls are related to tracking sales/stats ... and we all know how important is to keep customers happy :). I will keep you posted.

@yoanisgil
Copy link
Author

@thibaultcha I have put together a test scenario which illustrates de issue we're facing, see here:

https://github.com/yoanisgil/kong-encode-request

The demo itself it's quite simple:

  • There is a container running an echo server using NodeJS (see here)
  • There is a Kong container which is configured to forward requests to the echo-server.
  • There is an nginx container configured to forward requests to the echo-server as welll.

When I call the echo server through Kong I see how %2F% gets decoded and then forwarded. When a direct call is made, or through nginx, the character %2F does not get decoded.

Bests,

Yoanis.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants