allow for parameter mapping in path based routing #677
Comments
|
Native support for path parameters in Kong seems like a pretty significant missing piece. Right off the bat, I want to say that it is clear to me that Kong can proxy to path parameter based APIs. Let us say I have two backend GET APIs However, not having native support for path parameters is problematic. There is probably more to it but I see two resulting side effects of Kong not supporting path parameters: Kong BlindnessIn the above example, Kong is blind to the difference between these two APIs. Even though these are two very different APIS, Kong cannot tell them apart. This leads to problems with using plugins and other features of Kong effectively. Let us say that Kong did support path parameters and let me define these two APIs separately:
In my use case, I want to allow anonymous access to Additional complexity of collapsing multiple APIsAs a developer using Kong, I now have to parse all my API definitions and if there are situations like the above |
|
while this is a limitation of the functionality, i wouldn't say its a "significant missing piece" Kong aspires to be a solution for ALL API providers, and the described is just one of many abstractions and methods used. we'll have to balance that equally against other feature requests and with the community's feedback and support we can prioritize appropriately :) |
|
@ahmadnassri Path parameters are widely used and are even part of open standards like swagger. Completely ignoring them in an API middleware does not seem right. Even Amazon API Gateway has some support for them. I don't wish to be incessant about how we classify this :) but in the interest of community's feedback and support to help Kong developers prioritize, I'd certainly qualify it as a "significant missing piece". |
|
related: the official spec: RFC 6570 describing proper URL Templating usage. |
|
@ahmadnassri Any update on when we can expect this? |
|
+1 This would meet a major use-case/need of ours. |
|
+1 |
3 similar comments
|
+1 |
|
+1 |
|
+1 |
|
Wildcard/parameter mapping would be helpful. In my specific use case I'm looking to do something like:
|
|
+1 |
3 similar comments
|
+1 |
|
+1 |
|
+1 |
|
+1, really need it, we are stucked now with this missing feature and we have review our mircoservice architecture :'( The main problem is related to API depth in our services. For example, I can't have different microservice for an endpoint like |
|
+1 for RFC6570 |
|
+1 |
|
+1 |
1 similar comment
|
+1 |
|
@ahmadnassri is there any movement on this? We're currently implementing a work around with a nginx proxy in between kong and our services, but this is a less than ideal setup. We'd definitely prefer to have kong handle this path matching on it's own. If you could provide the likelihood or progress on this issue it'd aid us greatly in planning out our future direction. |
|
This is still on our roadmap but no work has been done as of today yet. Expect such a feature for 0.11 or 0.12, eventually. |
|
Any word on this? Kong is unusable for modern apis without this feature. It's 2016; any petty arguments about this are null and void, in my opinion. Almost every major api in the world uses url parameters. |
APIs can now receive user-specified regexes in their `uris` property. Those regexes will be treated as such by the router. A URI is considered a regex when its character set does not respect the reserved list from RFC 3986. Implements: #677 (partially)
We now perform efficient matching for URIs based on its property and the regex being evaluated. We favorise running `ngx.re.find` when the more expensive `ngx.re.match` is not necessary. When a regex URI has capture groups, we return them from the router `exec()` for further processing inside the core, which will be responsible for passing them along to plugins. This also has the advantage of being more performant on router cache hits, since URI stripping is done only once, at matching type, and cached in the LRU structure. The same way, groups matching is cached as well. Implements: #677 (partially)
APIs can now receive user-specified regexes in their `uris` property. Those regexes will be treated as such by the router. A URI is considered a regex when its character set does not respect the reserved list from RFC 3986. Implements: #677 (partially)
We now perform efficient matching for URIs based on its property and the regex being evaluated. We favorise running `ngx.re.find` when the more expensive `ngx.re.match` is not necessary. When a regex URI has capture groups, we return them from the router `exec()` for further processing inside the core, which will be responsible for passing them along to plugins. This also has the advantage of being more performant on router cache hits, since URI stripping is done only once, at matching type, and cached in the LRU structure. The same way, groups matching is cached as well. Implements: #677 (partially)
When a value from an API's `uris` property does not respect the reserved character set from RFC 3986, the router now assumes this value is a user-specified regex. User regexes are stored in another index than what we consider "URI prefixes" (non-regex `uris` values), because while the later needs to be evaluated based on their length, the same does not apply for the former. The order in which regexes are specified (and ultimately, APIs are registered) will be the order in which they will be evaluated. Because users might specify their own capturing groups once we support parameters extraction, the URI stripping now uses named capturing groups. This requires PCRE 7.2+. The original implementation used numbered capturing groups, but this is not aligned with our long-term goal of allowing dynamic URI rewriting as part of our Plugins API or request-transformer plugin. Breaking changes: requires PCRE 7.2+ Implements: #677 (partially)
APIs can now receive user-specified regexes in their `uris` property. Those regexes will be treated as such by the router. A URI is considered a regex when its character set does not respect the reserved list from RFC 3986. Implements: #677 (partially)
We now perform efficient matching for URIs based on its property and the regex being evaluated. We favorise running `ngx.re.find` when the more expensive `ngx.re.match` is not necessary. When a regex URI has capture groups, we return them from the router `exec()` for further processing inside the core, which will be responsible for passing them along to plugins. This also has the advantage of being more performant on router cache hits, since URI stripping is done only once, at matching type, and cached in the LRU structure. The same way, groups matching is cached as well. Implements: #677 (partially)
When a value from an API's `uris` property does not respect the reserved character set from RFC 3986, the router now assumes this value is a user-specified regex. User regexes are stored in another index than what we consider "URI prefixes" (non-regex `uris` values), because while the later needs to be evaluated based on their length, the same does not apply for the former. The order in which regexes are specified (and ultimately, APIs are registered) will be the order in which they will be evaluated. Because users might specify their own capturing groups once we support parameters extraction, the URI stripping now uses named capturing groups. This requires PCRE 7.2+. The original implementation used numbered capturing groups, but this is not aligned with our long-term goal of allowing dynamic URI rewriting as part of our Plugins API or request-transformer plugin. Breaking changes: requires PCRE 7.2+ Implements: #677 (partially)
APIs can now receive user-specified regexes in their `uris` property. Those regexes will be treated as such by the router. A URI is considered a regex when its character set does not respect the reserved list from RFC 3986. Implements: #677 (partially)
We now perform efficient matching for URIs based on its property and the regex being evaluated. We favorise running `ngx.re.find` when the more expensive `ngx.re.match` is not necessary. When a regex URI has capture groups, we return them from the router `exec()` for further processing inside the core, which will be responsible for passing them along to plugins. This also has the advantage of being more performant on router cache hits, since URI stripping is done only once, at matching type, and cached in the LRU structure. The same way, groups matching is cached as well. Implements: #677 (partially)
When a value from an API's `uris` property does not respect the reserved character set from RFC 3986, the router now assumes this value is a user-specified regex. User regexes are stored in another index than what we consider "URI prefixes" (non-regex `uris` values), because while the later needs to be evaluated based on their length, the same does not apply for the former. The order in which regexes are specified (and ultimately, APIs are registered) will be the order in which they will be evaluated. Because users might specify their own capturing groups once we support parameters extraction, the URI stripping now uses named capturing groups. This requires PCRE 7.2+. The original implementation used numbered capturing groups, but this is not aligned with our long-term goal of allowing dynamic URI rewriting as part of our Plugins API or request-transformer plugin. Breaking changes: requires PCRE 7.2+ Implements: #677 (partially)
APIs can now receive user-specified regexes in their `uris` property. Those regexes will be treated as such by the router. A URI is considered a regex when its character set does not respect the reserved list from RFC 3986. Implements: #677 (partially)
We now perform efficient matching for URIs based on its property and the regex being evaluated. We favorise running `ngx.re.find` when the more expensive `ngx.re.match` is not necessary. When a regex URI has capture groups, we return them from the router `exec()` for further processing inside the core, which will be responsible for passing them along to plugins. This also has the advantage of being more performant on router cache hits, since URI stripping is done only once, at matching type, and cached in the LRU structure. The same way, groups matching is cached as well. Implements: #677 (partially)
|
Hi @thibaultcha! First of all - thank so much for working on it. Do I understand correctly that this feature allows dynamically building upstream URLs from available data such as consumer_ids? For instance, when using auth plugin, consumer_id is sent currently in the header to upstream API. Will this allow me to dynamically build upstream routes using this variable like that: Kind regards. |
|
@rdzak Hi,
It does not. This work enabled URIs regexp matching and capturing groups extraction. It then exposes the captured groups to the plugins for the lifecycle of the request. It is now up to the plugin to be updated and to use those capturing groups to do something. That is beyond the scope of the proposed PR, and still in the scope of this issue. We are working on it. Additionally, if the value you want to use comes from the request headers, then it is not dependent on the regexp PR in any way, but entirely on the request-transformer plugin.
We have an RC 2 coming up next week, and stable 0.11 in the upcoming weeks if RC 2 is successful. |
|
Hi everyone! As of today, all the work that we planned for this so far has been released in 0.11.0. So, before closing the conversation here, I would like to recap a few things. This issue actually described two features:
As of today, 1. is available starting from Kong 0.11.0 🎉, and 2. is available as part of our Mashape Enterprise subscription. In our most recent release for our Enterprise customers, we've improved the You can find more information about our Mashape Enterprise subscription and its additional features on top of Kong at: https://mashape.com/enterprise. As a "bonus feature", I would like to highlight that Kong 0.11.0 also includes:
This should allow for some good hacking sessions in your Kong custom plugins 😉 With that said, I believe we can now close this conversation and consider it implemented under the scope provided by this issue! You can checkout Kong 0.11.0 here: https://github.com/Mashape/kong/releases/tag/0.11.0 Thanks! |
|
At least, we don't forget that kong is also an entreprise product 🤔 Thx for the work, this feature really change the game for us. For the first time we can purpose your product to our customer because it fits our development. 💌 I hope that the next big step will be an integration of Swagger (at least params/body/query sync validation) 👍 |
|
What is the correct way to do a regex in a uris attribute, when i do it via curl it does not yield the results im after: |
|
Is it possible to use this feature with QueryString parameters? I'm having issue with getting the regex to match the "?" character in the request uri. I want to set up 2 apis with the only difference being the presence of the query string (such as "?debug=true") is added. Example (get user by ID with debug querystring) I've attempted to escape and double escape the '?' but it seems to be ignored. If I use another character such as '&', everything works fine. There seems to be some issue with the reserved regex characters. I've also tried using the url encoded "%3F" but that doesn't get sent properly to my service and it does not pick up the query string parameters. |
|
@techtyler Copying here my answer to you from Gitter for future reference to this thread (please avoid cross-posintg in the future): Also, I will now be locking this issue to avoid further noise. The usual support channels (Gitter/Freenode/Mailing list) are the preferred channel of communication for getting help and asking questions. Today, GitHub issues are only looked at when actually reporting bugs. Thank you for your cooperation! |
and others
e.g. allow
request_pathto contain values such as/api/{foo}/barwhich map to theupstream_url/upstream/my/api/bar/{foo}this could also be more globally addressed with regex matching groups e.g.
The text was updated successfully, but these errors were encountered: