Do not use translated path #3051

william-gr · 2018-06-04T17:57:59Z

What do these changes do?

Allow redirect to work when using url encoded paths and normalize path middleware, e.g.

/foo/bar%2Fbar will redirect to /foo/bar%2Fbar/ instead of /foo/bar/bar/

Are there changes in behavior for the user?

Not as far as I am aware, since its a bug fix.

codecov-io · 2018-06-04T18:15:45Z

Codecov Report

Merging #3051 into master will not change coverage.
The diff coverage is 100%.

@@           Coverage Diff           @@
##           master    #3051   +/-   ##
=======================================
  Coverage   98.11%   98.11%           
=======================================
  Files          42       42           
  Lines        7743     7743           
  Branches     1349     1349           
=======================================
  Hits         7597     7597           
  Misses         51       51           
  Partials       95       95

Impacted Files	Coverage Δ
aiohttp/web_middlewares.py	`100% <100%> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e641373...b45080c. Read the comment docs.

webknjaz · 2018-06-04T18:18:58Z

Please fill out the PR template, so that it would be easier to understand why and what you are fixing.

asvetlov · 2018-06-05T12:48:00Z

Sorry, I don't follow what is failed and what is expected behavior.

Unit test and changelog entry are required.

webknjaz · 2018-06-05T14:48:46Z

https://raw.githubusercontent.com/aio-libs/aiohttp/master/.github/PULL_REQUEST_TEMPLATE.md

william-gr · 2018-06-05T18:08:55Z

@asvetlov @webknjaz PR updated and summary changed to follow template.

Let me know if something is missing.

Thanks

Redirect would fail in case we have encoded url, e.g. /foo/bar%2Fbar -> /foo/bar/bar/

william-gr · 2018-06-07T15:58:03Z

Ping. Just want to make sure there isn't anything missing here. I will be travelling the next few days and want to leave it ready for a review.

asvetlov · 2018-06-07T19:29:10Z

aiohttp/web_middlewares.py

@@ -73,7 +73,7 @@ def normalize_path_middleware(
                resolves, request = await _check_request_resolves(
                    request, path)
                if resolves:
-                    raise redirect_class(request.path + query)


I wonder if request.path can be replaced with request.raw_path

Thats the same as path, no? request.raw_path is being used above to populate paths_to_check.

No. It is the percent encoded canonical path representation.

And thats exactly the issue I am solving here, and its passing all the tests so I dont get it.

So replace the call please

william-gr · 2018-06-08T11:19:56Z

@asvetlov call updated. Thanks for the helpful explanation as of why.

asvetlov · 2018-06-09T09:53:20Z

thanks

================== Features -------- - Add type hints (`3049 <https://github.com/aio-libs/aiohttp/pull/3049>`_) - Add ``raise_for_status`` request parameter (`3073 <https://github.com/aio-libs/aiohttp/pull/3073>`_) - Add type hints to HTTP client (`3092 <https://github.com/aio-libs/aiohttp/pull/3092>`_) - Minor server optimizations (`3095 <https://github.com/aio-libs/aiohttp/pull/3095>`_) - Preserve the cause when `HTTPException` is raised from another exception. (`3096 <https://github.com/aio-libs/aiohttp/pull/3096>`_) - Add `close_boundary` option in `MultipartWriter.write` method. Support streaming (`3104 <https://github.com/aio-libs/aiohttp/pull/3104>`_) - Added a ``remove_slash`` option to the ``normalize_path_middleware`` factory. (`3173 <https://github.com/aio-libs/aiohttp/pull/3173>`_) - The class `AbstractRouteDef` is importable from `aiohttp.web`. (`3183 <https://github.com/aio-libs/aiohttp/pull/3183>`_) Bugfixes -------- - Prevent double closing when client connection is released before the last ``data_received()`` callback. (`3031 <https://github.com/aio-libs/aiohttp/pull/3031>`_) - Make redirect with `normalize_path_middleware` work when using url encoded paths. (`3051 <https://github.com/aio-libs/aiohttp/pull/3051>`_) - Postpone web task creation to connection establishment. (`3052 <https://github.com/aio-libs/aiohttp/pull/3052>`_) - Fix ``sock_read`` timeout. (`3053 <https://github.com/aio-libs/aiohttp/pull/3053>`_) - When using a server-request body as the `data=` argument of a client request, iterate over the content with `readany` instead of `readline` to avoid `Line too long` errors. (`3054 <https://github.com/aio-libs/aiohttp/pull/3054>`_) - fix `UrlDispatcher` has no attribute `add_options`, add `web.options` (`3062 <https://github.com/aio-libs/aiohttp/pull/3062>`_) - correct filename in content-disposition with multipart body (`3064 <https://github.com/aio-libs/aiohttp/pull/3064>`_) - Many HTTP proxies has buggy keepalive support. Let's not reuse connection but close it after processing every response. (`3070 <https://github.com/aio-libs/aiohttp/pull/3070>`_) - raise 413 "Payload Too Large" rather than raising ValueError in request.post() Add helpful debug message to 413 responses (`3087 <https://github.com/aio-libs/aiohttp/pull/3087>`_) - Fix `StreamResponse` equality, now that they are `MutableMapping` objects. (`3100 <https://github.com/aio-libs/aiohttp/pull/3100>`_) - Fix server request objects comparison (`3116 <https://github.com/aio-libs/aiohttp/pull/3116>`_) - Do not hang on `206 Partial Content` response with `Content-Encoding: gzip` (`3123 <https://github.com/aio-libs/aiohttp/pull/3123>`_) - Fix timeout precondition checkers (`3145 <https://github.com/aio-libs/aiohttp/pull/3145>`_) Improved Documentation ---------------------- - Add a new FAQ entry that clarifies that you should not reuse response objects in middleware functions. (`3020 <https://github.com/aio-libs/aiohttp/pull/3020>`_) - Add FAQ section "Why is creating a ClientSession outside of an event loop dangerous?" (`3072 <https://github.com/aio-libs/aiohttp/pull/3072>`_) - Fix link to Rambler (`3115 <https://github.com/aio-libs/aiohttp/pull/3115>`_) - Fix TCPSite documentation on the Server Reference page. (`3146 <https://github.com/aio-libs/aiohttp/pull/3146>`_) - Fix documentation build configuration file for Windows. (`3147 <https://github.com/aio-libs/aiohttp/pull/3147>`_) - Remove no longer existing lingering_timeout parameter of Application.make_handler from documentation. (`3151 <https://github.com/aio-libs/aiohttp/pull/3151>`_) - Mention that ``app.make_handler`` is deprecated, recommend to use runners API instead. (`3157 <https://github.com/aio-libs/aiohttp/pull/3157>`_) Deprecations and Removals ------------------------- - Drop ``loop.current_task()`` from ``helpers.current_task()`` (`2826 <https://github.com/aio-libs/aiohttp/pull/2826>`_) - Drop ``reader`` parameter from ``request.multipart()``. (`3090 <https://github.com/aio-libs/aiohttp/pull/3090>`_)

This PR updates [aiohttp](https://pypi.org/project/aiohttp) from **3.3.2** to **3.4.4**. <details> <summary>Changelog</summary> ### 3.4.4 ``` ================== - Fix installation from sources when compiling toolkit is not available (`3241 <https://github.com/aio-libs/aiohttp/pull/3241>`_) ``` ### 3.4.3 ``` ================== - Add ``app.pre_frozen`` state to properly handle startup signals in sub-applications. (`3237 <https://github.com/aio-libs/aiohttp/pull/3237>`_) ``` ### 3.4.2 ``` ================== - Fix ``iter_chunks`` type annotation (`3230 <https://github.com/aio-libs/aiohttp/pull/3230>`_) ``` ### 3.4.1 ``` ================== - Fix empty header parsing regression. (`3218 <https://github.com/aio-libs/aiohttp/pull/3218>`_) - Fix BaseRequest.raw_headers doc. (`3215 <https://github.com/aio-libs/aiohttp/pull/3215>`_) - Fix documentation building on ReadTheDocs (`3221 <https://github.com/aio-libs/aiohttp/pull/3221>`_) ``` ### 3.4.0 ``` ================== Features -------- - Add type hints (`3049 <https://github.com/aio-libs/aiohttp/pull/3049>`_) - Add ``raise_for_status`` request parameter (`3073 <https://github.com/aio-libs/aiohttp/pull/3073>`_) - Add type hints to HTTP client (`3092 <https://github.com/aio-libs/aiohttp/pull/3092>`_) - Minor server optimizations (`3095 <https://github.com/aio-libs/aiohttp/pull/3095>`_) - Preserve the cause when `HTTPException` is raised from another exception. (`3096 <https://github.com/aio-libs/aiohttp/pull/3096>`_) - Add `close_boundary` option in `MultipartWriter.write` method. Support streaming (`3104 <https://github.com/aio-libs/aiohttp/pull/3104>`_) - Added a ``remove_slash`` option to the ``normalize_path_middleware`` factory. (`3173 <https://github.com/aio-libs/aiohttp/pull/3173>`_) - The class `AbstractRouteDef` is importable from `aiohttp.web`. (`3183 <https://github.com/aio-libs/aiohttp/pull/3183>`_) Bugfixes -------- - Prevent double closing when client connection is released before the last ``data_received()`` callback. (`3031 <https://github.com/aio-libs/aiohttp/pull/3031>`_) - Make redirect with `normalize_path_middleware` work when using url encoded paths. (`3051 <https://github.com/aio-libs/aiohttp/pull/3051>`_) - Postpone web task creation to connection establishment. (`3052 <https://github.com/aio-libs/aiohttp/pull/3052>`_) - Fix ``sock_read`` timeout. (`3053 <https://github.com/aio-libs/aiohttp/pull/3053>`_) - When using a server-request body as the `data=` argument of a client request, iterate over the content with `readany` instead of `readline` to avoid `Line too long` errors. (`3054 <https://github.com/aio-libs/aiohttp/pull/3054>`_) - fix `UrlDispatcher` has no attribute `add_options`, add `web.options` (`3062 <https://github.com/aio-libs/aiohttp/pull/3062>`_) - correct filename in content-disposition with multipart body (`3064 <https://github.com/aio-libs/aiohttp/pull/3064>`_) - Many HTTP proxies has buggy keepalive support. Let's not reuse connection but close it after processing every response. (`3070 <https://github.com/aio-libs/aiohttp/pull/3070>`_) - raise 413 "Payload Too Large" rather than raising ValueError in request.post() Add helpful debug message to 413 responses (`3087 <https://github.com/aio-libs/aiohttp/pull/3087>`_) - Fix `StreamResponse` equality, now that they are `MutableMapping` objects. (`3100 <https://github.com/aio-libs/aiohttp/pull/3100>`_) - Fix server request objects comparison (`3116 <https://github.com/aio-libs/aiohttp/pull/3116>`_) - Do not hang on `206 Partial Content` response with `Content-Encoding: gzip` (`3123 <https://github.com/aio-libs/aiohttp/pull/3123>`_) - Fix timeout precondition checkers (`3145 <https://github.com/aio-libs/aiohttp/pull/3145>`_) Improved Documentation ---------------------- - Add a new FAQ entry that clarifies that you should not reuse response objects in middleware functions. (`3020 <https://github.com/aio-libs/aiohttp/pull/3020>`_) - Add FAQ section "Why is creating a ClientSession outside of an event loop dangerous?" (`3072 <https://github.com/aio-libs/aiohttp/pull/3072>`_) - Fix link to Rambler (`3115 <https://github.com/aio-libs/aiohttp/pull/3115>`_) - Fix TCPSite documentation on the Server Reference page. (`3146 <https://github.com/aio-libs/aiohttp/pull/3146>`_) - Fix documentation build configuration file for Windows. (`3147 <https://github.com/aio-libs/aiohttp/pull/3147>`_) - Remove no longer existing lingering_timeout parameter of Application.make_handler from documentation. (`3151 <https://github.com/aio-libs/aiohttp/pull/3151>`_) - Mention that ``app.make_handler`` is deprecated, recommend to use runners API instead. (`3157 <https://github.com/aio-libs/aiohttp/pull/3157>`_) Deprecations and Removals ------------------------- - Drop ``loop.current_task()`` from ``helpers.current_task()`` (`2826 <https://github.com/aio-libs/aiohttp/pull/2826>`_) - Drop ``reader`` parameter from ``request.multipart()``. (`3090 <https://github.com/aio-libs/aiohttp/pull/3090>`_) ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/aiohttp - Changelog: https://pyup.io/changelogs/aiohttp/ - Repo: https://github.com/aio-libs/aiohttp </details>

lock · 2019-10-28T04:04:05Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a [new issue] for related bugs.
If you feel like there's important points made in this discussion, please include those exceprts into that [new issue].
[new issue]: https://github.com/aio-libs/aiohttp/issues/new

william-gr force-pushed the issues/normalize_raw branch from 1e4914a to 8e7590a Compare June 5, 2018 18:06

Do not use translated path

b6fd755

Redirect would fail in case we have encoded url, e.g. /foo/bar%2Fbar -> /foo/bar/bar/

william-gr force-pushed the issues/normalize_raw branch from 8e7590a to b6fd755 Compare June 5, 2018 18:10

asvetlov reviewed Jun 7, 2018

View reviewed changes

Use request.raw_path as mandated by @asvetlov

b45080c

asvetlov approved these changes Jun 9, 2018

View reviewed changes

asvetlov merged commit 4b54daf into aio-libs:master Jun 9, 2018

william-gr deleted the issues/normalize_raw branch June 12, 2018 16:04

dependencies bot mentioned this pull request Aug 26, 2018

aiohttp versions available: 3.4.0 Harmon758/Harmonbot#301

Closed

lock bot added the outdated label Oct 28, 2019

lock bot locked as resolved and limited conversation to collaborators Oct 28, 2019

psf-chronographer bot added the bot:chronographer:provided There is a change note present in this PR label Oct 28, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Do not use translated path #3051

Do not use translated path #3051

william-gr commented Jun 4, 2018 •

edited

Loading

codecov-io commented Jun 4, 2018 •

edited

Loading

webknjaz commented Jun 4, 2018

asvetlov commented Jun 5, 2018

webknjaz commented Jun 5, 2018

william-gr commented Jun 5, 2018 •

edited

Loading

william-gr commented Jun 7, 2018

asvetlov Jun 7, 2018

william-gr Jun 7, 2018

asvetlov Jun 7, 2018

william-gr Jun 7, 2018

asvetlov Jun 8, 2018

william-gr commented Jun 8, 2018

asvetlov commented Jun 9, 2018

lock bot commented Oct 28, 2019

Do not use translated path #3051

Do not use translated path #3051

Conversation

william-gr commented Jun 4, 2018 • edited Loading

What do these changes do?

Are there changes in behavior for the user?

codecov-io commented Jun 4, 2018 • edited Loading

Codecov Report

webknjaz commented Jun 4, 2018

asvetlov commented Jun 5, 2018

webknjaz commented Jun 5, 2018

william-gr commented Jun 5, 2018 • edited Loading

william-gr commented Jun 7, 2018

asvetlov Jun 7, 2018

Choose a reason for hiding this comment

william-gr Jun 7, 2018

Choose a reason for hiding this comment

asvetlov Jun 7, 2018

Choose a reason for hiding this comment

william-gr Jun 7, 2018

Choose a reason for hiding this comment

asvetlov Jun 8, 2018

Choose a reason for hiding this comment

william-gr commented Jun 8, 2018

asvetlov commented Jun 9, 2018

lock bot commented Oct 28, 2019

william-gr commented Jun 4, 2018 •

edited

Loading

codecov-io commented Jun 4, 2018 •

edited

Loading

william-gr commented Jun 5, 2018 •

edited

Loading