Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

21.3deprecations #1998

Closed
wants to merge 106 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
106 commits
Select commit Hold shift + click to select a range
6279eac
Streaming request by async for.
Tronic Feb 21, 2020
fe64a27
Make all requests streaming and preload body for non-streaming handlers.
Tronic Feb 21, 2020
f609a48
Cleanup of code and avoid mixing streaming responses.
Tronic Feb 21, 2020
f6a0b4a
Async http protocol loop.
Tronic Feb 24, 2020
3d05e1e
Change of test: don't require early bad request error but only after …
Tronic Feb 24, 2020
ef4f233
Add back streaming requests.
Tronic Feb 24, 2020
42d86bc
Rewritten request body parser.
Tronic Feb 26, 2020
6b9f0ec
Misc. cleanup, down to 4 failing tests.
Tronic Feb 26, 2020
b87364b
All tests OK.
Tronic Feb 26, 2020
29c6f3c
Entirely remove request body queue.
Tronic Feb 26, 2020
6d8f598
Let black f*ckup the layout
Tronic Feb 26, 2020
b2476bd
Better testing error messages on protocol errors.
Tronic Feb 28, 2020
d597137
Remove StreamBuffer tests because the type is about to be removed.
Tronic Feb 28, 2020
85c67a0
Remove tests using the deprecated get_headers function that can no lo…
Tronic Feb 28, 2020
85b1ad5
Major refactoring of HTTP protocol handling (new module http.py added…
Tronic Feb 28, 2020
8a1baeb
Terminate check_timeouts once connection_task finishes.
Tronic Feb 28, 2020
57202bf
Code cleanup, 14 tests failing.
Tronic Feb 29, 2020
a553e64
Much cleanup, 12 failing...
Tronic Feb 29, 2020
7f41c5f
Even more cleanup and error checking, 8 failing tests.
Tronic Feb 29, 2020
85be576
Remove keep-alive header from responses. First of all, it should say …
Tronic Mar 1, 2020
2840e4c
Everything but CustomServer OK.
Tronic Mar 1, 2020
f829893
Merge branch 'master' into streaming
Tronic Mar 1, 2020
5086076
Linter
Tronic Mar 1, 2020
fc16594
Disable custom protocol test
Tronic Mar 1, 2020
c0a0b50
Remove unnecessary variables, optimise performance.
Tronic Mar 1, 2020
5a96996
A test was missing that body_init/body_push/body_finish are never cal…
Tronic Mar 1, 2020
1c42a5e
Minor fixes.
Tronic Mar 1, 2020
0712026
Remove unused code.
Tronic Mar 1, 2020
d918655
Py 3.8 check for deprecated loop argument.
Tronic Mar 1, 2020
31a8706
Fix a middleware cancellation handling test with py38.
Tronic Mar 1, 2020
0835363
Linter 'n fixes
Tronic Mar 1, 2020
7e93ee1
Typing
Tronic Mar 1, 2020
9baa241
Merge branch 'master' into streaming
Tronic Mar 2, 2020
50cca39
Stricter handling of request header size
Tronic Mar 2, 2020
e73f26b
More specific error messages on Payload Too Large.
Tronic Mar 2, 2020
96a8b5c
Init http.response = None
Tronic Mar 2, 2020
85d58d7
Messages further tuned.
Tronic Mar 2, 2020
9c21457
Always try to consume request body, plus minor cleanup.
Tronic Mar 2, 2020
c2e5674
Add a missing check in case of close_if_idle on a dead connection.
Tronic Mar 2, 2020
cbabe7e
Avoid error messages on PayloadTooLarge.
Tronic Mar 2, 2020
dc6b492
Add test for new API.
Tronic Mar 4, 2020
eb66621
json takes str, not bytes
Tronic Mar 4, 2020
730de6a
Default to no maximum request size for streaming handlers.
Tronic Mar 4, 2020
4c34508
Merge branch 'master' into streaming
Tronic Mar 6, 2020
cbfeb1c
Fix chunked mode crash.
Tronic Mar 8, 2020
990ac52
Header values should be strictly ASCII but both UTF-8 and Latin-1 exi…
Tronic Mar 8, 2020
d348bb4
Refactoring and cleanup.
Tronic Mar 8, 2020
5351cda
Unify response header processing of ASGI and asyncio modes.
Tronic Mar 8, 2020
c86c29e
Avoid special handling of StreamingHTTPResponse.
Tronic Mar 8, 2020
a0e61ae
35 % speedup in HTTP/1.1 response formatting (not so much overall eff…
Tronic Mar 9, 2020
32ee539
Duplicate set-cookie headers were being produced.
Tronic Mar 9, 2020
a9d984e
Cleanup processed_headers some more.
Tronic Mar 9, 2020
9dc2ec9
Linting
Tronic Mar 9, 2020
2b63d2b
Import ordering
Tronic Mar 9, 2020
2adcc72
Response middleware ran by async request.respond().
Tronic Mar 9, 2020
d2d6008
Need to check if transport is closing to avoid getting stuck in sendi…
Tronic Mar 10, 2020
17d1004
Middleware and error handling refactoring.
Tronic Mar 10, 2020
9098493
Linter
Tronic Mar 10, 2020
23e54fc
Fix tracking of HTTP stage when writing to transport fails.
Tronic Mar 20, 2020
f1c85eb
Add clarifying comment
Tronic Mar 22, 2020
42af6e1
Add a check for request body functions and a test for NotImplementedE…
Tronic Mar 25, 2020
5832764
Merge branch 'master' into streaming
Tronic Mar 25, 2020
6bedec9
Linter and typing
Tronic Mar 25, 2020
f928ad2
These must be tuples + hack mypy warnings away.
Tronic Mar 25, 2020
1aac4f5
Merge branch 'master' into streaming
Tronic Mar 26, 2020
01480c4
New streaming test and minor fixes.
Tronic Mar 26, 2020
e4a9b43
Constant receive buffer size.
Tronic Mar 26, 2020
abc1e3e
256 KiB send and receive buffers.
Tronic Mar 26, 2020
39d10bf
Revert "256 KiB send and receive buffers."
Tronic Mar 26, 2020
c84163f
app.handle_exception already sends the response.
Tronic Mar 20, 2020
fcae70b
Improved handling of errors during request.
Tronic Mar 21, 2020
232177d
An odd hack to avoid an httpx limitation that causes test failures.
Tronic Mar 26, 2020
b6a4bd3
Merge branch 'master' into streaming
Tronic Mar 29, 2020
95b8b7c
Merge branch 'master' into streaming
Tronic Apr 7, 2020
9b6f259
Merge branch 'master' into streaming
Tronic Apr 9, 2020
36ffaa7
Merge branch 'master' into streaming
ahopkins May 13, 2020
ef10275
Merge branch 'master' into streaming
ahopkins Jun 3, 2020
149f943
Limit request header size to 8 KiB at most.
Tronic Jun 4, 2020
6dfb653
Merge branch 'master' into streaming
Tronic Jun 4, 2020
26f7e57
Merge branch 'master' into streaming
Tronic Jun 21, 2020
4fd493e
Remove unnecessary use of format string.
Tronic Jun 21, 2020
dd87583
Merge branch 'master' into streaming
Tronic Jun 28, 2020
742bfa1
Bring back to current master
ahopkins Oct 24, 2020
5c9a9c7
Merge in master
ahopkins Dec 29, 2020
310442e
Cleanup tests
ahopkins Dec 29, 2020
e9056bb
Remove artifact
ahopkins Dec 29, 2020
1c9ad0d
Fix type checking
ahopkins Dec 29, 2020
335637f
Mark test for skipping
ahopkins Dec 29, 2020
86722a6
Cleanup some edge cases
ahopkins Dec 29, 2020
a4e3c77
Add ignore_body flag to safe methods
ahopkins Dec 29, 2020
b9740bd
Add unit tests for timeout logic
ahopkins Dec 30, 2020
4ec61ea
Add unit tests for timeout logic
ahopkins Dec 30, 2020
ad254c9
Fix Mock usage in timeout test
ahopkins Dec 30, 2020
8a39604
Change logging test to only logger in handler
ahopkins Dec 30, 2020
529d595
Windows py3.8 logging issue with current testing client
ahopkins Dec 30, 2020
9d70b0d
Add test_header_size_exceeded
ahopkins Dec 30, 2020
8f44347
Merge in master
ahopkins Jan 7, 2021
7dc6c77
Resolve merge conflicts
ahopkins Jan 10, 2021
468a0dd
Add request middleware to hard exception handling
ahopkins Jan 10, 2021
01e6be7
Add request middleware to hard exception handling
ahopkins Jan 10, 2021
668ffba
Request middleware on exception handlers
ahopkins Jan 10, 2021
904d420
Merge branch 'master' of github.com:sanic-org/sanic into streaming
ahopkins Jan 10, 2021
41a0a94
Linting
ahopkins Jan 10, 2021
14753c7
Cleanup deprecations
ahopkins Jan 10, 2021
a2bff74
Remove config deprecations and fix config compat
ahopkins Jan 10, 2021
ef9a041
Merge branch 'master' of github.com:sanic-org/sanic into 21.3deprecat…
ahopkins Jan 10, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 16 additions & 21 deletions sanic/config.py
Original file line number Diff line number Diff line change
@@ -1,14 +1,8 @@
from inspect import isclass
from os import environ
from pathlib import Path
from typing import Any, Union

# NOTE(tomaszdrozdz): remove in version: 21.3
# We replace from_envvar(), from_object(), from_pyfile() config object methods
# with one simpler update_config() method.
# We also replace "loading module from file code" in from_pyfile()
# in a favour of load_module_from_file_location().
# Please see pull request: 1903
# and issue: 1895
from .deprecated import from_envvar, from_object, from_pyfile # noqa
from .utils import load_module_from_file_location, str_to_bool


Expand Down Expand Up @@ -68,17 +62,6 @@ def __getattr__(self, attr):
def __setattr__(self, attr, value):
self[attr] = value

# NOTE(tomaszdrozdz): remove in version: 21.3
# We replace from_envvar(), from_object(), from_pyfile() config object
# methods with one simpler update_config() method.
# We also replace "loading module from file code" in from_pyfile()
# in a favour of load_module_from_file_location().
# Please see pull request: 1903
# and issue: 1895
from_envvar = from_envvar
from_pyfile = from_pyfile
from_object = from_object

def load_environment_vars(self, prefix=SANIC_PREFIX):
"""
Looks for prefixed environment variables and applies
Expand Down Expand Up @@ -129,12 +112,24 @@ class C:
B = 2
config.update_config(C)"""

if isinstance(config, (bytes, str)):
if isinstance(config, (bytes, str, Path)):
config = load_module_from_file_location(location=config)

if not isinstance(config, dict):
config = config.__dict__
cfg = {}
if not isclass(config):
cfg.update(
{
key: getattr(config, key)
for key in config.__class__.__dict__.keys()
}
)

config = dict(config.__dict__)
config.update(cfg)

config = dict(filter(lambda i: i[0].isupper(), config.items()))

self.update(config)

load = update_config
106 changes: 0 additions & 106 deletions sanic/deprecated.py

This file was deleted.

95 changes: 63 additions & 32 deletions sanic/utils.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,13 @@
import types

from importlib.util import module_from_spec, spec_from_file_location
from os import environ as os_environ
from pathlib import Path
from re import findall as re_findall
from typing import Union

from .exceptions import LoadFileException
from sanic.exceptions import LoadFileException, PyFileError
from sanic.helpers import import_string


def str_to_bool(val: str) -> bool:
Expand Down Expand Up @@ -39,7 +43,7 @@ def str_to_bool(val: str) -> bool:


def load_module_from_file_location(
location: Union[bytes, str], encoding: str = "utf8", *args, **kwargs
location: Union[bytes, str, Path], encoding: str = "utf8", *args, **kwargs
):
"""Returns loaded module provided as a file path.

Expand Down Expand Up @@ -67,33 +71,60 @@ def load_module_from_file_location(
"/some/path/${some_env_var}"
)
"""

# 1) Parse location.
if isinstance(location, bytes):
location = location.decode(encoding)

# A) Check if location contains any environment variables
# in format ${some_env_var}.
env_vars_in_location = set(re_findall(r"\${(.+?)}", location))

# B) Check these variables exists in environment.
not_defined_env_vars = env_vars_in_location.difference(os_environ.keys())
if not_defined_env_vars:
raise LoadFileException(
"The following environment variables are not set: "
f"{', '.join(not_defined_env_vars)}"
)

# C) Substitute them in location.
for env_var in env_vars_in_location:
location = location.replace("${" + env_var + "}", os_environ[env_var])

# 2) Load and return module.
name = location.split("/")[-1].split(".")[
0
] # get just the file name without path and .py extension
_mod_spec = spec_from_file_location(name, location, *args, **kwargs)
module = module_from_spec(_mod_spec)
_mod_spec.loader.exec_module(module) # type: ignore

return module
if isinstance(location, Path) or "/" in location or "$" in location:
if isinstance(location, bytes):
location = location.decode(encoding)

if not isinstance(location, Path):
# A) Check if location contains any environment variables
# in format ${some_env_var}.
env_vars_in_location = set(re_findall(r"\${(.+?)}", location))

# B) Check these variables exists in environment.
not_defined_env_vars = env_vars_in_location.difference(
os_environ.keys()
)
if not_defined_env_vars:
raise LoadFileException(
"The following environment variables are not set: "
f"{', '.join(not_defined_env_vars)}"
)

# C) Substitute them in location.
for env_var in env_vars_in_location:
location = location.replace(
"${" + env_var + "}", os_environ[env_var]
)

location = str(location)
if ".py" in location:
name = location.split("/")[-1].split(".")[
0
] # get just the file name without path and .py extension
_mod_spec = spec_from_file_location(
name, location, *args, **kwargs
)
module = module_from_spec(_mod_spec)
_mod_spec.loader.exec_module(module) # type: ignore

else:
module = types.ModuleType("config")
module.__file__ = str(location)
try:
with open(location) as config_file:
exec( # nosec
compile(config_file.read(), location, "exec"),
module.__dict__,
)
except IOError as e:
e.strerror = "Unable to load configuration file (e.strerror)"
raise
except Exception as e:
raise PyFileError(location) from e

return module
else:
try:
return import_string(location)
except ValueError:
raise IOError("Unable to load configuration %s" % str(location))
Loading