Merge branch 'master' into asgi-streaming

CHANGELOG.md

@@ -4,6 +4,21 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
+## 0.13.3 (May 29th, 2020)
+
+### Fixed
+
+* Include missing keepalive expiry configuration. (Pull #1005)
+* Improved error message when URL redirect has a custom scheme. (Pull #1002)
+
+## 0.13.2 (May 27th, 2020)
+
+### Fixed
+
+* Include explicit "Content-Length: 0" on POST, PUT, PATCH if no request body is used. (Pull #995)
+* Add `http2` option to `httpx.Client`. (Pull #982)
+* Tighten up API typing in places. (Pull #992, #999)
+
 ## 0.13.1 (May 22nd, 2020)
 
 ### Fixed

docs/api.md

@@ -151,7 +151,8 @@ True
 'application/json'
 ```
 
-* `def __init__(self, headers)`
+* `def __init__(self, headers, encoding=None)`
+* `def copy()` - **Headers**
 
 ## `Cookies`
 

docs/async.md

@@ -182,20 +182,3 @@ async with httpx.AsyncClient(transport=transport, base_url="http://testserver")
 ```
 
 See [the ASGI documentation](https://asgi.readthedocs.io/en/latest/specs/www.html#connection-scope) for more details on the `client` and `root_path` keys.
-
-## Unix Domain Sockets
-
-The async client provides support for connecting through a unix domain socket via the `uds` parameter. This is useful when making requests to a server that is bound to a socket file rather than an IP address.
-
-Here's an example requesting the Docker Engine API:
-
-```python
-import httpx
-
-
-async with httpx.AsyncClient(uds="/var/run/docker.sock") as client:
-    # This request will connect through the socket file.
-    resp = await client.get("http://localhost/version")
-```
-
-This functionality is not currently available in the synchronous client.

docs/contributing.md

@@ -84,7 +84,7 @@ Documentation pages are located under the `docs/` folder.
 To run the documentation site locally (useful for previewing changes), use:
 
 ```shell
-$ scripts/docs-serve
+$ scripts/docs
 ```
 
 ## Resolving Build / Travis Failures
@@ -93,15 +93,15 @@ Once you've submitted your pull request, the test suite will automatically run,
 If the test suite fails, you'll want to click through to the "Details" link, and try to identify why the test suite failed.
 
 <p align="center" style="margin: 0 0 10px">
-  <img src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/travis-fail.png" alt='Failing PR commit status'>
+  <img src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/gh-actions-fail.png" alt='Failing PR commit status'>
 </p>
 
 Here are some common ways the test suite can fail:
 
 ### Check Job Failed
 
 <p align="center" style="margin: 0 0 10px">
-  <img src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/travis-fail-check.png" alt='Failing Travis lint job'>
+  <img src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/gh-actions-fail-check.png" alt='Failing GitHub action lint job'>
 </p>
 
 This job failing means there is either a code formatting issue or type-annotation issue.
@@ -122,7 +122,7 @@ a variety of reasons like invalid markdown or missing configuration within `mkdo
 ### Python 3.X Job Failed
 
 <p align="center" style="margin: 0 0 10px">
-  <img src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/travis-fail-test.png" alt='Failing Travis test job'>
+  <img src="https://raw.githubusercontent.com/encode/httpx/master/docs/img/gh-actions-fail-test.png" alt='Failing GitHub action test job'>
 </p>
 
 This job failing means the unit tests failed or not all code paths are covered by unit tests.
@@ -131,13 +131,11 @@ If tests are failing you will see this message under the coverage report:
 
 `=== 1 failed, 435 passed, 1 skipped, 1 xfailed in 11.09s ===`
 
-If tests succeed but coverage isn't 100% you will see this message under the coverage report:
+If tests succeed but coverage doesn't reach our current threshold, you will see this
+message under the coverage report:
 
 `FAIL Required test coverage of 100% not reached. Total coverage: 99.00%`
 
-Look at the [coverage report from codecov](https://codecov.io/gh/encode/httpx/pulls)
-for the pull request for help debugging coverage.
-
 ## Releasing
 
 *This section is targeted at HTTPX maintainers.*
@@ -153,9 +151,19 @@ Before releasing a new version, create a pull request that includes:
         - Keep it concise and to-the-point. 🎯
 - **A version bump**: see `__version__.py`.
 
-For an example, see [#362](https://github.com/encode/httpx/pull/362).
+For an example, see [#1006](https://github.com/encode/httpx/pull/1006).
+
+Once the release PR is merged, create a
+[new release](https://github.com/encode/httpx/releases/new) including:
+
+- Tag version like `0.13.3`.
+- Release title `Version 0.13.3`
+- Description copied from the changelog.
+
+Once created this release will be automatically uploaded to PyPI.
 
-Once the release PR is merged, run `$ scripts/publish` to publish the new release to PyPI.
+If something goes wrong with the PyPI job the release can be published using the
+`scripts/publish` script.
 
 ## Development proxy setup
 

docs/quickstart.md

@@ -249,8 +249,8 @@ We can raise an exception for any Client or Server error responses (4xx or 5xx s
 >>> not_found.raise_for_status()
 Traceback (most recent call last):
   File "/Users/tomchristie/GitHub/encode/httpcore/httpx/models.py", line 776, in raise_for_status
-    raise HttpError(message)
-httpx.exceptions.HttpError: 404 Not Found
+    raise HTTPError(message)
+httpx.HTTPError: 404 Not Found
 ```
 
 Any successful response codes will simply return `None` rather than raising an exception.

httpx/__version__.py

@@ -1,3 +1,3 @@
 __title__ = "httpx"
 __description__ = "A next generation HTTP client, for Python 3."
-__version__ = "0.13.1"
+__version__ = "0.13.3"

httpx/_client.py

@@ -46,6 +46,8 @@
 
 logger = get_logger(__name__)
 
+KEEPALIVE_EXPIRY = 5.0
+
 
 class BaseClient:
     def __init__(
@@ -322,6 +324,10 @@ def redirect_url(self, request: Request, response: Response) -> URL:
 
         url = URL(location, allow_relative=True)
 
+        # Check that we can handle the scheme
+        if url.scheme and url.scheme not in ("http", "https"):
+            raise InvalidURL(f'Scheme "{url.scheme}" not supported.')
+
         # Handle malformed 'Location' headers that are "absolute" form, have no host.
         # See: https://github.com/encode/httpx/issues/771
         if url.scheme and not url.host:
@@ -513,6 +519,7 @@ def init_transport(
             ssl_context=ssl_context,
             max_keepalive=pool_limits.max_keepalive,
             max_connections=pool_limits.max_connections,
+            keepalive_expiry=KEEPALIVE_EXPIRY,
             http2=http2,
         )
 
@@ -536,6 +543,7 @@ def init_proxy_transport(
             ssl_context=ssl_context,
             max_keepalive=pool_limits.max_keepalive,
             max_connections=pool_limits.max_connections,
+            keepalive_expiry=KEEPALIVE_EXPIRY,
             http2=http2,
         )
 
@@ -1058,6 +1066,7 @@ def init_transport(
             ssl_context=ssl_context,
             max_keepalive=pool_limits.max_keepalive,
             max_connections=pool_limits.max_connections,
+            keepalive_expiry=KEEPALIVE_EXPIRY,
             http2=http2,
         )
 
@@ -1081,6 +1090,7 @@ def init_proxy_transport(
             ssl_context=ssl_context,
             max_keepalive=pool_limits.max_keepalive,
             max_connections=pool_limits.max_connections,
+            keepalive_expiry=KEEPALIVE_EXPIRY,
             http2=http2,
         )
 
@@ -1529,5 +1539,3 @@ async def __aexit__(
     ) -> None:
         assert isinstance(self.client, AsyncClient)
         await self.response.aclose()
-        if self.close_client:
-            await self.client.aclose()

httpx/_models.py

@@ -54,9 +54,6 @@
     warn_deprecated,
 )
 
-if typing.TYPE_CHECKING:  # pragma: no cover
-    from ._dispatch.base import AsyncDispatcher  # noqa: F401
-
 
 class URL:
     def __init__(
@@ -616,6 +613,9 @@ def prepare(self) -> None:
         auto_headers: typing.List[typing.Tuple[bytes, bytes]] = []
 
         has_host = "host" in self.headers
+        has_content_length = (
+            "content-length" in self.headers or "transfer-encoding" in self.headers
+        )
         has_user_agent = "user-agent" in self.headers
         has_accept = "accept" in self.headers
         has_accept_encoding = "accept-encoding" in self.headers
@@ -626,6 +626,8 @@ def prepare(self) -> None:
             if url.userinfo:
                 url = url.copy_with(username=None, password=None)
             auto_headers.append((b"host", url.authority.encode("ascii")))
+        if not has_content_length and self.method in ("POST", "PUT", "PATCH"):
+            auto_headers.append((b"content-length", b"0"))
         if not has_user_agent:
             auto_headers.append((b"user-agent", USER_AGENT.encode("ascii")))
         if not has_accept:
@@ -824,7 +826,7 @@ def is_redirect(self) -> bool:
 
     def raise_for_status(self) -> None:
         """
-        Raise the `HttpError` if one occurred.
+        Raise the `HTTPError` if one occurred.
         """
         message = (
             "{0.status_code} {error_type}: {0.reason_phrase} for url: {0.url}\n"

httpx/_transports/asgi.py

@@ -104,17 +104,17 @@ class ASGITransport(httpcore.AsyncHTTPTransport):
     client = httpx.AsyncClient(app=app)
     ```
 
-    Alternatively, you can setup the dispatch instance explicitly.
+    Alternatively, you can setup the transport instance explicitly.
     This allows you to include any additional configuration arguments specific
     to the ASGITransport class:
 
     ```
-    dispatch = httpx.ASGITransport(
+    transport = httpx.ASGITransport(
         app=app,
         root_path="/submount",
         client=("1.2.3.4", 123)
     )
-    client = httpx.AsyncClient(dispatch=dispatch)
+    client = httpx.AsyncClient(transport=transport)
     ```
 
     Arguments:

httpx/_transports/wsgi.py

@@ -25,17 +25,17 @@ class WSGITransport(httpcore.SyncHTTPTransport):
     client = httpx.Client(app=app)
     ```
 
-    Alternatively, you can setup the dispatch instance explicitly.
+    Alternatively, you can setup the transport instance explicitly.
     This allows you to include any additional configuration arguments specific
     to the WSGITransport class:
 
     ```
-    dispatch = httpx.WSGITransport(
+    transport = httpx.WSGITransport(
         app=app,
         script_name="/submount",
         remote_addr="1.2.3.4"
     )
-    client = httpx.Client(dispatch=dispatch)
+    client = httpx.Client(transport=transport)
     ```
 
     Arguments:

httpx/_types.py

@@ -24,7 +24,6 @@
     from ._config import Proxy, Timeout  # noqa: F401
     from ._models import URL, Cookies, Headers, QueryParams, Request  # noqa: F401
 
-StrOrBytes = Union[str, bytes]
 
 PrimitiveData = Optional[Union[str, int, float, bool]]
 
@@ -38,7 +37,11 @@
 ]
 
 HeaderTypes = Union[
-    "Headers", Dict[StrOrBytes, StrOrBytes], Sequence[Tuple[StrOrBytes, StrOrBytes]],
+    "Headers",
+    Dict[str, str],
+    Dict[bytes, bytes],
+    Sequence[Tuple[str, str]],
+    Sequence[Tuple[bytes, bytes]],
 ]
 
 CookieTypes = Union["Cookies", CookieJar, Dict[str, str]]

httpx/_utils.py

@@ -16,7 +16,7 @@
 from urllib.request import getproxies
 
 from ._exceptions import NetworkError
-from ._types import PrimitiveData, StrOrBytes
+from ._types import PrimitiveData
 
 if typing.TYPE_CHECKING:  # pragma: no cover
     from ._models import URL
@@ -31,7 +31,9 @@
 )
 
 
-def normalize_header_key(value: StrOrBytes, encoding: str = None) -> bytes:
+def normalize_header_key(
+    value: typing.Union[str, bytes], encoding: str = None
+) -> bytes:
     """
     Coerce str/bytes into a strictly byte-wise HTTP header key.
     """
@@ -40,7 +42,9 @@ def normalize_header_key(value: StrOrBytes, encoding: str = None) -> bytes:
     return value.encode(encoding or "ascii").lower()
 
 
-def normalize_header_value(value: StrOrBytes, encoding: str = None) -> bytes:
+def normalize_header_value(
+    value: typing.Union[str, bytes], encoding: str = None
+) -> bytes:
     """
     Coerce str/bytes into a strictly byte-wise HTTP header value.
     """
@@ -206,8 +210,8 @@ def parse_header_links(value: str) -> typing.List[typing.Dict[str, str]]:
 
 
 def obfuscate_sensitive_headers(
-    items: typing.Iterable[typing.Tuple[StrOrBytes, StrOrBytes]]
-) -> typing.Iterator[typing.Tuple[StrOrBytes, StrOrBytes]]:
+    items: typing.Iterable[typing.Tuple[typing.AnyStr, typing.AnyStr]]
+) -> typing.Iterator[typing.Tuple[typing.AnyStr, typing.AnyStr]]:
     for k, v in items:
         if to_str(k.lower()) in SENSITIVE_HEADERS:
             v = to_bytes_or_str("[secure]", match_type_of=v)
@@ -303,7 +307,7 @@ def to_str(value: typing.Union[str, bytes], encoding: str = "utf-8") -> str:
     return value if isinstance(value, str) else value.decode(encoding)
 
 
-def to_bytes_or_str(value: str, match_type_of: StrOrBytes) -> StrOrBytes:
+def to_bytes_or_str(value: str, match_type_of: typing.AnyStr) -> typing.AnyStr:
     return value if isinstance(match_type_of, str) else value.encode()
 
 

scripts/coverage

@@ -8,4 +8,4 @@ export SOURCE_FILES="httpx tests"
 
 set -x
 
-${PREFIX}coverage report --show-missing --skip-covered --fail-under=97
+${PREFIX}coverage report --show-missing --skip-covered --fail-under=99