Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds upload_files functionality for request transport #244

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions docs/usage/file_upload.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ File uploads
============

GQL supports file uploads with the :ref:`aiohttp transport <aiohttp_transport>`
and the :ref:`requests transport <requests_transport>`
using the `GraphQL multipart request spec`_.

.. _GraphQL multipart request spec: https://github.com/jaydenseric/graphql-multipart-request-spec
Expand All @@ -18,6 +19,7 @@ In order to upload a single file, you need to:
.. code-block:: python

transport = AIOHTTPTransport(url='YOUR_URL')
# Or transport = RequestsHTTPTransport(url='YOUR_URL')

client = Client(transport=transport)

Expand Down Expand Up @@ -45,6 +47,7 @@ It is also possible to upload multiple files using a list.
.. code-block:: python

transport = AIOHTTPTransport(url='YOUR_URL')
# Or transport = RequestsHTTPTransport(url='YOUR_URL')

client = Client(transport=transport)

Expand Down Expand Up @@ -84,6 +87,9 @@ We provide methods to do that for two different uses cases:
* Sending local files
* Streaming downloaded files from an external URL to the GraphQL API

.. note::
Streaming is only supported with the :ref:`aiohttp transport <aiohttp_transport>`

Streaming local files
^^^^^^^^^^^^^^^^^^^^^

Expand Down
73 changes: 68 additions & 5 deletions gql/transport/requests.py
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
import io
import json
import logging
from typing import Any, Dict, Optional, Union
from typing import Any, Dict, Optional, Tuple, Type, Union

import requests
from graphql import DocumentNode, ExecutionResult, print_ast
from requests.adapters import HTTPAdapter, Retry
from requests.auth import AuthBase
from requests.cookies import RequestsCookieJar
from requests_toolbelt.multipart.encoder import MultipartEncoder

from gql.transport import Transport

from ..utils import extract_files
from .exceptions import (
TransportAlreadyConnected,
TransportClosed,
Expand All @@ -27,6 +30,8 @@ class RequestsHTTPTransport(Transport):
The transport uses the requests library to send HTTP POST requests.
"""

file_classes: Tuple[Type[Any], ...] = (io.IOBase,)

def __init__(
self,
url: str,
Expand Down Expand Up @@ -104,6 +109,7 @@ def execute( # type: ignore
operation_name: Optional[str] = None,
timeout: Optional[int] = None,
extra_args: Dict[str, Any] = None,
upload_files: bool = False,
) -> ExecutionResult:
"""Execute GraphQL query.

Expand All @@ -116,6 +122,7 @@ def execute( # type: ignore
Only required in multi-operation documents (Default: None).
:param timeout: Specifies a default timeout for requests (Default: None).
:param extra_args: additional arguments to send to the requests post method
:param upload_files: Set to True if you want to put files in the variable values
:return: The result of execution.
`data` is the result of executing the query, `errors` is null
if no errors occurred, and is a non-empty array if an error occurred.
Expand All @@ -126,21 +133,77 @@ def execute( # type: ignore

query_str = print_ast(document)
payload: Dict[str, Any] = {"query": query_str}
if variable_values:
payload["variables"] = variable_values

if operation_name:
payload["operationName"] = operation_name

data_key = "json" if self.use_json else "data"
post_args = {
"headers": self.headers,
"auth": self.auth,
"cookies": self.cookies,
"timeout": timeout or self.default_timeout,
"verify": self.verify,
data_key: payload,
}

if upload_files:
# If the upload_files flag is set, then we need variable_values
assert variable_values is not None

# If we upload files, we will extract the files present in the
# variable_values dict and replace them by null values
nulled_variable_values, files = extract_files(
variables=variable_values, file_classes=self.file_classes,
)

# Save the nulled variable values in the payload
payload["variables"] = nulled_variable_values

# Add the payload to the operations field
operations_str = json.dumps(payload)
log.debug("operations %s", operations_str)

# Generate the file map
# path is nested in a list because the spec allows multiple pointers
# to the same file. But we don't support that.
# Will generate something like {"0": ["variables.file"]}
file_map = {str(i): [path] for i, path in enumerate(files)}

# Enumerate the file streams
# Will generate something like {'0': <_io.BufferedReader ...>}
file_streams = {str(i): files[path] for i, path in enumerate(files)}

# Add the file map field
file_map_str = json.dumps(file_map)
log.debug("file_map %s", file_map_str)

fields = {"operations": operations_str, "map": file_map_str}

# Add the extracted files as remaining fields
for k, v in file_streams.items():
fields[k] = (getattr(v, "name", k), v)

# Prepare requests http to send multipart-encoded data
data = MultipartEncoder(fields=fields)

post_args["data"] = data

if post_args["headers"] is None:
post_args["headers"] = {}
else:
post_args["headers"] = {**post_args["headers"]}

post_args["headers"]["Content-Type"] = data.content_type

else:
if variable_values:
payload["variables"] = variable_values

if log.isEnabledFor(logging.INFO):
log.info(">>> %s", json.dumps(payload))

data_key = "json" if self.use_json else "data"
post_args[data_key] = payload

# Log the payload
if log.isEnabledFor(logging.INFO):
log.info(">>> %s", json.dumps(payload))
Expand Down
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@

install_requests_requires = [
"requests>=2.23,<3",
"requests_toolbelt>=0.9.1,<1",
]

install_websockets_requires = [
Expand Down
Loading