Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pin openapi-core==0.15.0 and openapi-spec-validator<0.5.0,>=0.4.0 #690

Merged
merged 10 commits into from
Nov 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,8 @@ liccheck = "==0.6.0"
memory-profiler = "==0.57.0"
# ^ still used?
numpy = ">=1.18.1"
openapi-core = "==0.13.2"
openapi-spec-validator = "==0.2.8"
openapi-core = "==0.15.0"
openapi-spec-validator = "<0.5.0,>=0.4.0"
packaging = "<24.0,>=23.1"
pexpect = "==4.8.0"
protobuf = "<4.25.0,>=4.21.6"
Expand Down
2 changes: 1 addition & 1 deletion docs/http-connection-and-skill.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Add the http server connection package:
mkdir packages
aea create my_aea
cd my_aea
aea add connection valory/http_server:0.22.0:bafybeid4nl6ruidpto3ynwjmc76nf42egcroqlhqq6krh2onwktu4ywpne --remote
aea add connection valory/http_server:0.22.0:bafybeiesafy2af2wakqvsal4hz6ldycb73j5w62szhwy25iyhlf2wgyg44 --remote
aea push connection valory/http_server --local
aea add protocol fetchai/default:1.0.0:bafybeibtqp56jkijwjsohk4z5vqp6pfkiexmnmk5uleteotbsgrypy6gxm --remote
aea push protocol fetchai/default --local
Expand Down
2 changes: 1 addition & 1 deletion docs/http-echo-demo.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ The easiest way to get started with the http server is to use our pre-built exam

``` bash
pipenv shell
aea fetch open_aea/http_echo:0.1.0:bafybeiah75oosb24i3nwtbotqjd44waxg7ldmxgle4iiumemgbrvlosxsu --remote
aea fetch open_aea/http_echo:0.1.0:bafybeibph3jlsb3hmxmqf7pmaxxhlfqhf6gq6lon2yelufdy7enrlz6mkq --remote
cd http_echo
aea generate-key ethereum; aea add-key ethereum
aea install
Expand Down
2 changes: 1 addition & 1 deletion packages/open_aea/agents/http_echo/aea-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ fingerprint:
README.md: bafybeibkr6ecv5efx3hwxvxposvpmr76ugrj6kydeasb7bppo3ibynnjcu
fingerprint_ignore_patterns: []
connections:
- valory/http_server:0.22.0:bafybeid4nl6ruidpto3ynwjmc76nf42egcroqlhqq6krh2onwktu4ywpne
- valory/http_server:0.22.0:bafybeiesafy2af2wakqvsal4hz6ldycb73j5w62szhwy25iyhlf2wgyg44
contracts: []
protocols:
- fetchai/default:1.0.0:bafybeibtqp56jkijwjsohk4z5vqp6pfkiexmnmk5uleteotbsgrypy6gxm
Expand Down
4 changes: 2 additions & 2 deletions packages/packages.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"protocol/valory/ledger_api/1.0.0": "bafybeige5agrztgzfevyglf7mb4o7pzfttmq4f6zi765y4g2zvftbyowru",
"connection/fetchai/stub/0.21.0": "bafybeictgpdqbpyppmoxn2g7jkaxvulihew7zaszv4xyhgvsntq7tqs7wi",
"connection/valory/ledger/0.19.0": "bafybeia47rr37ianvwsh77tjjpv3nwif5sywhhy2fbdshnz4a2icwln76a",
"connection/valory/http_server/0.22.0": "bafybeid4nl6ruidpto3ynwjmc76nf42egcroqlhqq6krh2onwktu4ywpne",
"connection/valory/http_server/0.22.0": "bafybeiesafy2af2wakqvsal4hz6ldycb73j5w62szhwy25iyhlf2wgyg44",
"connection/valory/p2p_libp2p/0.1.0": "bafybeiaykya7tvir7k5scovjzuagpfcftvptxoi2od5qqqvukwglsrrtzy",
"connection/valory/p2p_libp2p_client/0.1.0": "bafybeihge56dn3xep2dzomu7rtvbgo4uc2qqh7ljl3fubqdi2lq44gs5lq",
"connection/valory/p2p_libp2p_mailbox/0.1.0": "bafybeiajf6msadxrr2rgkgebwlzoiznxdccsyzjopm2pkommcgcamsw27m",
Expand All @@ -26,7 +26,7 @@
"agent/fetchai/gym_aea/0.25.0": "bafybeif5c657r3ixuosbyihtsdzr2tqifikxgy5e6t7ur2jmhtiolina3m",
"agent/fetchai/my_first_aea/0.27.0": "bafybeicrgya4gln4lqr3ba76t2cuyik37lcgr64kzna2fkoseseobytaa4",
"agent/open_aea/gym_aea/0.1.0": "bafybeibyyl4wtnrfxeoed3kjnlftphkcikdqrgf76u34v4wbvwgsqfqrgy",
"agent/open_aea/http_echo/0.1.0": "bafybeiah75oosb24i3nwtbotqjd44waxg7ldmxgle4iiumemgbrvlosxsu",
"agent/open_aea/http_echo/0.1.0": "bafybeibph3jlsb3hmxmqf7pmaxxhlfqhf6gq6lon2yelufdy7enrlz6mkq",
"agent/open_aea/my_first_aea/0.1.0": "bafybeibv7nlyxldyj5ntivsu74ylul4dltpfvkfa46k2pbveetfpkvz4jm",
"connection/fetchai/local/0.20.0": "bafybeigukufpgy2sdkv5qqde47dhigbjitxcrxwg6sl6unlpbc7wbxqyhm",
"connection/valory/http_client/0.23.0": "bafybeiddrfvomrmgvh5yuv2coq7ci72wcdf663stayi3m5aawnj4srggce",
Expand Down
47 changes: 33 additions & 14 deletions packages/valory/connections/http_server/connection.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,9 +33,9 @@

from aiohttp import web
from aiohttp.web_request import BaseRequest
from openapi_core import create_spec
from openapi_core.validation.request.datatypes import OpenAPIRequest, RequestParameters
from openapi_core.validation.request.shortcuts import validate_request
from openapi_core import validate_request
from openapi_core.schema.specs import Spec
from openapi_core.validation.request.datatypes import RequestParameters
from openapi_core.validation.request.validators import RequestValidator
from openapi_spec_validator.exceptions import ( # pylint: disable=wrong-import-order
OpenAPIValidationError,
Expand Down Expand Up @@ -64,7 +64,7 @@
REQUEST_TIMEOUT = 408
SERVER_ERROR = 500

_default_logger = logging.getLogger("aea.packages.fetchai.connections.http_server")
_default_logger = logging.getLogger("aea.packages.valory.connections.http_server")

RequestId = DialogueLabel
PUBLIC_ID = PublicId.from_str("valory/http_server:0.22.0")
Expand Down Expand Up @@ -115,9 +115,28 @@ def headers_to_string(headers: Dict) -> str:
return msg.as_string()


class Request(OpenAPIRequest):
class Request:
"""Generic request object."""

def __init__(
self,
host_url: str,
path: str,
full_url_pattern: str,
method: str,
parameters: RequestParameters,
body: bytes,
mimetype: str,
) -> None:
"""Initialize Request object."""
self.host_url = host_url
self.path = path
self.full_url_pattern = full_url_pattern
self.method = method
self.parameters = parameters
self.body = body
self.mimetype = mimetype

@property
def is_id_set(self) -> bool:
"""Check if id is set."""
Expand All @@ -142,24 +161,20 @@ async def create(cls, http_request: BaseRequest) -> "Request":
:return: a request
"""
method = http_request.method.lower()

parsed_path = urlparse(http_request.path_qs)

url = http_request.url

body = await http_request.read()

mimetype = http_request.content_type

query_params = parse_qs(parsed_path.query, keep_blank_values=True)

parameters = RequestParameters(
query=ImmutableMultiDict(query_params), # type: ignore
header=headers_to_string(dict(http_request.headers)),
path={},
)

request = Request(
host_url=str(url.with_path("/")),
path=parsed_path.path,
full_url_pattern=str(url),
method=method,
parameters=parameters,
Expand Down Expand Up @@ -258,8 +273,8 @@ def __init__(
api_spec_dict = read_yaml_file(api_spec_path)
if server is not None:
api_spec_dict["servers"] = [{"url": server}]
api_spec = create_spec(api_spec_dict)
self._validator = RequestValidator(api_spec)
self.api_spec = Spec.create(data=api_spec_dict)
self._validator = RequestValidator(self.api_spec)
except OpenAPIValidationError as e: # pragma: nocover
self.logger.error(
f"API specification YAML source file not correctly formatted: {str(e)}"
Expand All @@ -282,7 +297,11 @@ def verify(self, request: Request) -> bool:
return True

try:
validate_request(self._validator, request)
validate_request(
spec=self.api_spec,
request=request,
validator=self._validator,
)
except Exception: # pragma: nocover # pylint: disable=broad-except
self.logger.exception("APISpec verify error")
return False
Expand Down
7 changes: 4 additions & 3 deletions packages/valory/connections/http_server/connection.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ aea_version: '>=1.0.0, <2.0.0'
fingerprint:
README.md: bafybeifepluovb4so2eem34ulpcp4svegb2dqfpmtojtofhfkb3j2xanui
__init__.py: bafybeifykou5sazojmc7hdqnsdp4mncd4zh3xys3mdgdzwks23mvhzu2ga
connection.py: bafybeidc6zysa47tppkvq3ddq4jgpind7h6zzpsl4s7vukbbj5y4ts2zxm
connection.py: bafybeie42txykvedioambi2dq4nurktilg5v6qleo2iaesc7psmrobdhva
tests/__init__.py: bafybeifqaf7cnc4oczjkbwmv4ahrkbiqxrojwgowej3kbri3skz4lzt43i
tests/data/certs/server.crt: bafybeiev5i3xxkvn36wflf633gkumuxexsw4y2bubwbvl7edrz4igfgv34
tests/data/certs/server.csr: bafybeicvp7xdl5w3o4bzikkudpduitss3bpp6xqfwlxbw6kabdangohy5u
Expand All @@ -35,8 +35,9 @@ dependencies:
aiohttp:
version: <4.0.0,>=3.8.5
openapi-core:
version: ==0.13.2
version: ==0.15.0
openapi-spec-validator:
version: ==0.2.8
version: <0.5.0,>=0.4.0
werkzeug: {}
attrs: {}
is_abstract: false
3 changes: 3 additions & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -324,6 +324,9 @@ ignore_missing_imports=True
[mypy-toml.*]
ignore_missing_imports=True

[mypy-attr.*]
ignore_missing_imports=True

[darglint]
docstring_style=sphinx
strictness=short
Expand Down
4 changes: 2 additions & 2 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,10 @@ def get_all_extras() -> Dict:
cli_deps = [
"click==8.0.2",
"pyyaml==6.0.1",
"jsonschema<=4.19.0,>=4.16.0",
"packaging>=23.1,<24.0",
"pytest>=7.0.0,<7.3.0",
"coverage>=6.4.4,<8.0.0",
"jsonschema<4.4.0,>=4.3.0",
"semver>=2.9.1,<3.0.0",
]

Expand All @@ -55,7 +55,7 @@ def get_all_extras() -> Dict:
base_deps = [
"semver>=2.9.1,<3.0.0",
"base58>=1.0.3,<3.0.0",
"jsonschema<=4.19.0,>=4.16.0",
"jsonschema<4.4.0,>=4.3.0",
"packaging>=23.1,<24.0",
"protobuf<4.25.0,>=4.21.6",
"pymultihash==0.8.2",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ aea create my_aea
cd my_aea
```
``` bash
aea add connection valory/http_server:0.22.0:bafybeid4nl6ruidpto3ynwjmc76nf42egcroqlhqq6krh2onwktu4ywpne --remote
aea add connection valory/http_server:0.22.0:bafybeiesafy2af2wakqvsal4hz6ldycb73j5w62szhwy25iyhlf2wgyg44 --remote
```
``` bash
aea config set agent.default_connection valory/http_server:0.22.0
Expand Down Expand Up @@ -48,7 +48,7 @@ models:
mkdir packages
aea create my_aea
cd my_aea
aea add connection valory/http_server:0.22.0:bafybeid4nl6ruidpto3ynwjmc76nf42egcroqlhqq6krh2onwktu4ywpne --remote
aea add connection valory/http_server:0.22.0:bafybeiesafy2af2wakqvsal4hz6ldycb73j5w62szhwy25iyhlf2wgyg44 --remote
aea push connection valory/http_server --local
aea add protocol fetchai/default:1.0.0:bafybeibtqp56jkijwjsohk4z5vqp6pfkiexmnmk5uleteotbsgrypy6gxm --remote
aea push protocol fetchai/default --local
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
``` bash
pipenv shell
aea fetch open_aea/http_echo:0.1.0:bafybeiah75oosb24i3nwtbotqjd44waxg7ldmxgle4iiumemgbrvlosxsu --remote
aea fetch open_aea/http_echo:0.1.0:bafybeibph3jlsb3hmxmqf7pmaxxhlfqhf6gq6lon2yelufdy7enrlz6mkq --remote
cd http_echo
aea generate-key ethereum; aea add-key ethereum
aea install
Expand Down
4 changes: 2 additions & 2 deletions tox.ini
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@ deps =
gym==0.15.6
hypothesis==6.21.6
numpy>=1.18.1
openapi-core==0.13.2
openapi-spec-validator==0.2.8
openapi-core==0.15.0
openapi-spec-validator<0.5.0,>=0.4.0
asn1crypto<1.5.0,>=1.4.0
tomte[tests]==0.2.13

Expand Down
Loading