Skip to content

Commit

Permalink
FDK Context| deadline | generic response, etc. (fnproject#16)
Browse files Browse the repository at this point in the history
* FDK context

* Generic response object

* Support for FN_DEADLINE

* Fixing status codes problem

* Custom response object tests

* Test functions that raising exceptions

 - use safe way to recover from exceptions for both JSON/HTTP handlers
 - confirm that status codes are set properly for edge cases like deadline

* Cleaning up before release

* Data coercing tests

* Cleanin up comments

* Updating samples
  • Loading branch information
denismakogon authored Dec 30, 2017
1 parent 63e6ad9 commit d06f3dd
Show file tree
Hide file tree
Showing 44 changed files with 699 additions and 403 deletions.
3 changes: 2 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ FROM python:3.6.2

RUN mkdir /code
ADD . /code/
RUN pip install -e /code/
RUN pip3 install -r /code/requirements.txt
RUN pip3 install -e /code/

WORKDIR /code/fdk/tests/fn/traceback
ENTRYPOINT ["python3", "func.py"]
54 changes: 28 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,14 +14,14 @@ In order to utilise this, you can write your `app.py` as follows:
```python
import fdk

from fdk.http import response
from fdk import response


def handler(context, data=None, loop=None):
return response.RawResponse(
http_proto_version=context.version,
status_code=200,
headers={},
context,
status_code=200,
headers={},
response_data=data.readall()
)

Expand All @@ -31,7 +31,7 @@ if __name__ == "__main__":

```

Automatic HTTP input coercions
Automatic input coercions
------------------------------

Decorators are provided that will attempt to coerce input values to Python types.
Expand All @@ -40,7 +40,8 @@ Some attempt is made to coerce return values from these functions also:
```python
import fdk

@fdk.coerce_http_input_to_content_type

@fdk.coerce_input_to_content_type
def handler(context, data=None, loop=None):
"""
body is a request body, it's type depends on content type
Expand All @@ -53,24 +54,24 @@ if __name__ == "__main__":

```

Working with async automatic HTTP input coercions
Working with async automatic input coercions
-------------------------------------------------

Latest version supports async coroutines as a request body processors:
```python
import asyncio
import fdk

from fdk.http import response
from fdk import response


@fdk.coerce_http_input_to_content_type
@fdk.coerce_input_to_content_type
async def handler(context, data=None, loop=None):
headers = {
"Content-Type": "text/plain",
}
return response.RawResponse(
http_proto_version=context.version,
context,
status_code=200,
headers=headers,
response_data="OK"
Expand All @@ -82,7 +83,7 @@ if __name__ == "__main__":
fdk.handle(handler, loop=loop)

```
As you can see `app` function is no longer callable, because its type: coroutine, so we need to bypass event loop inside
As you can see `app` function is no longer callable, because its type: coroutine, so we need to bypass event loop inside

Handling Hot JSON Functions
---------------------------
Expand Down Expand Up @@ -126,7 +127,7 @@ if __name__ == "__main__":
Applications powered by Fn: Concept
-----------------------------------

FDK is not only about developing functions, but providing necessary API to build serverless applications
FDK is not only about developing functions, but providing necessary API to build serverless applications
that look like nothing but classes with methods powered by Fn.

```python
Expand Down Expand Up @@ -163,6 +164,7 @@ class Application(object):
r.raise_for_status()
return r.text


if __name__ == "__main__":
app = Application(config={})

Expand Down Expand Up @@ -190,15 +192,15 @@ if __name__ == "__main__":
In order to identify to which Fn instance code needs to talk set following env var:

```bash
export API_URL=http://localhost:8080
export API_URL = http: // localhost: 8080
```
with respect to IP address or domain name where Fn lives.


Applications powered by Fn: supply data to a function
-----------------------------------------------------

At this moment those helper-decorators let developers interact with Fn-powered functions as with regular class methods.
At this moment those helper - decorators let developers interact with Fn - powered functions as with regular class methods.
In order to pass necessary data into a function developer just needs to do following
```python

Expand All @@ -208,13 +210,13 @@ if __name__ == "__main__":
app.env(keyone="blah", keytwo="blah", somethingelse=3)

```
Key-value args will be turned into JSON instance and will be sent to a function as payload body.
Key - value args will be turned into JSON instance and will be sent to a function as payload body.


Applications powered by Fn: working with function's result
----------------------------------------------------------

In order to work with result from function you just need to read key-value argument `fn_data`:
In order to work with result from function you just need to read key - value argument `fn_data`:
```python
@decorators.with_fn(fn_image="denismakogon/py-traceback-test:0.0.1",
fn_format="http")
Expand All @@ -225,7 +227,7 @@ In order to work with result from function you just need to read key-value argum
Applications powered by Fn: advanced serverless functions
---------------------------------------------------------

Since release v0.0.3 developer can consume new API to build truly serverless functions
Since release v0.0.3 developer can consume new API to build truly serverless functions
without taking care of Docker images, application, etc.

```python
Expand All @@ -247,29 +249,29 @@ Each function decorated with `@decorator.fn` will become truly serverless and di
So, how it works?

* A developer writes function
* FDK (Fn-powered app) creates a recursive Pickle v4.0 with 3rd-party dependencies
* FDK (Fn-powered app) transfers pickled object to a function based on Python3 GPI (general purpose image)
* FDK unpickles function and its 3rd-party dependencies and runs it
* Function sends response back to Fn-powered application function caller
* FDK(Fn - powered app) creates a recursive Pickle v4.0 with 3rd - party dependencies
* FDK(Fn - powered app) transfers pickled object to a function based on Python3 GPI(general purpose image)
* FDK unpickles function and its 3rd - party dependencies and runs it
* Function sends response back to Fn - powered application function caller

So, each CPU-intensive functions can be sent to Fn with the only load on networking (given example creates 7kB of traffic between app's host and Fn).
So, each CPU - intensive functions can be sent to Fn with the only load on networking(given example creates 7kB of traffic between app's host and Fn).


Applications powered by Fn: exceptions
--------------------------------------

Applications powered by Fn are following Go-like errors concept. It gives you full control on errors whether raise them or not.
Applications powered by Fn are following Go - like errors concept. It gives you full control on errors whether raise them or not.
```python
res, err = app.env()
if err:
raise err
print(res)

```
Each error is an instance fn `FnError` that encapsulates certain logic that makes hides HTTP errors and turns them into regular Python-like exceptions.
Each error is an instance fn `FnError` that encapsulates certain logic that makes hides HTTP errors and turns them into regular Python - like exceptions.

TODOs
-----

- generic response class
- use fdk.headers.GoLikeHeaders in http
- generic response class
- use fdk.headers.GoLikeHeaders in http
51 changes: 47 additions & 4 deletions fdk/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,57 @@
# License for the specific language governing permissions and limitations
# under the License.

from fdk.http import handle as http_handler
from fdk import runner
import functools
import io
import ujson

from fdk import runner

coerce_http_input_to_content_type = http_handler.coerce_input_to_content_type
handle = runner.generic_handle


def coerce_input_to_content_type(request_data_processor):

@functools.wraps(request_data_processor)
def app(context, data=None, loop=None):
"""
Request handler app dispatcher decorator
:param context: request context
:type context: request.RequestContext
:param data: request body
:type data: io.BufferedIOBase
:param loop: asyncio event loop
:type loop: asyncio.AbstractEventLoop
:return: raw response
:rtype: response.RawResponse
:return:
"""
body = data
content_type = context.Headers().get("content-type")
try:

if hasattr(data, "readable"):
request_body = io.TextIOWrapper(data)
else:
request_body = data

if content_type == "application/json":
if isinstance(request_body, str):
body = ujson.loads(request_body)
else:
body = ujson.load(request_body)
elif content_type in ["text/plain"]:
body = request_body.read()

except Exception as ex:
raise context.DispatchError(
context, 500, "Unexpected error: {}".format(str(ex)))

return request_data_processor(context, data=body, loop=loop)

return app


__all__ = [
'coerce_http_input_to_content_type',
'handle'
]
87 changes: 68 additions & 19 deletions fdk/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,29 +12,78 @@
# License for the specific language governing permissions and limitations
# under the License.

from fdk import errors


class RequestContext(object):

def __init__(self, method=None, url=None,
query_parameters=None, headers=None,
version=None):
def __init__(self, app_name, route, call_id,
fntype, config=None, headers=None, arguments=None):
"""
Request context here to be a placeholder
for request-specific attributes
:param method: HTTP request method
:type method: str
:param url: HTTP request URL
:type url: str
:param query_parameters: HTTP request query parameters
:type query_parameters: dict
:param headers: HTTP request headers
:type headers: object
:param version: HTTP proto version
:type version: tuple
"""
# TODO(xxx): app name, path, memory, type, config
self.method = method
self.url = url
self.query_parameters = query_parameters
self.headers = headers
self.version = version
self.__app_name = app_name
self.__app_route = route
self.__call_id = call_id
self.__config = config if config else {}
self.__headers = headers if headers else {}
self.__arguments = {} if not arguments else arguments
self.__type = fntype

def AppName(self):
return self.__app_name

def Route(self):
return self.__app_route

def CallID(self):
return self.__call_id

def Config(self):
return self.__config

def Headers(self):
return self.__headers

def Arguments(self):
return self.__arguments

def Type(self):
return self.__type


class HTTPContext(RequestContext):

def __init__(self, app_name, route,
call_id, fntype="http",
config=None, headers=None,
method=None, url=None,
query_parameters=None,
version=None):
arguments = {
"method": method,
"URL": url,
"query": query_parameters,
"http_version": version
}
self.DispatchError = errors.HTTPDispatchException
super(HTTPContext, self).__init__(
app_name, route, call_id, fntype,
config=config, headers=headers, arguments=arguments)


class JSONContext(RequestContext):

def __init__(self, app_name, route, call_id,
fntype="json", config=None, headers=None):
self.DispatchError = errors.JSONDispatchException
super(JSONContext, self).__init__(
app_name, route, call_id, fntype, config=config, headers=headers)


def fromType(fntype, *args, **kwargs):
if fntype == "json":
return JSONContext(*args, **kwargs)
if fntype == "http":
return HTTPContext(*args, **kwargs)
Loading

0 comments on commit d06f3dd

Please sign in to comment.