Custom JSON encoders and decoders

[elemoine]

Cf. #99 (and #108 for the preliminary work on this).

This PR makes it possible for the user to provide custom JSON encoders and decoders.

The custom encoder and decoder are provided as JSON dumps and loads functions on the App object.

Here's an example dealing with datetime objects:

import functools
import json

from procrastinate import App, PostgresJobStore

# Function used for encoding datetime objects
def encode(obj):
    if isinstance(obj, datetime.datetime):
        return obj.isoformat()
    raise TypeError()

dumps = functools.partial(json.dumps, default=encode)

# Function used for decoding datetime objects
def decode(dict_):
    if "dt" in dict_:
        dict_["dt"] = datetime.datetime.fromisoformat(dict_["dt"])
    return dict_

loads = functools.partial(json.loads, object_hook=decode)

app = App(job_store=PostgresJobStore(), dumps=dumps, loads=loads)

I think the approach provides a maximum of flexibility to the users. They can use the json module the way they want. They can even use a different JSON implementation than json, such as UltraJSON, if they want to.

I am creating this PR for discussion. The code in my branch (attached to this PR) basically works and is tested, although there's still work to do (typing, documentation, more tests maybe, etc.).

Any feedback?

Successful PR Checklist:

• Tests
• Documentation (optionally: run spell checking)
• Had a good time contributing? (if not, feel free to give some feedback)

[codecov-io]

Codecov Report

Merging #131 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #131   +/-   ##
=====================================
  Coverage     100%   100%           
=====================================
  Files          19     19           
  Lines         736    743    +7     
  Branches       70     71    +1     
=====================================
+ Hits          736    743    +7

Impacted Files	Coverage Δ
procrastinate/store.py	100% <100%> (ø)	⬆️
procrastinate/cli.py	100% <100%> (ø)	⬆️
procrastinate/aiopg_connector.py	100% <100%> (ø)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3d3601e...793ff64. Read the comment docs.

[DainDwarf]

I was wondering, would it be better to store the functions in the job_store rather than the app? I think it would reduce passing around those functions through all the wrapping, and keep the code cleaner. And imo it makes sense that "encode/decoding functions" are configured to the instance of what is dealing with matching jobs on postgresql.

Taking your example, for the user point of view, you would only need to change the last line to

app = App(job_store=PostgresJobStore(dumps=dumps, loads=loads))

[elemoine]

I was wondering, would it be better to store the functions in the job_store rather than the app?

Great suggestion! I'll look into it…