JSON decimal values should be parsed as Python decimal.Decimal in targets when reading Singer messages

[MeltyBot]

Migrated from GitLab: https://gitlab.com/meltano/sdk/-/issues/346

Originally created by @edgarrmondragon on 2022-03-09 16:17:31

---

Summary

JSON decimal values should be parsed as Python decimal.Decimal in targets when reading Singer messages.

Steps to reproduce

Try piping a tap that outputs similar to {"type": "number", "multipleOf": "0.001"} to a target that performs JSON schema validation. Values that should pass validation will fail.

What is the current bug behavior?

Decimals are loaded as float values which may cause problems when their precision is important, e.g. when validating against JSON schema `multipleOf' .

What is the expected correct behavior?

Decimals should be loaded as decimal.Decimal to preserve precision.

Relevant logs and/or screenshots

>>> import jsonschema
>>> jsonschema.validate(48.867082, {"type": "number", "multipleOf": 0.000001})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/edgarramirez/Library/Caches/pypoetry/virtualenvs/singer-sdk-S0flctL8-py3.9/lib/python3.9/site-packages/jsonschema/validators.py", line 934, in validate
    raise error
jsonschema.exceptions.ValidationError: 48.867082 is not a multiple of 1e-06

Failed validating 'multipleOf' in schema:
    {'multipleOf': 1e-06, 'type': 'number'}

On instance:
    48.867082
>>> import decimal
>>> jsonschema.validate(decimal.Decimal("48.867082"), {"type": "number", "multipleOf": decimal.Decimal("0.000001")})

Possible fixes

The jsonschema library dev recommends loading JSON decimals as Python decimal.Decimal: python-jsonschema/jsonschema#810 (comment).

Using json.loads(..., parse_float=decimal.Decimal) in https://gitlab.com/meltano/sdk/-/blob/b394e1a9c0d68a554640974c05be1c9d548270ee/singer_sdk/io_base.py#L77 should fix the issue.

[MeltyBot]

View 1 previous comment from the original issue on GitLab

[pnadolny13]

@tayloramurphy @edgarrmondragon this came up in slack https://meltano.slack.com/archives/C01TCRBBJD7/p1687952240803039. The suggested fix seems like its pretty straight forward but would probably need a unit test to assert its working.

The updated link to the suggested json.loads( fix is

sdk/singer_sdk/io_base.py

Line 67 in d81c3f7

line_dict = json.loads(line)

.