-
Notifications
You must be signed in to change notification settings - Fork 404
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Problem with set_type_codec and array of json/jsonb #623
Comments
This is because the binary I/O format for |
Some documentation regarding this would maybe be helpful... Also the non binary case does not work for both json and jsonb versions... In any case it's really strange from the user prospective that setting the |
Since the problem is clearly the encoder overriding, is is possible to only override the decoder? Could this be considered as a request? |
Documenting wire formats for PostgreSQL data types is arguably not in the scope of user documentation, considering that even PostgreSQL doesn't document them.
It doesn't work because you need to account for NULL values correctly and return
I'm not sure what you mean. Isn't the purpose of
I guess we can add some magic constant like |
The documentation could just mention that asyncpg sends the received value directly to PostgreSQL, so the wire format for pg should be returned by the function when
the string
Yes, but an user could only want to change one of the two and not necessary both at the same time |
I've tried this and returning The encoder I tried is Maybe I did not understand what you meant. Could you provide an example? Thanks |
I tried exactly this and ran into the same error. Guess I'm confused in the same way. Following deeper into the array encoder, I found this for #82: asyncpg/asyncpg/protocol/codecs/array.pyx Lines 201 to 202 in a2e53ab
And the following patch fixed @CaselIT 's text codec use cases: diff --git a/asyncpg/protocol/codecs/array.pyx b/asyncpg/protocol/codecs/array.pyx
index e975b9f..25ae254 100644
--- a/asyncpg/protocol/codecs/array.pyx
+++ b/asyncpg/protocol/codecs/array.pyx
@@ -209,7 +209,9 @@ cdef _write_textarray_data(ConnectionSettings settings, object obj,
try:
if not apg_strcasecmp_char(elem_str, b'NULL'):
- array_data.write_bytes(b'"NULL"')
+ array_data.write_bytes(b'"')
+ array_data.write_cstr(elem_str, 4)
+ array_data.write_bytes(b'"')
else:
quoted_elem_len = elem_len
need_quoting = False I'm only not certain about if this is appropriate for all scenarios. cc @elprans |
Yeah, good catch @fantix, please submit a PR. |
When given a textual json codec that yields 'null', the array encoder should generate b'["null"]' instead of b'["NULL"]' for a JSON[] type. Refs: MagicStack#623
When given a textual json codec that yields 'null', the array encoder should generate b'["null"]' instead of b'["NULL"]' for a JSON[] type. Refs: #623
the issue with a local PostgreSQL install?: no
uvloop?: not tried
I think there is an issue in how asyncpg handles the json input data when an user sets a type codec for
json
and/orjsonb
and the column type isJSON[]
orJSONB[]
.For the type
json[]
there is actually a working configuration, by setting theformat='binary'
when setting the type codec, but the same configuration fails onjsonb
This came up while trying to normalize the behavior of all pg dbapi for sqlalchemy sqlalchemy/sqlalchemy#5584
Below is a reproduction case that uses only asyncpg, without sqlalchemy.
The output I get is:
When the test works the data that is inserted is correct.
The expected result is that all cases work, or that at least a set_codec configuration for both
json
andjsonb
is functioning.cc @fantix
The text was updated successfully, but these errors were encountered: