Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Domain basetypes aren't loaded as encoders/decoders to the client cache #886

Closed
QuantumTM opened this issue Feb 9, 2022 · 3 comments
Closed

Comments

@QuantumTM
Copy link
Contributor

  • asyncpg version: 0.22.0 (also tested on 0.25.0)
  • PostgreSQL version: 11
  • Do you use a PostgreSQL SaaS? If so, which? Can you reproduce
    the issue with a local PostgreSQL install?
    : N/A
  • Python version: 3.8
  • Platform: linux
  • Do you use pgbouncer?: No
  • Did you install asyncpg with pip?: Yes
  • If you built asyncpg locally, which version of Cython did you use?: N/A
  • Can the issue be reproduced under both asyncio and
    uvloop?
    : Not tested

When attempting an insert on a domain type I hit an issue where asyncpg was unable to handle the numeric[] type;

Traceback (most recent call last):
  File "test.py", line 40, in <module>
    asyncio.run(main())
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "test.py", line 36, in main
    await insert_num_array(conn, [1, 2])
  File "test.py", line 29, in insert_num_array
    await conn.execute(sql, (num_array,))
  File "/home/quantumtm/src/asyncpg/asyncpg/connection.py", line 320, in execute
    _, status, _ = await self._execute(
  File "/home/quantumtm/src/asyncpg/asyncpg/connection.py", line 1659, in _execute
    result, _ = await self.__execute(
  File "/home/quantumtm/src/asyncpg/asyncpg/connection.py", line 1684, in __execute
    return await self._do_execute(
  File "/home/quantumtm/src/asyncpg/asyncpg/connection.py", line 1711, in _do_execute
    stmt = await self._get_statement(
  File "/home/quantumtm/src/asyncpg/asyncpg/connection.py", line 416, in _get_statement
    settings.register_data_types(types)
  File "asyncpg/protocol/settings.pyx", line 35, in asyncpg.protocol.protocol.ConnectionSettings.register_data_types
    cpdef inline register_data_types(self, types):
  File "asyncpg/protocol/settings.pyx", line 36, in asyncpg.protocol.protocol.ConnectionSettings.register_data_types
    self._data_codecs.add_types(types)
  File "asyncpg/protocol/codecs/base.pyx", line 556, in asyncpg.protocol.protocol.DataCodecConfig.add_types
    elem_codec = self.declare_fallback_codec(
  File "asyncpg/protocol/codecs/base.pyx", line 706, in asyncpg.protocol.protocol.DataCodecConfig.declare_fallback_codec
    raise exceptions.UnsupportedClientFeatureError(
asyncpg.exceptions._base.UnsupportedClientFeatureError: unhandled standard data type 'numeric[]' (OID 1231)

On analysis this error can occure for any, none str, array type.

A minimal replication case is given below

import asyncio
import asyncpg
from asyncpg.connection import Connection

async def connect():
    return await asyncpg.connect(user="quantumtm", password="quantumtm", database="test", host="127.0.0.1")

async def setup(conn: Connection):
    sql = """
    DROP TABLE IF EXISTS test;
    DROP DOMAIN IF EXISTS num_array;
    CREATE DOMAIN num_array numeric[];
    CREATE TABLE test (
        num num_array
    );
    """
    await conn.execute(sql)

async def insert_num_array(conn: Connection, num_array):
    sql = "INSERT INTO test (num) VALUES ($1)"
    await conn.execute(sql, (num_array,))

async def main():
    conn = await connect()
    await setup(conn)
    await insert_num_array(conn, [1, 2])

if __name__ == '__main__':
    asyncio.run(main())
QuantumTM pushed a commit to QuantumTM/asyncpg that referenced this issue Feb 9, 2022
Domain encoding/decoding is handled by its basetype. This change forces
those basetypes to be inspected and loaded to the client cache avoiding
the issue where base types weren't encodable/decodable.
@QuantumTM
Copy link
Contributor Author

I've created a pull request #887 which fixes the issue.

In the interim, if other hit this issue, casting to the base type in the sql forces asynpg to load the missing encoders/decoders which fixes the issue. eg/. changing the above insert sql to INSERT INTO test (num) VALUES ($1::numeric[]) avoids the issue.

QuantumTM pushed a commit to QuantumTM/asyncpg that referenced this issue Feb 17, 2022
Add test for domain basetype introspections and loading
@QuantumTM
Copy link
Contributor Author

@elprans with #887 being merged I would concider this issue fix. Would you like me to close the ticket?

@elprans
Copy link
Member

elprans commented May 10, 2022

Yes. Closing.

@elprans elprans closed this as completed May 10, 2022
elprans added a commit that referenced this issue Jul 7, 2022
Changes
-------

* Add support to use awaitable object in password function.   (#889)
  (by @kwarunek in fb3b6bf for #889)

* Support direct TLS connections (i.e. no STARTTLS) (#923)
  (by @jackwotherspoon in f2a937d for #923)

Fixes
-----

* Fix invalid `pyproject.toml` (#900)
  (by @Rongronggg9 in eddb649 for #900)

* Add record_class parameter Pool.fetch and Pool.fetchrow (#896)
  (by @baltitenger in 2519cf3 for #896)

* Domain basetypes are introspected (#886) (#887)
  (by @QuantumTM in cca4a2d for #886)

* Properly handle exceptions raised while handling server auth messages (#862)
  (by @elprans in bd19262 for #862)
elprans added a commit that referenced this issue Jul 7, 2022
Changes
-------

* Add support to use awaitable object in password function.   (#889)
  (by @kwarunek in fb3b6bf for #889)

* Support direct TLS connections (i.e. no STARTTLS) (#923)
  (by @jackwotherspoon in f2a937d for #923)

Fixes
-----

* Fix invalid `pyproject.toml` (#900)
  (by @Rongronggg9 in eddb649 for #900)

* Add record_class parameter Pool.fetch and Pool.fetchrow (#896)
  (by @baltitenger in 2519cf3 for #896)

* Domain basetypes are introspected (#886) (#887)
  (by @QuantumTM in cca4a2d for #886)

* Properly handle exceptions raised while handling server auth messages (#862)
  (by @elprans in bd19262 for #862)
elprans added a commit that referenced this issue Jul 7, 2022
Changes
-------

* Add support to use awaitable object in password function.   (#889)
  (by @kwarunek in fb3b6bf for #889)

* Support direct TLS connections (i.e. no STARTTLS) (#923)
  (by @jackwotherspoon in f2a937d for #923)

Fixes
-----

* Fix invalid `pyproject.toml` (#900)
  (by @Rongronggg9 in eddb649 for #900)

* Add record_class parameter Pool.fetch and Pool.fetchrow (#896)
  (by @baltitenger in 2519cf3 for #896)

* Domain basetypes are introspected (#886) (#887)
  (by @QuantumTM in cca4a2d for #886)

* Properly handle exceptions raised while handling server auth messages (#862)
  (by @elprans in bd19262 for #862)
@elprans elprans mentioned this issue Jul 7, 2022
rohitsanj pushed a commit to noteable-io/asyncpg-crdb-noteable that referenced this issue May 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants