Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash when using where true query #151

Open
martijnthe opened this issue Nov 1, 2024 · 5 comments
Open

Crash when using where true query #151

martijnthe opened this issue Nov 1, 2024 · 5 comments

Comments

@martijnthe
Copy link

martijnthe commented Nov 1, 2024

Library version: v0.8.1

When including where true in an APL query, the library crashes with this exception:

ValueError: 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5' is not a valid MessageCode

See below for the full backtrace.

In the Axiom webapp/console, the query runs fine. I do see a warning in the response:
image

Even though it's a useless query clause, the library shouldn't crash on the response.

The same happens with where false, but then the value that the library chokes on is apl_whereclausealwaysevaluatestoFALSEwhichwillexcludealldata_5.

  File "/home/redacted/.venv/lib/python3.10/site-packages/axiom/client.py", line 272, in query
    result = Util.from_dict(QueryResult, res.json())
             │              │            └ <Response [200]>
             │              └ <class 'axiom.query.result.QueryResult'>
             └ <class 'axiom.util.Util'>
  File "/home/redacted/.venv/lib/python3.10/site-packages/axiom/util.py", line 34, in from_dict
    return dacite.from_dict(data_class=data_class, data=data, config=cfg)
           │                           │                │            └ Config(type_hooks={<enum 'QueryKind'>: <enum 'QueryKind'>, <class 'datetime.datetime'>: <bound method Util.convert_string_to_dat...
           │                           │                └ {'format': 'legacy', 'status': {'elapsedTime': 5707, 'minCursor': '0d5aufdk9qf40-0779121980003ba2-00000001', 'maxCursor': '0d5b2...
           │                           └ <class 'axiom.query.result.QueryResult'>
           └ <module 'dacite' from '/home/redacted/.venv/lib/python3.10/site-packages/dacite/__init__.py'>
  File "/home/redacted/.venv/lib/python3.10/site-packages/dacite/core.py", line 64, in from_dict
    value = _build_value(type_=field_type, data=field_data, config=config)
    │       │                  │                │                  └ Config(type_hooks={<enum 'QueryKind'>: <enum 'QueryKind'>, <class 'datetime.datetime'>: <bound method Util.convert_string_to_dat...
    │       │                  │                └ {'elapsedTime': 5707, 'minCursor': '0d5aufdk9qf40-0779121980003ba2-00000001', 'maxCursor': '0d5b230gs1ds0-0779121bf500433c-00000...
    │       │                  └ <class 'axiom.query.result.QueryStatus'>
    │       └ <function _build_value at 0x765fee897d90>
    └ QueryLegacy(startTime=datetime.datetime(2024, 9, 2, 19, 51, 22, 393873, tzinfo=datetime.timezone.utc), endTime=datetime.datetime...
  File "/home/redacted/.venv/lib/python3.10/site-packages/dacite/core.py", line 99, in _build_value
    data = from_dict(data_class=type_, data=data, config=config)
    │      │                    │           │            └ Config(type_hooks={<enum 'QueryKind'>: <enum 'QueryKind'>, <class 'datetime.datetime'>: <bound method Util.convert_string_to_dat...
    │      │                    │           └ {'elapsedTime': 5707, 'minCursor': '0d5aufdk9qf40-0779121980003ba2-00000001', 'maxCursor': '0d5b230gs1ds0-0779121bf500433c-00000...
    │      │                    └ <class 'axiom.query.result.QueryStatus'>
    │      └ <function from_dict at 0x765fee895d80>
    └ {'elapsedTime': 5707, 'minCursor': '0d5aufdk9qf40-0779121980003ba2-00000001', 'maxCursor': '0d5b230gs1ds0-0779121bf500433c-00000...
  File "/home/redacted/.venv/lib/python3.10/site-packages/dacite/core.py", line 64, in from_dict
    value = _build_value(type_=field_type, data=field_data, config=config)
    │       │                  │                │                  └ Config(type_hooks={<enum 'QueryKind'>: <enum 'QueryKind'>, <class 'datetime.datetime'>: <bound method Util.convert_string_to_dat...
    │       │                  │                └ [{'priority': 'warn', 'count': 1, 'code': 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5', 'msg': 'line: 5, col:...
    │       │                  └ typing.List[axiom.query.result.Message]
    │       └ <function _build_value at 0x765fee897d90>
    └ '2024-11-01T18:33:00Z'
  File "/home/redacted/.venv/lib/python3.10/site-packages/dacite/core.py", line 97, in _build_value
    data = _build_value_for_collection(collection=type_, data=data, config=config)
    │      │                                      │           │            └ Config(type_hooks={<enum 'QueryKind'>: <enum 'QueryKind'>, <class 'datetime.datetime'>: <bound method Util.convert_string_to_dat...
    │      │                                      │           └ [{'priority': 'warn', 'count': 1, 'code': 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5', 'msg': 'line: 5, col:...
    │      │                                      └ typing.List[axiom.query.result.Message]
    │      └ <function _build_value_for_collection at 0x765fee8b4670>
    └ [{'priority': 'warn', 'count': 1, 'code': 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5', 'msg': 'line: 5, col:...
  File "/home/redacted/.venv/lib/python3.10/site-packages/dacite/core.py", line 154, in _build_value_for_collection
    return data_type(_build_value(type_=item_type, data=item, config=config) for item in data)
           │         │                  │                            │                   └ [{'priority': 'warn', 'count': 1, 'code': 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5', 'msg': 'line: 5, col:...
           │         │                  │                            └ Config(type_hooks={<enum 'QueryKind'>: <enum 'QueryKind'>, <class 'datetime.datetime'>: <bound method Util.convert_string_to_dat...
           │         │                  └ <class 'axiom.query.result.Message'>
           │         └ <function _build_value at 0x765fee897d90>
           └ <class 'list'>
  File "/home/redacted/.venv/lib/python3.10/site-packages/dacite/core.py", line 154, in <genexpr>
    return data_type(_build_value(type_=item_type, data=item, config=config) for item in data)
                     │                  │               │            │           └ {'priority': 'warn', 'count': 1, 'code': 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5', 'msg': 'line: 5, col: ...
                     │                  │               │            └ Config(type_hooks={<enum 'QueryKind'>: <enum 'QueryKind'>, <class 'datetime.datetime'>: <bound method Util.convert_string_to_dat...
                     │                  │               └ {'priority': 'warn', 'count': 1, 'code': 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5', 'msg': 'line: 5, col: ...
                     │                  └ <class 'axiom.query.result.Message'>
                     └ <function _build_value at 0x765fee897d90>
  File "/home/redacted/.venv/lib/python3.10/site-packages/dacite/core.py", line 99, in _build_value
    data = from_dict(data_class=type_, data=data, config=config)
    │      │                    │           │            └ Config(type_hooks={<enum 'QueryKind'>: <enum 'QueryKind'>, <class 'datetime.datetime'>: <bound method Util.convert_string_to_dat...
    │      │                    │           └ {'priority': 'warn', 'count': 1, 'code': 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5', 'msg': 'line: 5, col: ...
    │      │                    └ <class 'axiom.query.result.Message'>
    │      └ <function from_dict at 0x765fee895d80>
    └ {'priority': 'warn', 'count': 1, 'code': 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5', 'msg': 'line: 5, col: ...
  File "/home/redacted/.venv/lib/python3.10/site-packages/dacite/core.py", line 64, in from_dict
    value = _build_value(type_=field_type, data=field_data, config=config)
    │       │                  │                │                  └ Config(type_hooks={<enum 'QueryKind'>: <enum 'QueryKind'>, <class 'datetime.datetime'>: <bound method Util.convert_string_to_dat...
    │       │                  │                └ 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5'
    │       │                  └ <enum 'MessageCode'>
    │       └ <function _build_value at 0x765fee897d90>
    └ 1
  File "/home/redacted/.venv/lib/python3.10/site-packages/dacite/core.py", line 91, in _build_value
    data = config.type_hooks[type_](data)
    │      │                 │      └ 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5'
    │      │                 └ <enum 'MessageCode'>
    │      └ Config(type_hooks={<enum 'QueryKind'>: <enum 'QueryKind'>, <class 'datetime.datetime'>: <bound method Util.convert_string_to_dat...
    └ 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5'
  File "/home/redacted/lib/python3.10/enum.py", line 385, in __call__
    return cls.__new__(cls, value)
           │           │    └ 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5'
           │           └ <enum 'MessageCode'>
           └ <enum 'MessageCode'>
  File "/home/redacted/lib/python3.10/enum.py", line 710, in __new__
    raise ve_exc
          └ None
ValueError: 'apl_whereclausealwaysevaluatestoTRUEwhichwillincludealldata_5' is not a valid MessageCode
@bahlo
Copy link
Member

bahlo commented Nov 8, 2024

Hi, thank you for opening this issue! Definitely a bug, I'll take a look later or early next week 😌

@bahlo
Copy link
Member

bahlo commented Nov 9, 2024

Wrote a test in #152, which passes. There is also no MessageCode in v0.8.1, are you sure you're using that version?

@martijnthe
Copy link
Author

There is also no MessageCode in v0.8.1, are you sure you're using that version?

Yes. Looking at the backtrace, MessageCode seems to be coming from the dacite dependency.

@darach
Copy link
Contributor

darach commented Nov 11, 2024

Hi @martijnthe,

Could you provide a raw failing event json like from your screen capture above? I may have mis-transcribed something when I tried to replicate your issue:

from dataclasses import dataclass
from dacite import from_dict
from enum import Enum
from typing import Optional
from axiom_py.query import QueryResult, MessagePriority
import json

testcase = {
    "format": "legacy",
    "status": {
        "elapsedTime": 0,
        "blocksExamined": 0,
        "rowsExamined": 0,
        "rowsMatched": 0,
        "numGroups": 0,
        "isPartial": False,
        "continuationToken": None,
        "isEstimate": None,
        "minBlockTime": None,
        "maxBlockTime": None,
        "messages": [
            {
                "priority": MessagePriority.WARN,
                "count": 1,
                "code": "apl_possibly_causing harm which is bad - sorry martin!",
                "msg": "line 5, col 20 where clause evaluates to computer says no"
            }
        ],
        "minCursor": None,
        "maxCursor": None,
    }
}

def dataclass_to_dict(obj):
    if hasattr(obj, "__dataclass_fields__"):
        return {k: dataclass_to_dict(v) for k, v in obj.__dict__.items()}
    elif isinstance(obj, list):
        return [dataclass_to_dict(i) for i in obj]
    elif isinstance(obj, Enum):
        return obj.value
    else:
        return obj

user = from_dict(QueryResult, testcase)
as_json = dataclass_to_dict(user)
print(json.dumps(as_json))

Oddly no direct reference to MessageCode that I can see in the axiom python SDK for v0.8.1 or recent revisions so wondering how you could have produced the trace above. What output do you get from the script attached?

By running:

$ python3 test.py | jq

I get a JSON document as follows:

{
  "request": null,
  "status": {
    "elapsedTime": 0,
    "blocksExamined": 0,
    "rowsExamined": 0,
    "rowsMatched": 0,
    "numGroups": 0,
    "isPartial": false,
    "continuationToken": null,
    "isEstimate": null,
    "minBlockTime": null,
    "maxBlockTime": null,
    "messages": [
      {
        "priority": "warn",
        "count": 1,
        "code": "apl_possibly_causing harm which is bad - sorry martin!",
        "msg": "line 5, col 20 where clause evaluates to computer says no"
      }
    ],
    "maxCursor": null,
    "minCursor": null
  },
  "matches": null,
  "buckets": null,
  "tables": null,
  "dataset_names": [],
  "savedQueryID": null
}

If you can an error/exception that would be interesting.

Python version for the above:

$ python --version
Python 3.13.0

@darach
Copy link
Contributor

darach commented Nov 11, 2024

Another useful check would be:

pip show axiom_py

And make sure the version is v0.7 or more recent. Prior to v0.7 MessageCode as indicated by
your trace was a valid dataclass so you could be picking up a stale version of the library in your
environment.

Please let us know how you get along @martijnthe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants