Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle operations without an amount #59

Closed
wants to merge 1 commit into from

Conversation

sbihel
Copy link

@sbihel sbihel commented Jun 10, 2021

Some origination operations don't have an amount field, which made my indexer crash with:

INFO     dipdup.datasources.tzkt.datasource  Got operation message
DEBUG    dipdup.datasources.tzkt.datasource  [{'type': 1, 'data': [{'type': 'origination', 'id': 54424511, 'level': 1509431, 'timestamp': '2021-06-10T13:18:18Z', 'block': 'BMPjTv12Eut47qFokKs2uZMQU4AARPDczUQaU73xGPcwL2wtwHp', 'hash': 'opBN1RgYYq9bUNBiHn6sHMyoxKoAFiRK9S4Nbm9c5U4q3wzd4bv', 'counter': 11549146, 'sender': {'alias': 'NFT PROTECTOR', 'address': 'tz1QGCWjNpYmcS6T9qFGYSam25e36WeFUCK4'}, 'gasLimit': 14767, 'gasUsed': 14667, 'storageLimit': 3336, 'storageUsed': 3079, 'bakerFee': 4692, 'storageFee': 769750, 'allocationFee': 64250, 'contractBalance': 0, 'storage': {'admin': 'tz1TWrPXuG3T3rR9NR5EqsBegJMwiMcobjkt', 'objkt': 'KT1RJ6PbjHpwc3M5rw5s2Nbmefwbuwbdxton', 'swaps': 4573, 'swap_id': '0', 'metadata': 4572, 'hicetnunc': 'tz1UBZUkXpKGhYsP5KtzDNqLLchwF4uHrGjw', 'hicetnunc_fee': '25', 'management_fee': '25'}, 'diffs': [{'bigmap': 4573, 'path': 'swaps', 'action': 'allocate'}, {'bigmap': 4572, 'path': 'metadata', 'action': 'allocate'}, {'bigmap': 4572, 'path': 'metadata', 'action': 'add_key', 'content': {'hash': 'expru5X1yxJG6ezR2uHMotwMLNmSzQyh5t1vUnhjx4cS6Pv9qE1Sdo', 'key': '', 'value': '74657a6f732d73746f726167653a6d657461'}}, {'bigmap': 4572, 'path': 'metadata', 'action': 'add_key', 'content': {'hash': 'expruWXebQNHqRvNSDLvYyjftKavCNKzKo6GGnXqivGidSJBgXiYZR', 'key': 'meta', 'value': '7b2276657273696f6e223a2276302e302e31222c226e616d65223a224f424a4b542e424944222c22617574686f7273223a5b22404e465450524f544543544f52203c6e667470726f746563746f724070726f746f6e6d61696c2e636f6d3e222c22406f6b74756f7267203c68747470733a2f2f7777772e747769747465722e636f6d2f6f6b74756f72673e225d2c226465736372697074696f6e223a224120646563656e7472616c6973656420706c6174666f726d20666f722062696464696e67206f6e204f424a4b5473222c22686f6d6570616765223a2268747470733a2f2f7777772e6f626a6b742e626964222c22696e7465726661636573223a5b22545a49502d303136225d7d'}}], 'status': 'applied', 'originatedContract': {'kind': 'smart_contract', 'address': 'KT1D3S6wLGGtSA8kun9reLeJa5EFe5THaWJ3', 'typeHash': 1534500745, 'codeHash': 2066995472}}], 'state': 1509431}]
INFO     dipdup.datasources.tzkt.datasource  Acquiring callback lock
DEBUG    db_client                           Closed connection pool None with params: {'host': 'db', 'port': 5432, 'user': 'dipdup', 'database': 'dipdup', 'min_size': 1, 'max_size': 5, 'connection_class': <class 'asyncpg.connection.Connection'>, 'loop': None, 'server_settings': {'search_path': 'public'}}
INFO     tortoise                            Tortoise-ORM shutdown
DEBUG    websockets.protocol                 client - event = data_received(<13 bytes>)
DEBUG    websockets.protocol                 client ! timed out waiting for TCP close
DEBUG    websockets.protocol                 client x closing TCP connection
DEBUG    websockets.protocol                 client - event = eof_received()
DEBUG    websockets.protocol                 client - event = connection_lost(None)
DEBUG    websockets.protocol                 client - state = CLOSED
DEBUG    websockets.protocol                 client x code = 1006, reason = [no reason]
Traceback (most recent call last):
  File "/usr/local/bin/dipdup", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.9/site-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dipdup/cli.py", line 22, in wrapper
    return asyncio.run(fn(*args, **kwargs))
  File "/usr/local/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.9/site-packages/dipdup/cli.py", line 65, in run
    await dipdup.run(reindex)
  File "/usr/local/lib/python3.9/site-packages/dipdup/dipdup.py", line 121, in run
    await asyncio.gather(*run_tasks)
  File "/usr/local/lib/python3.9/site-packages/dipdup/datasources/tzkt/datasource.py", line 378, in start
    await asyncio.gather(
  File "/usr/local/lib/python3.9/site-packages/dipdup/datasources/tzkt/datasource.py", line 106, in run
    await coro
  File "/usr/local/lib/python3.9/site-packages/dipdup/datasources/tzkt/datasource.py", line 602, in on_operation_message
    operation = self.convert_operation(operation_json)
  File "/usr/local/lib/python3.9/site-packages/dipdup/datasources/tzkt/datasource.py", line 823, in convert_operation
    amount=operation_json['amount'],
KeyError: 'amount'

I have tested it by letting my indexer run until it would seemingly hang (which I assume is because it doesn't process operation older than a certain threshold). That happens after a long time (half an hour maybe?).

@m-kus m-kus requested a review from droserasprout June 11, 2021 09:59
@m-kus
Copy link
Member

m-kus commented Jun 12, 2021

Hey @sbihel thanks for the patch! Actually this change is already in #49 will likely release that next week

@sbihel sbihel closed this Jun 12, 2021
@sbihel sbihel deleted the fix/origination_amount branch April 29, 2024 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants