Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamic configuration, context and rollback improvements, bugfixes #49

Merged
merged 82 commits into from
Jun 15, 2021
Merged
Show file tree
Hide file tree
Changes from 64 commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
26d6197
Dexter initial
droserasprout May 19, 2021
cb88d2e
Trader stats for dexter and quipu
droserasprout May 19, 2021
4d99175
Symbol
droserasprout May 19, 2021
78d4cfb
WIP
droserasprout May 20, 2021
c699f03
WIP
droserasprout May 20, 2021
7d5900e
update_totals trigger
droserasprout May 20, 2021
c43692d
Fix typenames
droserasprout May 20, 2021
1f91abb
Fis some bugs, add config generator
droserasprout May 21, 2021
93f87e5
WIP
droserasprout May 24, 2021
3d15242
Merge branch 'master' into feat/quipuswap-dexter
droserasprout May 24, 2021
9d06749
WIP
droserasprout May 24, 2021
798edd8
WIP
droserasprout May 25, 2021
da1d8a4
Revert quipuswap, WIP
droserasprout May 25, 2021
5749714
WIP
droserasprout May 25, 2021
48c2e22
WIP
droserasprout May 25, 2021
03325ff
WIP
droserasprout May 25, 2021
de945a1
WIP
droserasprout May 25, 2021
22b64d1
WIP
droserasprout May 25, 2021
2c51c52
WIP
droserasprout May 25, 2021
8972ce5
WIP
droserasprout May 25, 2021
d2aa1c0
Codegen
droserasprout May 25, 2021
51c62c1
Fix updating summary view
droserasprout May 28, 2021
3f20655
Merge branch 'master' into feat/quipuswap-dexter
droserasprout May 31, 2021
425dd05
Fix divesting liquidity to empty pools
droserasprout May 31, 2021
9ef950e
WIP
droserasprout May 31, 2021
1a1aff8
Cleanup
droserasprout May 31, 2021
d704a29
Fix tests
droserasprout May 31, 2021
f05bde1
Backport quipuswap fixes
droserasprout May 31, 2021
bad1ddf
WIP
droserasprout Jun 1, 2021
145deb5
Merge branch 'master' into feat/quipuswap-dexter
droserasprout Jun 1, 2021
bcca181
WIP
droserasprout Jun 1, 2021
3cb734c
WIP
droserasprout Jun 1, 2021
6bf0391
Drop dynamic templates
droserasprout Jun 2, 2021
0b831ec
WIP
droserasprout Jun 2, 2021
dbf055c
WIP
droserasprout Jun 2, 2021
52cd097
Codegen docs
droserasprout Jun 3, 2021
7e47e15
Docs
droserasprout Jun 3, 2021
682969a
More docs
droserasprout Jun 3, 2021
0e5fdd5
WIP
droserasprout Jun 3, 2021
2d4086f
Bugfixes
droserasprout Jun 3, 2021
92f20ef
Pass originations to multiple indexes
droserasprout Jun 3, 2021
391de4d
Bugfixes
droserasprout Jun 6, 2021
8c27fea
WIP executor refactoring
droserasprout Jun 6, 2021
4d4eaf5
Merge branch 'master' into feat/quipuswap-dexter
droserasprout Jun 6, 2021
9a88c43
Fix executor, biig map matching
droserasprout Jun 6, 2021
5037086
Rewrite executor, fix updating index level
droserasprout Jun 6, 2021
e4cc892
WIP
droserasprout Jun 6, 2021
46d33c0
Atomicity fix WIP
droserasprout Jun 7, 2021
2a6a333
Codegen fixes
droserasprout Jun 7, 2021
b505747
Limit queue to 2000 (still blocking loop), more docs
droserasprout Jun 7, 2021
a1ea5cc
Lint, sync twice on start
droserasprout Jun 8, 2021
e904543
Docs
droserasprout Jun 8, 2021
fc79339
WIP
droserasprout Jun 10, 2021
44e68cf
WIP
droserasprout Jun 10, 2021
631bccf
BigMap WIP
droserasprout Jun 10, 2021
74861c2
Move State to indexes
droserasprout Jun 10, 2021
d443336
Oneshot flag
droserasprout Jun 10, 2021
ab0f586
Move and rename context classes
droserasprout Jun 10, 2021
b9811a6
Fix handlers, fix oneshot mode
droserasprout Jun 10, 2021
17a49a4
Fix integration tests
droserasprout Jun 10, 2021
083a2e1
Factory context helpers
droserasprout Jun 10, 2021
0bfc1aa
Move hashes fetching from config, fix bigmap message callback
droserasprout Jun 10, 2021
1f2deb1
Make pipelines green
droserasprout Jun 10, 2021
88def3b
Cleanup
droserasprout Jun 10, 2021
d1f36fc
Migration for handlers
droserasprout Jun 11, 2021
38004cf
Even cooler migrations
droserasprout Jun 11, 2021
6fa5378
Migrate demos
droserasprout Jun 11, 2021
cc5c166
Do not restart on migration
droserasprout Jun 11, 2021
c74a86b
Rewrite rollbacks
droserasprout Jun 11, 2021
a1a91cb
Cleanup, fix tests
droserasprout Jun 11, 2021
1d1e82d
Fix schema fetching
droserasprout Jun 11, 2021
0d7c1e7
Monkeypatch Tortoise to fix get_or_create
droserasprout Jun 14, 2021
49a42c3
Lock Tortoise to 0.17.4
droserasprout Jun 14, 2021
1cab662
Make keepalive_timeout configurable, log DB ConnectionErrors on retry
droserasprout Jun 14, 2021
5b9ced5
Do not close connector, log request before sending
droserasprout Jun 14, 2021
a47b46b
Usa single session
droserasprout Jun 14, 2021
c7b03a6
Keepalive
droserasprout Jun 14, 2021
e91d4af
Lock asyncpg
droserasprout Jun 14, 2021
845fe9a
Drop tortoise hacks, close aiohttp sessions on exit
droserasprout Jun 14, 2021
dd7471c
Lint
droserasprout Jun 14, 2021
a64dcd8
Enforce using transaction inside `in_global_transaction` wrapper
droserasprout Jun 15, 2021
245feba
Tests for in_global_transaction
droserasprout Jun 15, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
version: "3.8"

services:

hic_et_nunc:
build: .
depends_on:
Expand Down
60 changes: 36 additions & 24 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ pydantic = "^1.8.1"
aiosignalrcore = "^0.9.2"
fcache = "^0.4.7"
click = "^8.0.1"
pyee = "^8.1.0"

[tool.poetry.dev-dependencies]
black = "^20.8b1"
Expand Down
5 changes: 3 additions & 2 deletions src/demo_hic_et_nunc/handlers/on_cancel_swap.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
import demo_hic_et_nunc.models as models
from demo_hic_et_nunc.types.hen_minter.parameter.cancel_swap import CancelSwapParameter
from demo_hic_et_nunc.types.hen_minter.storage import HenMinterStorage
from dipdup.models import OperationHandlerContext, TransactionContext
from dipdup.context import OperationHandlerContext
from dipdup.models import Transaction


async def on_cancel_swap(
ctx: OperationHandlerContext,
cancel_swap: TransactionContext[CancelSwapParameter, HenMinterStorage],
cancel_swap: Transaction[CancelSwapParameter, HenMinterStorage],
) -> None:
swap = await models.Swap.filter(id=int(cancel_swap.parameter.__root__)).get()
swap.status = models.SwapStatus.CANCELED
Expand Down
5 changes: 3 additions & 2 deletions src/demo_hic_et_nunc/handlers/on_collect.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
import demo_hic_et_nunc.models as models
from demo_hic_et_nunc.types.hen_minter.parameter.collect import CollectParameter
from demo_hic_et_nunc.types.hen_minter.storage import HenMinterStorage
from dipdup.models import OperationHandlerContext, TransactionContext
from dipdup.context import OperationHandlerContext
from dipdup.models import Transaction


async def on_collect(
ctx: OperationHandlerContext,
collect: TransactionContext[CollectParameter, HenMinterStorage],
collect: Transaction[CollectParameter, HenMinterStorage],
) -> None:
swap = await models.Swap.filter(id=collect.parameter.swap_id).get()
seller = await swap.creator
Expand Down
7 changes: 4 additions & 3 deletions src/demo_hic_et_nunc/handlers/on_mint.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,14 @@
from demo_hic_et_nunc.types.hen_minter.storage import HenMinterStorage
from demo_hic_et_nunc.types.hen_objkts.parameter.mint import MintParameter
from demo_hic_et_nunc.types.hen_objkts.storage import HenObjktsStorage
from dipdup.models import OperationHandlerContext, TransactionContext
from dipdup.context import OperationHandlerContext
from dipdup.models import Transaction


async def on_mint(
ctx: OperationHandlerContext,
mint_objkt: TransactionContext[MintOBJKTParameter, HenMinterStorage],
mint: TransactionContext[MintParameter, HenObjktsStorage],
mint_objkt: Transaction[MintOBJKTParameter, HenMinterStorage],
mint: Transaction[MintParameter, HenObjktsStorage],
) -> None:
holder, _ = await models.Holder.get_or_create(address=mint.parameter.address)
token = models.Token(
Expand Down
5 changes: 3 additions & 2 deletions src/demo_hic_et_nunc/handlers/on_swap.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
import demo_hic_et_nunc.models as models
from demo_hic_et_nunc.types.hen_minter.parameter.swap import SwapParameter
from demo_hic_et_nunc.types.hen_minter.storage import HenMinterStorage
from dipdup.models import OperationHandlerContext, TransactionContext
from dipdup.context import OperationHandlerContext
from dipdup.models import Transaction


async def on_swap(
ctx: OperationHandlerContext,
swap: TransactionContext[SwapParameter, HenMinterStorage],
swap: Transaction[SwapParameter, HenMinterStorage],
) -> None:
holder, _ = await models.Holder.get_or_create(address=swap.data.sender_address)
swap_model = models.Swap(
Expand Down
5 changes: 1 addition & 4 deletions src/demo_hic_et_nunc/models.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,7 @@
from enum import Enum, IntEnum
from enum import IntEnum

from tortoise import Model, fields

# on mint token, holder
# on_swap new, cancel_swap, collect


class SwapStatus(IntEnum):
ACTIVE = 0
Expand Down
19 changes: 10 additions & 9 deletions src/demo_quipuswap/handlers/on_fa12_divest_liquidity.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@
from demo_quipuswap.types.fa12_token.storage import Fa12TokenStorage
from demo_quipuswap.types.quipu_fa12.parameter.divest_liquidity import DivestLiquidityParameter
from demo_quipuswap.types.quipu_fa12.storage import QuipuFa12Storage
from dipdup.models import OperationHandlerContext, TransactionContext
from dipdup.context import OperationHandlerContext
from dipdup.models import Transaction


async def on_fa12_divest_liquidity(
ctx: OperationHandlerContext,
divest_liquidity: TransactionContext[DivestLiquidityParameter, QuipuFa12Storage],
transfer: TransactionContext[TransferParameter, Fa12TokenStorage],
divest_liquidity: Transaction[DivestLiquidityParameter, QuipuFa12Storage],
transfer: Transaction[TransferParameter, Fa12TokenStorage],
) -> None:
if ctx.template_values is None:
raise Exception('This index must be templated')
Expand All @@ -32,12 +33,12 @@ async def on_fa12_divest_liquidity(

tez_pool = Decimal(storage.storage.tez_pool) / (10 ** 6)
token_pool = Decimal(storage.storage.token_pool) / (10 ** decimals)
if tez_pool and token_pool:
price = tez_pool / token_pool
else:
last_trade = await models.Trade.filter(symbol=symbol).order_by('-id').first()
assert last_trade
price = last_trade.price

# NOTE: Empty pools mean exchange is not initialized yet
if not tez_pool and not token_pool:
return

price = tez_pool / token_pool
share_px = (tez_qty + price * token_qty) / shares_qty

position.realized_pl += shares_qty * (share_px - position.avg_share_px)
Expand Down
7 changes: 4 additions & 3 deletions src/demo_quipuswap/handlers/on_fa12_invest_liquidity.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@
from demo_quipuswap.types.fa12_token.storage import Fa12TokenStorage
from demo_quipuswap.types.quipu_fa12.parameter.invest_liquidity import InvestLiquidityParameter
from demo_quipuswap.types.quipu_fa12.storage import QuipuFa12Storage
from dipdup.models import OperationHandlerContext, TransactionContext
from dipdup.context import OperationHandlerContext
from dipdup.models import Transaction


async def on_fa12_invest_liquidity(
ctx: OperationHandlerContext,
invest_liquidity: TransactionContext[InvestLiquidityParameter, QuipuFa12Storage],
transfer: TransactionContext[TransferParameter, Fa12TokenStorage],
invest_liquidity: Transaction[InvestLiquidityParameter, QuipuFa12Storage],
transfer: Transaction[TransferParameter, Fa12TokenStorage],
) -> None:
if ctx.template_values is None:
raise Exception('This index must be templated')
Expand Down
7 changes: 3 additions & 4 deletions src/demo_quipuswap/handlers/on_fa12_origination.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
from typing import Optional

import demo_quipuswap.models as models
from demo_quipuswap.types.quipu_fa12.storage import QuipuFa12Storage
from dipdup.models import OperationData, OperationHandlerContext, OriginationContext, TransactionContext
from dipdup.context import OperationHandlerContext
from dipdup.models import Origination


async def on_fa12_origination(
ctx: OperationHandlerContext,
quipu_fa12_origination: OriginationContext[QuipuFa12Storage],
quipu_fa12_origination: Origination[QuipuFa12Storage],
) -> None:
if ctx.template_values is None:
raise Exception('This index must be templated')
Expand Down
7 changes: 4 additions & 3 deletions src/demo_quipuswap/handlers/on_fa12_tez_to_token.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@
from demo_quipuswap.types.fa12_token.storage import Fa12TokenStorage
from demo_quipuswap.types.quipu_fa12.parameter.tez_to_token_payment import TezToTokenPaymentParameter
from demo_quipuswap.types.quipu_fa12.storage import QuipuFa12Storage
from dipdup.models import OperationHandlerContext, TransactionContext
from dipdup.context import OperationHandlerContext
from dipdup.models import Transaction


async def on_fa12_tez_to_token(
ctx: OperationHandlerContext,
tez_to_token_payment: TransactionContext[TezToTokenPaymentParameter, QuipuFa12Storage],
transfer: TransactionContext[TransferParameter, Fa12TokenStorage],
tez_to_token_payment: Transaction[TezToTokenPaymentParameter, QuipuFa12Storage],
transfer: Transaction[TransferParameter, Fa12TokenStorage],
) -> None:
if ctx.template_values is None:
raise Exception('This index must be templated')
Expand Down
7 changes: 4 additions & 3 deletions src/demo_quipuswap/handlers/on_fa12_token_to_tez.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@
from demo_quipuswap.types.fa12_token.storage import Fa12TokenStorage
from demo_quipuswap.types.quipu_fa12.parameter.token_to_tez_payment import TokenToTezPaymentParameter
from demo_quipuswap.types.quipu_fa12.storage import QuipuFa12Storage
from dipdup.models import OperationHandlerContext, TransactionContext
from dipdup.context import OperationHandlerContext
from dipdup.models import Transaction


async def on_fa12_token_to_tez(
ctx: OperationHandlerContext,
token_to_tez_payment: TransactionContext[TokenToTezPaymentParameter, QuipuFa12Storage],
transfer: TransactionContext[TransferParameter, Fa12TokenStorage],
token_to_tez_payment: Transaction[TokenToTezPaymentParameter, QuipuFa12Storage],
transfer: Transaction[TransferParameter, Fa12TokenStorage],
) -> None:
if ctx.template_values is None:
raise Exception('This index must be templated')
Expand Down
Loading