Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replacing/removing during swap proccess leads to arby crash #132

Open
raladev opened this issue Jan 7, 2021 · 0 comments
Open

Replacing/removing during swap proccess leads to arby crash #132

raladev opened this issue Jan 7, 2021 · 0 comments
Labels
P1 top priority

Comments

@raladev
Copy link
Contributor

raladev commented Jan 7, 2021

Steps:

  1. Simulate long swap process to keep order in hold
  2. Wait for order replacing by arby

Expected result:
fast solution - wait a little bit and try to replace again (not good for price movements)
a little bit harder solution - place order with different id after unsuccessful attempt (qty should be reduced by holded amount)

Actual result:
Unrecoverable err + arby restarts

2021-01-07 03:49:09.099 [Centralized] trace: New price: 0.032203
2021-01-07 03:49:10.1010 [Centralized] trace: New price: 0.032201
2021-01-07 03:49:10.1010 [Centralized] trace: New price: 0.0322
2021-01-07 03:49:11.1111 [Centralized] trace: New price: 0.032193
2021-01-07 03:49:11.1111 [OpenDEX] trace: Replacing ETH/BTC buy order with id arby-ETH/BTC-buy-order, quantity 0.36923458 and price 0.031710105
2021-01-07 03:49:11.1111 [Centralized] trace: New price: 0.03219
2021-01-07 03:49:11.1111 [GLOBAL] info: Unrecoverable error: {"code":3,"details":"order with local id arby-ETH/BTC-buy-order has a quantity of 10000 satoshis on hold, try again later","metadata":{"internalRepr":{},"options":{}}} - cleaning up.
2021-01-07 03:49:11.1111 [Centralized] trace: ETHBTC stream closed
2021-01-07 03:49:11.1111 [OpenDEX] info: All OpenDEX orders have been removed
2021-01-07 03:49:11.1111 [Centralized] info: All CEX orders have been removed
Shutdown complete. Goodbye, Arby.
2021-01-07 15:49:12,157 INFO exited: arby (terminated by SIGSEGV (core dumped); not expected)
2021-01-07 15:49:13,163 INFO spawned: 'arby' with pid 86

> [email protected] start:arby /app
> npm run compile && node dist/arby.js


> [email protected] compile /app
> npm run lint && tsc && cross-os postcompile


> [email protected] lint /app
> eslint . --ext .js,.ts

2021-01-07 15:49:14,401 INFO success: arby entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-01-07 03:49:25.2525 [GLOBAL] info: Starting. Hello, Arby.
2021-01-07 03:49:25.2525 [GLOBAL] info: Running with config:
TEST_MODE: false
CEX: BINANCE
LOG_LEVEL: trace
DATA_DIR: /root/.arby
OPENDEX_CERT_PATH: /root/.xud/tls.cert
OPENDEX_RPC_HOST: xud
OPENDEX_RPC_PORT: 28886
MARGIN: 0.015
BASEASSET: ETH
QUOTEASSET: BTC
CEX_BASEASSET: ETH
CEX_QUOTEASSET: BTC
TEST_CENTRALIZED_EXCHANGE_BASEASSET_BALANCE: 123
TEST_CENTRALIZED_EXCHANGE_QUOTEASSET_BALANCE: 321
2021-01-07 03:49:26.2626 [OpenDEX] trace: Base asset balance 30 (maxinbound: 0.38759631, maxoutbound: 30) and quote asset balance 2.49999682 (maxinbound: 2.4499095, maxoutbound: 2.44999682).
2021-01-07 03:49:26.2626 [Centralized] info: Base asset balance 0.37296423 and quote asset balance 0.01946175
2021-01-07 03:49:26.2626 [Centralized] trace: ETHBTC established connection to wss://stream.binance.com:9443/ws/ethbtc@aggTrade
2021-01-07 03:49:26.2626 [Centralized] trace: heartbeat from ETHBTC socket
2021-01-07 03:49:26.2626 [Centralized] trace: New price: 0.032173
2021-01-07 03:49:26.2626 [OpenDEX] trace: Replacing ETH/BTC buy order with id arby-ETH/BTC-buy-order, quantity 0.36923458 and price 0.031690405
2021-01-07 03:49:26.2626 [OpenDEX] trace: Replacing ETH/BTC sell order with id arby-ETH/BTC-sell-order, quantity 0.5988603 and price 0.032655595
2021-01-07 03:49:26.2626 [OpenDEX] trace: Creating ETH/BTC sell order with id arby-ETH/BTC-sell-order, quantity 0.5988603 and price 0.032655595
2021-01-07 03:49:26.2626 [GLOBAL] info: Unrecoverable error: {"code":3,"details":"order with local id arby-ETH/BTC-buy-order has a quantity of 10000 satoshis on hold, try again later","metadata":{"internalRepr":{},"options":{}}} - cleaning up.
2021-01-07 15:49:27,337 INFO exited: arby (terminated by SIGSEGV (core dumped); not expected)
2021-01-07 15:49:28,342 INFO spawned: 'arby' with pid 165

> [email protected] start:arby /app
> npm run compile && node dist/arby.js


> [email protected] compile /app
> npm run lint && tsc && cross-os postcompile


> [email protected] lint /app
> eslint . --ext .js,.ts

2021-01-07 15:49:29,561 INFO success: arby entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-01-07 03:49:40.4040 [GLOBAL] info: Starting. Hello, Arby.
2021-01-07 03:49:40.4040 [GLOBAL] info: Running with config:
TEST_MODE: false
CEX: BINANCE
LOG_LEVEL: trace
DATA_DIR: /root/.arby
OPENDEX_CERT_PATH: /root/.xud/tls.cert
OPENDEX_RPC_HOST: xud
OPENDEX_RPC_PORT: 28886
MARGIN: 0.015
BASEASSET: ETH
QUOTEASSET: BTC
CEX_BASEASSET: ETH
CEX_QUOTEASSET: BTC
TEST_CENTRALIZED_EXCHANGE_BASEASSET_BALANCE: 123
TEST_CENTRALIZED_EXCHANGE_QUOTEASSET_BALANCE: 321
2021-01-07 03:49:41.4141 [OpenDEX] trace: Base asset balance 30 (maxinbound: 0.38759631, maxoutbound: 30) and quote asset balance 2.49999682 (maxinbound: 2.4499095, maxoutbound: 2.44999682).
2021-01-07 03:49:41.4141 [Centralized] info: Base asset balance 0.37296423 and quote asset balance 0.01946175
2021-01-07 03:49:42.4242 [Centralized] trace: ETHBTC established connection to wss://stream.binance.com:9443/ws/ethbtc@aggTrade
2021-01-07 03:49:42.4242 [Centralized] trace: heartbeat from ETHBTC socket
2021-01-07 03:49:43.4343 [Centralized] trace: New price: 0.03218
2021-01-07 03:49:43.4343 [OpenDEX] trace: Replacing ETH/BTC buy order with id arby-ETH/BTC-buy-order, quantity 0.36923458 and price 0.0316973
2021-01-07 03:49:43.4343 [OpenDEX] trace: Replacing ETH/BTC sell order with id arby-ETH/BTC-sell-order, quantity 0.59873003 and price 0.0326627
2021-01-07 03:49:43.4343 [OpenDEX] trace: Order arby-ETH/BTC-sell-order created
2021-01-07 03:49:43.4343 [Centralized] trace: New price: 0.032179
2021-01-07 03:49:43.4343 [Centralized] trace: New price: 0.032178
2021-01-07 03:49:43.4343 [GLOBAL] info: Unrecoverable error: {"code":3,"details":"order with local id arby-ETH/BTC-buy-order has a quantity of 10000 satoshis on hold, try again later","metadata":{"internalRepr":{},"options":{}}} - cleaning up.
2021-01-07 15:49:43,574 INFO exited: arby (terminated by SIGSEGV (core dumped); not expected)
2021-01-07 15:49:44,580 INFO spawned: 'arby' with pid 244

> [email protected] start:arby /app
> npm run compile && node dist/arby.js


> [email protected] compile /app
> npm run lint && tsc && cross-os postcompile


> [email protected] lint /app
> eslint . --ext .js,.ts

2021-01-07 15:49:45,819 INFO success: arby entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-01-07 03:49:57.5757 [GLOBAL] info: Starting. Hello, Arby.
2021-01-07 03:49:57.5757 [GLOBAL] info: Running with config:
TEST_MODE: false
CEX: BINANCE
LOG_LEVEL: trace
DATA_DIR: /root/.arby
OPENDEX_CERT_PATH: /root/.xud/tls.cert
OPENDEX_RPC_HOST: xud
OPENDEX_RPC_PORT: 28886
MARGIN: 0.015
BASEASSET: ETH
QUOTEASSET: BTC
CEX_BASEASSET: ETH
CEX_QUOTEASSET: BTC
TEST_CENTRALIZED_EXCHANGE_BASEASSET_BALANCE: 123
TEST_CENTRALIZED_EXCHANGE_QUOTEASSET_BALANCE: 321
2021-01-07 03:49:57.5757 [Centralized] info: Base asset balance 0.37296423 and quote asset balance 0.01946175
2021-01-07 03:49:57.5757 [OpenDEX] trace: Base asset balance 30 (maxinbound: 0.38759631, maxoutbound: 30) and quote asset balance 2.49999682 (maxinbound: 2.4499095, maxoutbound: 2.44999682).
2021-01-07 03:49:58.5858 [Centralized] trace: ETHBTC established connection to wss://stream.binance.com:9443/ws/ethbtc@aggTrade
2021-01-07 03:49:58.5858 [Centralized] trace: heartbeat from ETHBTC socket
2021-01-07 03:49:58.5858 [Centralized] trace: New price: 0.0322
2021-01-07 03:49:58.5858 [OpenDEX] trace: Replacing ETH/BTC buy order with id arby-ETH/BTC-buy-order, quantity 0.36923458 and price 0.031717
2021-01-07 03:49:58.5858 [OpenDEX] trace: Replacing ETH/BTC sell order with id arby-ETH/BTC-sell-order, quantity 0.59835815 and price 0.032683
2021-01-07 03:49:58.5858 [Centralized] trace: New price: 0.032203
2021-01-07 03:49:58.5858 [OpenDEX] trace: Order arby-ETH/BTC-sell-order created
2021-01-07 03:49:58.5858 [GLOBAL] info: Unrecoverable error: {"code":3,"details":"order with local id arby-ETH/BTC-buy-order has a quantity of 10000 satoshis on hold, try again later","metadata":{"internalRepr":{},"options":{}}} - cleaning up.
2021-01-07 03:49:58.5858 [Centralized] trace: ETHBTC stream closed
2021-01-07 03:49:58.5858 [OpenDEX] info: All OpenDEX orders have been removed
2021-01-07 03:49:59.5959 [Centralized] info: All CEX orders have been removed
Shutdown complete. Goodbye, Arby.
2021-01-07 15:49:59,035 INFO exited: arby (exit status 0; expected)
2021-01-07 15:50:00,040 INFO spawned: 'arby' with pid 323

> [email protected] start:arby /app
> npm run compile && node dist/arby.js


> [email protected] compile /app
> npm run lint && tsc && cross-os postcompile


> [email protected] lint /app
> eslint . --ext .js,.ts

2021-01-07 15:50:01,284 INFO success: arby entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-01-07 03:50:12.1212 [GLOBAL] info: Starting. Hello, Arby.
2021-01-07 03:50:12.1212 [GLOBAL] info: Running with config:
TEST_MODE: false
CEX: BINANCE
LOG_LEVEL: trace
DATA_DIR: /root/.arby
OPENDEX_CERT_PATH: /root/.xud/tls.cert
OPENDEX_RPC_HOST: xud
OPENDEX_RPC_PORT: 28886
MARGIN: 0.015
BASEASSET: ETH
QUOTEASSET: BTC
CEX_BASEASSET: ETH
CEX_QUOTEASSET: BTC
TEST_CENTRALIZED_EXCHANGE_BASEASSET_BALANCE: 123
TEST_CENTRALIZED_EXCHANGE_QUOTEASSET_BALANCE: 321
2021-01-07 03:50:13.1313 [OpenDEX] trace: Base asset balance 30 (maxinbound: 0.38759631, maxoutbound: 30) and quote asset balance 2.49999682 (maxinbound: 2.4499095, maxoutbound: 2.44999682).
2021-01-07 03:50:13.1313 [Centralized] info: Base asset balance 0.37296423 and quote asset balance 0.01946175
2021-01-07 03:50:14.1414 [Centralized] trace: ETHBTC established connection to wss://stream.binance.com:9443/ws/ethbtc@aggTrade
2021-01-07 03:50:14.1414 [Centralized] trace: heartbeat from ETHBTC socket
2021-01-07 03:50:14.1414 [Centralized] trace: New price: 0.032177
2021-01-07 03:50:14.1414 [OpenDEX] trace: Replacing ETH/BTC buy order with id arby-ETH/BTC-buy-order, quantity 0.36923458 and price 0.031694345
2021-01-07 03:50:14.1414 [OpenDEX] trace: Replacing ETH/BTC sell order with id arby-ETH/BTC-sell-order, quantity 0.59878585 and price 0.032659655
2021-01-07 03:50:14.1414 [Centralized] trace: New price: 0.032182
2021-01-07 03:50:14.1414 [OpenDEX] trace: Creating ETH/BTC sell order with id arby-ETH/BTC-sell-order, quantity 0.59878585 and price 0.032659655
2021-01-07 03:50:14.1414 [Centralized] trace: New price: 0.032183
2021-01-07 03:50:14.1414 [GLOBAL] info: Unrecoverable error: {"code":3,"details":"order with local id arby-ETH/BTC-buy-order has a quantity of 10000 satoshis on hold, try again later","metadata":{"internalRepr":{},"options":{}}} - cleaning up.
2021-01-07 03:50:14.1414 [Centralized] trace: ETHBTC stream closed
2021-01-07 03:50:14.1414 [OpenDEX] info: All OpenDEX orders have been removed
2021-01-07 03:50:14.1414 [Centralized] info: All CEX orders have been removed
Shutdown complete. Goodbye, Arby.
2021-01-07 15:50:14,698 INFO exited: arby (terminated by SIGSEGV (core dumped); not expected)
2021-01-07 15:50:15,704 INFO spawned: 'arby' with pid 402

> [email protected] start:arby /app
> npm run compile && node dist/arby.js


> [email protected] compile /app
> npm run lint && tsc && cross-os postcompile


> [email protected] lint /app
> eslint . --ext .js,.ts

2021-01-07 15:50:16,946 INFO success: arby entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-01-07 03:50:29.2929 [GLOBAL] info: Starting. Hello, Arby.
2021-01-07 03:50:29.2929 [GLOBAL] info: Running with config:
TEST_MODE: false
CEX: BINANCE
LOG_LEVEL: trace
DATA_DIR: /root/.arby
OPENDEX_CERT_PATH: /root/.xud/tls.cert
OPENDEX_RPC_HOST: xud
OPENDEX_RPC_PORT: 28886
MARGIN: 0.015
BASEASSET: ETH
QUOTEASSET: BTC
CEX_BASEASSET: ETH
CEX_QUOTEASSET: BTC
TEST_CENTRALIZED_EXCHANGE_BASEASSET_BALANCE: 123
TEST_CENTRALIZED_EXCHANGE_QUOTEASSET_BALANCE: 321
2021-01-07 03:50:29.2929 [OpenDEX] trace: Base asset balance 30 (maxinbound: 0.38759631, maxoutbound: 30) and quote asset balance 2.49999682 (maxinbound: 2.4499095, maxoutbound: 2.44999682).
2021-01-07 03:50:29.2929 [Centralized] info: Base asset balance 0.37296423 and quote asset balance 0.01946175
2021-01-07 03:50:30.3030 [Centralized] trace: ETHBTC established connection to wss://stream.binance.com:9443/ws/ethbtc@aggTrade
2021-01-07 03:50:30.3030 [Centralized] trace: heartbeat from ETHBTC socket
2021-01-07 03:50:30.3030 [Centralized] trace: New price: 0.032162
2021-01-07 03:50:30.3030 [OpenDEX] trace: Replacing ETH/BTC buy order with id arby-ETH/BTC-buy-order, quantity 0.36923458 and price 0.03167957
2021-01-07 03:50:30.3030 [OpenDEX] trace: Replacing ETH/BTC sell order with id arby-ETH/BTC-sell-order, quantity 0.59906512 and price 0.03264443
2021-01-07 03:50:30.3030 [GLOBAL] info: Unrecoverable error: {"code":3,"details":"order with local id arby-ETH/BTC-buy-order has a quantity of 10000 satoshis on hold, try again later","metadata":{"internalRepr":{},"options":{}}} - cleaning up.
2021-01-07 03:50:30.3030 [Centralized] trace: ETHBTC stream closed
2021-01-07 03:50:30.3030 [OpenDEX] info: All OpenDEX orders have been removed
2021-01-07 03:50:30.3030 [Centralized] info: All CEX orders have been removed
Shutdown complete. Goodbye, Arby.
2021-01-07 15:50:30,747 INFO exited: arby (terminated by SIGSEGV (core dumped); not expected)
2021-01-07 15:50:31,753 INFO spawned: 'arby' with pid 481

> [email protected] start:arby /app
> npm run compile && node dist/arby.js


> [email protected] compile /app
> npm run lint && tsc && cross-os postcompile


> [email protected] lint /app
> eslint . --ext .js,.ts

2021-01-07 15:50:32,981 INFO success: arby entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-01-07 03:50:44.4444 [GLOBAL] info: Starting. Hello, Arby.
2021-01-07 03:50:44.4444 [GLOBAL] info: Running with config:
TEST_MODE: false
CEX: BINANCE
LOG_LEVEL: trace
DATA_DIR: /root/.arby
OPENDEX_CERT_PATH: /root/.xud/tls.cert
OPENDEX_RPC_HOST: xud
OPENDEX_RPC_PORT: 28886
MARGIN: 0.015
BASEASSET: ETH
QUOTEASSET: BTC
CEX_BASEASSET: ETH
CEX_QUOTEASSET: BTC
TEST_CENTRALIZED_EXCHANGE_BASEASSET_BALANCE: 123
TEST_CENTRALIZED_EXCHANGE_QUOTEASSET_BALANCE: 321
2021-01-07 03:50:44.4444 [Centralized] info: Base asset balance 0.37296423 and quote asset balance 0.01946175
2021-01-07 03:50:44.4444 [OpenDEX] trace: Base asset balance 30 (maxinbound: 0.38759631, maxoutbound: 30) and quote asset balance 2.49999682 (maxinbound: 2.4499095, maxoutbound: 2.44999682).
2021-01-07 03:50:45.4545 [Centralized] trace: ETHBTC established connection to wss://stream.binance.com:9443/ws/ethbtc@aggTrade
2021-01-07 03:50:45.4545 [Centralized] trace: heartbeat from ETHBTC socket
2021-01-07 03:50:45.4545 [Centralized] trace: New price: 0.032157
2021-01-07 03:50:45.4545 [OpenDEX] trace: Replacing ETH/BTC buy order with id arby-ETH/BTC-buy-order, quantity 0.36923458 and price 0.031674645
2021-01-07 03:50:45.4545 [OpenDEX] trace: Replacing ETH/BTC sell order with id arby-ETH/BTC-sell-order, quantity 0.59915827 and price 0.032639355
2021-01-07 03:50:45.4545 [OpenDEX] trace: Creating ETH/BTC sell order with id arby-ETH/BTC-sell-order, quantity 0.59915827 and price 0.032639355
2021-01-07 03:50:45.4545 [GLOBAL] info: Unrecoverable error: {"code":3,"details":"order with local id arby-ETH/BTC-buy-order has a quantity of 10000 satoshis on hold, try again later","metadata":{"internalRepr":{},"options":{}}} - cleaning up.
2021-01-07 03:50:45.4545 [Centralized] trace: ETHBTC stream closed
2021-01-07 03:50:45.4545 [OpenDEX] info: All OpenDEX orders have been removed
2021-01-07 03:50:45.4545 [Centralized] info: All CEX orders have been removed
Shutdown complete. Goodbye, Arby.
2021-01-07 15:50:45,942 INFO exited: arby (exit status 0; expected)
2021-01-07 15:50:46,949 INFO spawned: 'arby' with pid 560

> [email protected] start:arby /app
> npm run compile && node dist/arby.js


> [email protected] compile /app
> npm run lint && tsc && cross-os postcompile


> [email protected] lint /app
> eslint . --ext .js,.ts

2021-01-07 15:50:48,203 INFO success: arby entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2021-01-07 03:50:59.5959 [GLOBAL] info: Starting. Hello, Arby.
2021-01-07 03:50:59.5959 [GLOBAL] info: Running with config:
TEST_MODE: false
CEX: BINANCE
LOG_LEVEL: trace
DATA_DIR: /root/.arby
OPENDEX_CERT_PATH: /root/.xud/tls.cert
OPENDEX_RPC_HOST: xud
OPENDEX_RPC_PORT: 28886
MARGIN: 0.015
BASEASSET: ETH
QUOTEASSET: BTC
CEX_BASEASSET: ETH
CEX_QUOTEASSET: BTC
TEST_CENTRALIZED_EXCHANGE_BASEASSET_BALANCE: 123
TEST_CENTRALIZED_EXCHANGE_QUOTEASSET_BALANCE: 321
2021-01-07 03:51:00.000 [OpenDEX] trace: Base asset balance 30 (maxinbound: 0.38769631, maxoutbound: 30) and quote asset balance 2.5 (maxinbound: 2.4499095, maxoutbound: 2.45).
2021-01-07 03:51:00.000 [Centralized] info: Base asset balance 0.37296423 and quote asset balance 0.01946175
2021-01-07 03:51:00.000 [Centralized] trace: ETHBTC established connection to wss://stream.binance.com:9443/ws/ethbtc@aggTrade
2021-01-07 03:51:00.000 [Centralized] trace: heartbeat from ETHBTC socket
2021-01-07 03:51:01.011 [Centralized] trace: New price: 0.03214
2021-01-07 03:51:01.011 [OpenDEX] trace: Replacing ETH/BTC buy order with id arby-ETH/BTC-buy-order, quantity 0.36923458 and price 0.0316579
2021-01-07 03:51:01.011 [OpenDEX] trace: Replacing ETH/BTC sell order with id arby-ETH/BTC-sell-order, quantity 0.59947518 and price 0.0326221
2021-01-07 03:51:01.011 [OpenDEX] trace: Creating ETH/BTC buy order with id arby-ETH/BTC-buy-order, quantity 0.36923458 and price 0.0316579
2021-01-07 03:51:01.011 [OpenDEX] trace: Order arby-ETH/BTC-buy-order created
2021-01-07 03:51:01.011 [OpenDEX] trace: Order arby-ETH/BTC-sell-order created
2021-01-07 03:51:02.022 [Centralized] trace: New price: 0.032141
2021-01-07 03:51:02.022 [Centralized] trace: New price: 0.03214
@raladev raladev added the P1 top priority label Jan 7, 2021
@raladev raladev assigned ghost Jan 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 top priority
Projects
None yet
Development

No branches or pull requests

1 participant