-
Notifications
You must be signed in to change notification settings - Fork 602
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Protorev smarter logic and more testing #4181
Conversation
5ff4ed4
to
8a27d62
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed, left 1 comment about the changes made. Going to fix up the doomsday test given the new point refund logic.
x/protorev/keeper/rebalance.go
Outdated
if err != nil { | ||
return sdk.Coin{}, sdk.ZeroInt(), err | ||
} else if minInProfit.LTE(sdk.ZeroInt()) { | ||
return sdk.Coin{}, sdk.ZeroInt(), fmt.Errorf("no profit for route %d", route.Route.PoolIds()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I imagine a majority of small swaps will result in no profits, so this will throw an error often. Feels kinda like we want to view no profit in a route as a non-error. maybe we log in some other way if we want to log?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Resolved in ad5ea73
Another thing I noticed is that we still have a hardcoded denom for atom: osmosis/x/protorev/types/constants.go Line 8 in e6c1ba6
This is no longer used in protorev itself, but used throughout our test suite. Is there a way for us to move this easily to only be for test suite and not in any of the core module files (to avoid changing all the tests that reference it, but not have any unnecessary code in core module files). I think this is a non-issue though |
Update, I deprecated it from the module. |
// Hot routes that are configured on genesis | ||
repeated TokenPairArbRoutes token_pairs = 2 [ (gogoproto.nullable) = false ]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for reviewers: deprecating the use of hot routes in genesis since we will not be using it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hrm, I feel like it should be added back (in a subsequent non-release affecting PR), so that ExportGenesis -> ImportGenesis still works. (So state export based testnets for instance can use this)
The ideal is that for state' = import(export(state))
, that state'
is effectively the same as state
.
MinGasPrice = "0.000" | ||
IbcSendAmount = 3300000000 | ||
ValidatorWalletName = "val" | ||
// chainA | ||
ChainAID = "osmo-test-a" | ||
OsmoBalanceA = 200000000000 | ||
OsmoBalanceA = 20000000000000 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for reviewers: osmo balance being increased so that the pools that protorev testing creates have enough funds
UstBalanceA = 500000000000000 | ||
LuncBalanceA = 500000000000000 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for reviewers: denoms used in protorev testing
@@ -52,24 +52,29 @@ const ( | |||
AtomDenom = "uatom" | |||
OsmoIBCDenom = "ibc/ED07A3391A112B175915CD8FAF43A2DA8E4790EDE12566649D0C2F97716B8518" | |||
StakeIBCDenom = "ibc/C053D637CCA2A2BA030E2C5EE1B28A16F71CCB0E45E8BE52766DC1B241B7787" | |||
UstIBCDenom = "ibc/BE1BB42D4BE3C30D50B68D7C41DB4DFCE9678E8EF8C539F6E6A9345048894FCC" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @davidterpay . Please manually backport just this file to v14.x
branch.
Once merged, please ping me, and I will give you an image tag that will need to be put here:
osmosis/tests/e2e/containers/config.go
Line 30 in dd1d910
previousVersionInitTag = "v14.x-f6813bcb-1674140667-manual" |
We do this to be able to build genesis and config files from v14.x to start up Osmosis containers of that version and perform the upgrade to the next (v15) version in your branch.
x/protorev/types/constants.go
Outdated
var StepSize = sdk.NewInt(1_000_000) | ||
// ExtendedMaxInputAmount is the upper bound index for finding the optimal in amount | ||
// when determining route profitability for an arb that's above the default range (2 ^ 17) = 131,072 | ||
var ExtendedMaxInputAmount = sdk.NewInt(131_072) | ||
|
||
// Max iterations for binary search (log2(16_384) = 14) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// Max iterations for binary search (log2(16_384) = 14) | |
// Max iterations for binary search (log2(131_072) = 17) |
|
||
for i := 0; i < b.N; i++ { | ||
b.StartTimer() | ||
suite.App.ProtoRevKeeper.UpdatePools(suite.Ctx) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This needs enough pools setup to be meaningful, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah awesome! Didn't realize this was the same setup test with everything else, that makes sense!
Added: - 4 pool mainnet route test - 2 pool mainnet route test - Non-osmo/atom denom test
- Currently panics due to solveCFMMBinarySearchMulti in stableswap calculation
- Tests that the per tx limit works properly - Tests that the per block limit works properly
- Adds a pre-check before the binary search that tells us if we need to run the binary search at all - Reduces overall computation/time by avoiding binary searching profit amounts over routes without any profit opportunity
- Added "test/3" denom to test non-osmo/atom denoms in previous PR - didn't notice it broke one test that expected us to only have 2 denoms, so adding "test/3" here so everything passes
- This implements a way for us to have a binary search bound that can increase in size to account for large trades above the default range - I believe this is the minimum-amount-of-new-code approach, but may not be the optimal approach (what this method avoids is having to save the amount in or amount out of the original swap, and then backtracking/converting that amount to the base denom for an arb, and creating bounds off of that)
- Doubling isn't the wanted behavior (only wanted if lower bound is 1), just increasing bound by the same range size is wanted (so adding MaxInputAmount achieves this)
- Increases max iterations to 17 to allow for situation when we need to increase the upper bound - Add new ExtendedMaxInputAmount variable for us to use as this new max bound range - Replace bound changing logic from iteratively changing the range to immediately giving our max range
- Tests binary search range extension logic works properly
- This currently fails, this is on purpose so we don't merge the PR and think we are good until this passes and the panics are handled properly
- Makes pool 41 reserves to have an arb opportunity in the doomsday routes - Changes tx limit and block limit specifically for doomsday testing
x/protorev/genesis.go
Outdated
panic(err) | ||
} | ||
|
||
// Update the pools on genesis | ||
if err := k.UpdatePools(ctx); err != nil { | ||
// Setting the admin account by default to a trusted address |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we document the source of this address? (Perhaps link to the commonwealth post once it exists)
(Lets make a followup issue for this)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup, will do. Just for reference this is an account we created and are storing on a ledger. We will use it to enable authz (which we have already test) for a different account so that the admin keys are safely stored.
eb19820
to
450fd52
Compare
{ // Pool 41 - Used for doomsday testing | ||
initialLiquidity: sdk.NewCoins( | ||
sdk.NewCoin("usdc", sdk.NewInt(1000000000000000)), | ||
sdk.NewCoin("usdt", sdk.NewInt(1000000000000000)), | ||
sdk.NewCoin("busd", sdk.NewInt(2000000000000000)), | ||
), | ||
poolParams: stableswap.PoolParams{ | ||
SwapFee: sdk.NewDecWithPrec(1, 4), | ||
ExitFee: sdk.NewDecWithPrec(0, 2), | ||
}, | ||
scalingFactors: []uint64{1, 1, 1}, | ||
}, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding all these pools to whats protorev configured in the tests!
inputCoin, profit, err := k.FindMaxProfitForRoute(ctx, route, inputDenom) | ||
if err != nil { | ||
k.Logger(ctx).Error("Error finding max profit for route: ", err) | ||
// Get the total number of pool points that can be consumed in this transaction |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice job on all the code comments in this file, quite easy to follow the logic :)
// If a cyclic arb exists with an optimal amount in above our minimum amount in, | ||
// then inputting the minimum amount in will result in a profit. So we check for that first. | ||
// If there is no profit, then we can return early and not run the binary search. | ||
_, minInProfit, err := k.EstimateMultihopProfit(ctx, inputDenom, curLeft.Mul(route.StepSize), route.Route) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice job on this! LGTM to merge!
* Add generalized tests Added: - 4 pool mainnet route test - 2 pool mainnet route test - Non-osmo/atom denom test * Add stableswap doomsday test - Currently panics due to solveCFMMBinarySearchMulti in stableswap calculation * Add tests for pool point limits - Tests that the per tx limit works properly - Tests that the per block limit works properly * Add pre-check before binary search - Adds a pre-check before the binary search that tells us if we need to run the binary search at all - Reduces overall computation/time by avoiding binary searching profit amounts over routes without any profit opportunity * Add denom to make test pass - Added "test/3" denom to test non-osmo/atom denoms in previous PR - didn't notice it broke one test that expected us to only have 2 denoms, so adding "test/3" here so everything passes * Implement minimum change to have smarter binary search bounds - This implements a way for us to have a binary search bound that can increase in size to account for large trades above the default range - I believe this is the minimum-amount-of-new-code approach, but may not be the optimal approach (what this method avoids is having to save the amount in or amount out of the original swap, and then backtracking/converting that amount to the base denom for an arb, and creating bounds off of that) * Switch range increasing logic - Doubling isn't the wanted behavior (only wanted if lower bound is 1), just increasing bound by the same range size is wanted (so adding MaxInputAmount achieves this) * Add logic to extend search bounds when finding optimal amount in - Increases max iterations to 17 to allow for situation when we need to increase the upper bound - Add new ExtendedMaxInputAmount variable for us to use as this new max bound range - Replace bound changing logic from iteratively changing the range to immediately giving our max range * Move range extension into it's own helper function * basic benchmark testing for posthandler and epoch hook * Add SwapAmountOut Test * Add extended range test - Tests binary search range extension logic works properly * Add panic catching test - This currently fails, this is on purpose so we don't merge the PR and think we are good until this passes and the panics are handled properly * dynamic step size * pool points only incremented if profitable * adding sanity checks for pool point calcs, nits * Update doomsday testing accounting for refund system - Makes pool 41 reserves to have an arb opportunity in the doomsday routes - Changes tx limit and block limit specifically for doomsday testing * Return nil for no profit opportunity * adding E2E, removing atom as a base denom, more testing for post handler * find max profit test fix * nit * backporting version tag to 14.x * comment update for protorev admin account --------- Co-authored-by: David Terpay <[email protected]> (cherry picked from commit 23b13a1) # Conflicts: # tests/e2e/configurer/chain/queries.go
* Protorev smarter logic and more testing (#4181) * Add generalized tests Added: - 4 pool mainnet route test - 2 pool mainnet route test - Non-osmo/atom denom test * Add stableswap doomsday test - Currently panics due to solveCFMMBinarySearchMulti in stableswap calculation * Add tests for pool point limits - Tests that the per tx limit works properly - Tests that the per block limit works properly * Add pre-check before binary search - Adds a pre-check before the binary search that tells us if we need to run the binary search at all - Reduces overall computation/time by avoiding binary searching profit amounts over routes without any profit opportunity * Add denom to make test pass - Added "test/3" denom to test non-osmo/atom denoms in previous PR - didn't notice it broke one test that expected us to only have 2 denoms, so adding "test/3" here so everything passes * Implement minimum change to have smarter binary search bounds - This implements a way for us to have a binary search bound that can increase in size to account for large trades above the default range - I believe this is the minimum-amount-of-new-code approach, but may not be the optimal approach (what this method avoids is having to save the amount in or amount out of the original swap, and then backtracking/converting that amount to the base denom for an arb, and creating bounds off of that) * Switch range increasing logic - Doubling isn't the wanted behavior (only wanted if lower bound is 1), just increasing bound by the same range size is wanted (so adding MaxInputAmount achieves this) * Add logic to extend search bounds when finding optimal amount in - Increases max iterations to 17 to allow for situation when we need to increase the upper bound - Add new ExtendedMaxInputAmount variable for us to use as this new max bound range - Replace bound changing logic from iteratively changing the range to immediately giving our max range * Move range extension into it's own helper function * basic benchmark testing for posthandler and epoch hook * Add SwapAmountOut Test * Add extended range test - Tests binary search range extension logic works properly * Add panic catching test - This currently fails, this is on purpose so we don't merge the PR and think we are good until this passes and the panics are handled properly * dynamic step size * pool points only incremented if profitable * adding sanity checks for pool point calcs, nits * Update doomsday testing accounting for refund system - Makes pool 41 reserves to have an arb opportunity in the doomsday routes - Changes tx limit and block limit specifically for doomsday testing * Return nil for no profit opportunity * adding E2E, removing atom as a base denom, more testing for post handler * find max profit test fix * nit * backporting version tag to 14.x * comment update for protorev admin account --------- Co-authored-by: David Terpay <[email protected]> (cherry picked from commit 23b13a1) # Conflicts: # tests/e2e/configurer/chain/queries.go * Fix merge conflict --------- Co-authored-by: Jeremy Liu <[email protected]> Co-authored-by: Dev Ojha <[email protected]>
* Add generalized tests Added: - 4 pool mainnet route test - 2 pool mainnet route test - Non-osmo/atom denom test * Add stableswap doomsday test - Currently panics due to solveCFMMBinarySearchMulti in stableswap calculation * Add tests for pool point limits - Tests that the per tx limit works properly - Tests that the per block limit works properly * Add pre-check before binary search - Adds a pre-check before the binary search that tells us if we need to run the binary search at all - Reduces overall computation/time by avoiding binary searching profit amounts over routes without any profit opportunity * Add denom to make test pass - Added "test/3" denom to test non-osmo/atom denoms in previous PR - didn't notice it broke one test that expected us to only have 2 denoms, so adding "test/3" here so everything passes * Implement minimum change to have smarter binary search bounds - This implements a way for us to have a binary search bound that can increase in size to account for large trades above the default range - I believe this is the minimum-amount-of-new-code approach, but may not be the optimal approach (what this method avoids is having to save the amount in or amount out of the original swap, and then backtracking/converting that amount to the base denom for an arb, and creating bounds off of that) * Switch range increasing logic - Doubling isn't the wanted behavior (only wanted if lower bound is 1), just increasing bound by the same range size is wanted (so adding MaxInputAmount achieves this) * Add logic to extend search bounds when finding optimal amount in - Increases max iterations to 17 to allow for situation when we need to increase the upper bound - Add new ExtendedMaxInputAmount variable for us to use as this new max bound range - Replace bound changing logic from iteratively changing the range to immediately giving our max range * Move range extension into it's own helper function * basic benchmark testing for posthandler and epoch hook * Add SwapAmountOut Test * Add extended range test - Tests binary search range extension logic works properly * Add panic catching test - This currently fails, this is on purpose so we don't merge the PR and think we are good until this passes and the panics are handled properly * dynamic step size * pool points only incremented if profitable * adding sanity checks for pool point calcs, nits * Update doomsday testing accounting for refund system - Makes pool 41 reserves to have an arb opportunity in the doomsday routes - Changes tx limit and block limit specifically for doomsday testing * Return nil for no profit opportunity * adding E2E, removing atom as a base denom, more testing for post handler * find max profit test fix * nit * backporting version tag to 14.x * comment update for protorev admin account --------- Co-authored-by: David Terpay <[email protected]>
* Add generalized tests Added: - 4 pool mainnet route test - 2 pool mainnet route test - Non-osmo/atom denom test * Add stableswap doomsday test - Currently panics due to solveCFMMBinarySearchMulti in stableswap calculation * Add tests for pool point limits - Tests that the per tx limit works properly - Tests that the per block limit works properly * Add pre-check before binary search - Adds a pre-check before the binary search that tells us if we need to run the binary search at all - Reduces overall computation/time by avoiding binary searching profit amounts over routes without any profit opportunity * Add denom to make test pass - Added "test/3" denom to test non-osmo/atom denoms in previous PR - didn't notice it broke one test that expected us to only have 2 denoms, so adding "test/3" here so everything passes * Implement minimum change to have smarter binary search bounds - This implements a way for us to have a binary search bound that can increase in size to account for large trades above the default range - I believe this is the minimum-amount-of-new-code approach, but may not be the optimal approach (what this method avoids is having to save the amount in or amount out of the original swap, and then backtracking/converting that amount to the base denom for an arb, and creating bounds off of that) * Switch range increasing logic - Doubling isn't the wanted behavior (only wanted if lower bound is 1), just increasing bound by the same range size is wanted (so adding MaxInputAmount achieves this) * Add logic to extend search bounds when finding optimal amount in - Increases max iterations to 17 to allow for situation when we need to increase the upper bound - Add new ExtendedMaxInputAmount variable for us to use as this new max bound range - Replace bound changing logic from iteratively changing the range to immediately giving our max range * Move range extension into it's own helper function * basic benchmark testing for posthandler and epoch hook * Add SwapAmountOut Test * Add extended range test - Tests binary search range extension logic works properly * Add panic catching test - This currently fails, this is on purpose so we don't merge the PR and think we are good until this passes and the panics are handled properly * dynamic step size * pool points only incremented if profitable * adding sanity checks for pool point calcs, nits * Update doomsday testing accounting for refund system - Makes pool 41 reserves to have an arb opportunity in the doomsday routes - Changes tx limit and block limit specifically for doomsday testing * Return nil for no profit opportunity * adding E2E, removing atom as a base denom, more testing for post handler * find max profit test fix * nit * backporting version tag to 14.x * comment update for protorev admin account --------- Co-authored-by: David Terpay <[email protected]>
What is the purpose of the change
This PR adds more testing to
x/protorev
and improves the arb calculation logic to be more efficient.Brief Changelog
Testing and Verifying