Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(perf): protorev tracker coin array #7240

Merged
merged 42 commits into from
Jan 9, 2024
Merged

chore(perf): protorev tracker coin array #7240

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
2e919c6
remove tx fee tracker
czarcas7ic Jan 3, 2024
3785022
add change log
czarcas7ic Jan 3, 2024
25b911d
track array of coins for performance
czarcas7ic Jan 4, 2024
3f217b9
Merge branch 'main' into adam/protorev-tracker-perf
czarcas7ic Jan 4, 2024
ccf65fb
changelog
czarcas7ic Jan 4, 2024
2d88b49
lints
czarcas7ic Jan 4, 2024
91d647e
lints
czarcas7ic Jan 4, 2024
867a1d0
fixes
czarcas7ic Jan 5, 2024
b98972c
Merge branch 'main' into adam/protorev-tracker-perf
czarcas7ic Jan 5, 2024
bc9d9b0
Auto: update go.mod after push to adam/protorev-tracker-perf that mod…
invalid-email-address Jan 5, 2024
12c0f74
fix tests
czarcas7ic Jan 5, 2024
ef5176b
event
czarcas7ic Jan 5, 2024
2e78405
nil error
czarcas7ic Jan 5, 2024
69c2df1
expect accounting height to always be exported
czarcas7ic Jan 5, 2024
e3c0e33
range backwards and add test
czarcas7ic Jan 6, 2024
ee425f0
reduce code duplication
czarcas7ic Jan 6, 2024
759831d
Auto: update go.mod after push to adam/protorev-tracker-perf that mod…
invalid-email-address Jan 6, 2024
7a09aef
init image should not check seq number
czarcas7ic Jan 6, 2024
11013db
revert change to proto rev bc might expect error case
czarcas7ic Jan 6, 2024
286e914
revert init tag for now
czarcas7ic Jan 6, 2024
68bf4ec
config tag
czarcas7ic Jan 6, 2024
a16bea6
lint spelling
czarcas7ic Jan 6, 2024
dfca598
init tag
czarcas7ic Jan 6, 2024
6d6cd16
revert init image
czarcas7ic Jan 6, 2024
ba0c68e
remove osmoutils method for coins to coin array
czarcas7ic Jan 7, 2024
166f039
Merge branch 'main' into adam/protorev-tracker-perf
czarcas7ic Jan 7, 2024
f19b819
continue tracking from same accounting height pre upgrade
czarcas7ic Jan 8, 2024
0a054b0
Merge branch 'main' into adam/protorev-tracker-perf
czarcas7ic Jan 8, 2024
a59dd33
Auto: update go.mod after push to adam/protorev-tracker-perf that mod…
invalid-email-address Jan 8, 2024
a27e3c8
update store helper
czarcas7ic Jan 9, 2024
673a360
Auto: update go.mod after push to adam/protorev-tracker-perf that mod…
invalid-email-address Jan 9, 2024
ba52471
Merge branch 'main' into adam/protorev-tracker-perf
czarcas7ic Jan 9, 2024
fd1a39d
go mod updates
czarcas7ic Jan 9, 2024
a582fa1
just use int
czarcas7ic Jan 9, 2024
46bd310
go mod updates
czarcas7ic Jan 9, 2024
c3f306d
revert statistics.go
czarcas7ic Jan 9, 2024
c665113
fix store key
czarcas7ic Jan 9, 2024
709b70d
update go mods
czarcas7ic Jan 9, 2024
b83740e
dont double append prefix
czarcas7ic Jan 9, 2024
ac2f006
update go mod
czarcas7ic Jan 9, 2024
17cb7c0
Merge branch 'main' into adam/protorev-tracker-perf
czarcas7ic Jan 9, 2024
1216b17
go mod changes
czarcas7ic Jan 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
* [#6991](https://github.com/osmosis-labs/osmosis/pull/6991) Fix: total liquidity poolmanager grpc gateway query
* [#7149](https://github.com/osmosis-labs/osmosis/pull/7149) Fix double emitting CacheCtx events (e.g. Epoch, Superfluid, CL)
* [#7237](https://github.com/osmosis-labs/osmosis/pull/7237) Removes tx_fee_tracker from the proto rev tracker, no longer tracks in state.
* [#7240](https://github.com/osmosis-labs/osmosis/pull/7240) Protorev tracker now tracks a coin array to improve gas efficiency.

### Features

Expand Down
7 changes: 4 additions & 3 deletions app/upgrades/v21/upgrades_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ import (

upgradetypes "github.com/cosmos/cosmos-sdk/x/upgrade/types"

"github.com/osmosis-labs/osmosis/osmoutils"
v21 "github.com/osmosis-labs/osmosis/v21/app/upgrades/v21"

abci "github.com/cometbft/cometbft/abci/types"
Expand Down Expand Up @@ -52,10 +53,10 @@ func (s *UpgradeTestSuite) TestUpgrade() {
s.Require().Equal(v21UpgradeHeight, allProtocolRevenue.TakerFeesTracker.HeightAccountingStartsFrom)
// s.Require().Equal(v21UpgradeHeight, allProtocolRevenue.TxFeesTracker.HeightAccountingStartsFrom)
// All values should be nill except for the cyclic arb profits, which should start at the value it was at time of upgrade
s.Require().Equal(sdk.Coins(nil), allProtocolRevenue.TakerFeesTracker.TakerFeesToCommunityPool)
s.Require().Equal(sdk.Coins(nil), allProtocolRevenue.TakerFeesTracker.TakerFeesToStakers)
s.Require().Equal([]sdk.Coin{}, allProtocolRevenue.TakerFeesTracker.TakerFeesToCommunityPool)
s.Require().Equal([]sdk.Coin{}, allProtocolRevenue.TakerFeesTracker.TakerFeesToStakers)
// s.Require().Equal(sdk.Coins(nil), allProtocolRevenue.TxFeesTracker.TxFees)
s.Require().Equal(cyclicArbProfits, allProtocolRevenue.CyclicArbTracker.CyclicArb)
s.Require().Equal(osmoutils.ConvertCoinsToCoinArray(cyclicArbProfits), allProtocolRevenue.CyclicArbTracker.CyclicArb)

}

Expand Down
3 changes: 3 additions & 0 deletions app/upgrades/v22/upgrades.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ func CreateUpgradeHandler(
return nil, err
}

// For sake of simplicity, we restart the taker fee tracker accounting height.
keepers.PoolManagerKeeper.SetTakerFeeTrackerStartHeight(ctx, ctx.BlockHeight())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we be keeping continuinity? THis is going to be much harder for every integrator for the future if we don't, no?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented here f19b819

@ValarDragon would appreciate an ack on the implementation so it doesn't get swept under the rug when merging


return migrations, nil
}
}
53 changes: 24 additions & 29 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ require (
github.com/go-redis/redis v6.15.9+incompatible
github.com/golang/mock v1.6.0
github.com/golang/protobuf v1.5.3
github.com/golangci/golangci-lint v1.54.2
github.com/golangci/golangci-lint v1.55.2
github.com/gorilla/mux v1.8.1
github.com/grpc-ecosystem/grpc-gateway v1.16.0
github.com/iancoleman/orderedmap v0.3.0
Expand Down Expand Up @@ -65,21 +65,18 @@ require (
cosmossdk.io/core v0.5.1 // indirect
cosmossdk.io/depinject v1.0.0-alpha.4 // indirect
cosmossdk.io/log v1.2.1 // indirect
github.com/4meepo/tagalign v1.3.2 // indirect
github.com/Abirdcfly/dupword v0.0.12 // indirect
github.com/Abirdcfly/dupword v0.0.13 // indirect
github.com/ChainSafe/go-schnorrkel v0.0.0-20200405005733-88cbf1b4c40d // indirect
github.com/Djarvur/go-err113 v0.1.0 // indirect
github.com/GaijinEntertainment/go-exhaustruct/v3 v3.1.0 // indirect
github.com/GaijinEntertainment/go-exhaustruct/v2 v2.3.0 // indirect
github.com/OneOfOne/xxhash v1.2.8 // indirect
github.com/OpenPeeDeeP/depguard/v2 v2.1.0 // indirect
github.com/alexkohler/nakedret/v2 v2.0.2 // indirect
github.com/OpenPeeDeeP/depguard v1.1.1 // indirect
github.com/alingse/asasalint v0.0.11 // indirect
github.com/aws/aws-sdk-go v1.44.224 // indirect
github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d // indirect
github.com/btcsuite/btcd/btcec/v2 v2.3.2 // indirect
github.com/btcsuite/btcd/btcutil v1.1.3 // indirect
github.com/btcsuite/btcd/chaincfg/chainhash v1.0.2 // indirect
github.com/butuzov/mirror v1.1.0 // indirect
github.com/ccojocar/zxcvbn-go v1.0.1 // indirect
github.com/chzyer/readline v1.5.1 // indirect
github.com/cockroachdb/apd/v2 v2.0.2 // indirect
Expand Down Expand Up @@ -115,6 +112,7 @@ require (
github.com/hashicorp/go-safetemp v1.0.0 // indirect
github.com/huandu/skiplist v1.2.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/junk1tm/musttag v0.4.5 // indirect
github.com/kkHAIKE/contextcheck v1.1.4 // indirect
github.com/kr/pretty v0.3.1 // indirect
github.com/kr/text v0.2.0 // indirect
Expand All @@ -128,7 +126,7 @@ require (
github.com/mitchellh/go-testing-interface v1.14.1 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/nunnatsa/ginkgolinter v0.13.5 // indirect
github.com/nunnatsa/ginkgolinter v0.14.1 // indirect
github.com/otiai10/copy v1.11.0 // indirect
github.com/rogpeppe/go-internal v1.11.0 // indirect
github.com/sagikazarmark/locafero v0.4.0 // indirect
Expand All @@ -144,11 +142,8 @@ require (
github.com/ulikunitz/xz v0.5.11 // indirect
github.com/valyala/bytebufferpool v1.0.0 // indirect
github.com/valyala/fasttemplate v1.2.2 // indirect
github.com/xen0n/gosmopolitan v1.2.1 // indirect
github.com/ykadowak/zerologlint v0.1.3 // indirect
github.com/zimmski/go-mutesting v0.0.0-20210610104036-6d9217011a00 // indirect
github.com/zondax/ledger-go v0.14.3 // indirect
go.tmz.dev/musttag v0.7.2 // indirect
golang.org/x/oauth2 v0.15.0 // indirect
golang.org/x/time v0.5.0 // indirect
golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
Expand Down Expand Up @@ -182,19 +177,19 @@ require (
github.com/bkielbasa/cyclop v1.2.1 // indirect
github.com/blizzy78/varnamelen v0.8.0 // indirect
github.com/bombsimon/wsl/v3 v3.4.0 // indirect
github.com/breml/bidichk v0.2.4 // indirect
github.com/breml/errchkjson v0.3.1 // indirect
github.com/butuzov/ireturn v0.2.0 // indirect
github.com/breml/bidichk v0.2.7 // indirect
github.com/breml/errchkjson v0.3.6 // indirect
github.com/butuzov/ireturn v0.2.2 // indirect
github.com/cenkalti/backoff/v4 v4.2.1 // indirect
github.com/cespare/xxhash v1.1.0 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/charithe/durationcheck v0.0.10 // indirect
github.com/chavacava/garif v0.0.0-20230227094218-b8c73b2037b8 // indirect
github.com/chavacava/garif v0.1.0 // indirect
github.com/confio/ics23/go v0.9.1 // indirect
github.com/containerd/continuity v0.3.0 // indirect
github.com/cosmos/btcutil v1.0.5
github.com/cosmos/ledger-cosmos-go v0.12.4 // indirect
github.com/daixiang0/gci v0.11.0 // indirect
github.com/daixiang0/gci v0.11.2 // indirect
github.com/danieljoos/wincred v1.1.2 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/denis-tingaikin/go-header v0.4.3 // indirect
Expand Down Expand Up @@ -235,11 +230,11 @@ require (
github.com/golangci/check v0.0.0-20180506172741-cfe4005ccda2 // indirect
github.com/golangci/dupl v0.0.0-20180902072040-3e9179ac440a // indirect
github.com/golangci/go-misc v0.0.0-20220329215616-d24fe342adfe // indirect
github.com/golangci/gofmt v0.0.0-20220901101216-f2edd75033f2 // indirect
github.com/golangci/gofmt v0.0.0-20231018234816-f50ced29576e // indirect
github.com/golangci/lint-1 v0.0.0-20191013205115-297bf364a8e0 // indirect
github.com/golangci/maligned v0.0.0-20180506175553-b1d89398deca // indirect
github.com/golangci/misspell v0.4.1 // indirect
github.com/golangci/revgrep v0.0.0-20220804021717-745bb2f7c2e6 // indirect
github.com/golangci/revgrep v0.5.2 // indirect
github.com/golangci/unconvert v0.0.0-20180507085042-28b1c447d1f4 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/google/gofuzz v1.2.0 // indirect
Expand All @@ -266,7 +261,7 @@ require (
github.com/improbable-eng/grpc-web v0.15.0 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/jessevdk/go-flags v1.5.0 // indirect
github.com/jgautheron/goconst v1.5.1 // indirect
github.com/jgautheron/goconst v1.6.0 // indirect
github.com/jingyugao/rowserrcheck v1.1.1 // indirect
github.com/jirfag/go-printf-func-name v0.0.0-20200119135958-7558a9eaa5af // indirect
github.com/jmhodges/levigo v1.0.0 // indirect
Expand All @@ -291,7 +286,7 @@ require (
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/mattn/go-runewidth v0.0.14 // indirect
github.com/mbilski/exhaustivestruct v1.2.0 // indirect
github.com/mgechev/revive v1.3.2 // indirect
github.com/mgechev/revive v1.3.4 // indirect
github.com/minio/highwayhash v1.0.2 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
Expand All @@ -308,7 +303,7 @@ require (
github.com/pelletier/go-toml/v2 v2.1.0 // indirect
github.com/petermattis/goid v0.0.0-20230317030725-371a4b8eda08 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/polyfloyd/go-errorlint v1.4.4 // indirect
github.com/polyfloyd/go-errorlint v1.4.5 // indirect
github.com/prometheus/client_golang v1.18.0
github.com/prometheus/client_model v0.5.0 // indirect
github.com/prometheus/common v0.45.0 // indirect
Expand All @@ -322,10 +317,10 @@ require (
github.com/rs/cors v1.9.0 // indirect
github.com/rs/zerolog v1.30.0 // indirect
github.com/ryancurrah/gomodguard v1.3.0 // indirect
github.com/ryanrolds/sqlclosecheck v0.4.0 // indirect
github.com/ryanrolds/sqlclosecheck v0.5.1 // indirect
github.com/sanposhiho/wastedassign/v2 v2.0.7 // indirect
github.com/sasha-s/go-deadlock v0.3.1 // indirect
github.com/securego/gosec/v2 v2.17.0 // indirect
github.com/securego/gosec/v2 v2.18.2 // indirect
github.com/shazow/go-diff v0.0.0-20160112020656-b6b7b6733b8c // indirect
github.com/sirupsen/logrus v1.9.3
github.com/sivchari/containedctx v1.0.3 // indirect
Expand All @@ -340,13 +335,13 @@ require (
github.com/syndtr/goleveldb v1.0.1-0.20220721030215-126854af5e6d
github.com/tdakkota/asciicheck v0.2.0 // indirect
github.com/tendermint/go-amino v0.16.0 // indirect
github.com/tetafro/godot v1.4.14 // indirect
github.com/tetafro/godot v1.4.15 // indirect
github.com/timakin/bodyclose v0.0.0-20230421092635-574207250966 // indirect
github.com/tomarrell/wrapcheck/v2 v2.8.1 // indirect
github.com/tommy-muehle/go-mnd/v2 v2.5.1 // indirect
github.com/ultraware/funlen v0.1.0 // indirect
github.com/ultraware/whitespace v0.0.5 // indirect
github.com/uudashr/gocognit v1.0.7 // indirect
github.com/uudashr/gocognit v1.1.2 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20190905194746-02993c407bfb // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xeipuuv/gojsonschema v1.2.0 // indirect
Expand All @@ -355,20 +350,20 @@ require (
github.com/zimmski/go-tool v0.0.0-20150119110811-2dfdc9ac8439 // indirect
github.com/zimmski/osutil v0.0.0-20190128123334-0d0b3ca231ac // indirect
github.com/zondax/hid v0.9.2 // indirect
gitlab.com/bosi/decorder v0.4.0 // indirect
gitlab.com/bosi/decorder v0.4.1 // indirect
go.etcd.io/bbolt v1.3.7 // indirect
go.opencensus.io v0.24.0 // indirect
golang.org/x/crypto v0.17.0 // indirect
golang.org/x/exp/typeparams v0.0.0-20230307190834-24139beb5833 // indirect
golang.org/x/mod v0.12.0 // indirect
golang.org/x/mod v0.13.0 // indirect
golang.org/x/net v0.19.0 // indirect
golang.org/x/sync v0.5.0 // indirect
golang.org/x/sys v0.15.0 // indirect
golang.org/x/term v0.15.0 // indirect
golang.org/x/text v0.14.0 // indirect
golang.org/x/tools v0.13.0 // indirect
golang.org/x/tools v0.14.0 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
honnef.co/go/tools v0.4.5 // indirect
honnef.co/go/tools v0.4.6 // indirect
mvdan.cc/interfacer v0.0.0-20180901003855-c20040233aed // indirect
mvdan.cc/lint v0.0.0-20170908181259-adc824a0674b // indirect
mvdan.cc/unparam v0.0.0-20230312165513-e84e2d14e3b8 // indirect
Expand Down
Loading