Enable Parallel Transfer Test Suite #6721

chatton · 2024-06-27T14:05:00Z

Description

The diff is a little large for this PR, but a lot of the changes are small shuffles of setup logic. The key changes are

The channel creation in SetupTest actually didn't workout that well when enabling t.Parallel(). This is because t.Name() is not deterministic when t.Parallel() has been enabled. The effects that were had on the tests was that relayers were not being correctly mapped to tests. This PR updates each of the tests to perform channel creation logic at the start of each test similar to what we had before, but the chain creation still happens in SetupSuite
As a result of the above point, we need to propagate testName to many of the functions to fetch the relevant paths, relayers, channels etc.
We have enabled t.Parallel() for the transfer test suite.
We have moved tests that modified chain params to their own test suite so they do not break the other transfer tests.
Commented out portions of the tests that do not work when sharing the chains such as tests for total denom escrow and begin blocker (which required a chain restart)
We enabled path creation to happen in parallel with the exception of channel creation . I tried to add as much information about the reasoning for this in the code, but basically we need to ensure that the channel that is created is correctly mapped to the test that is executing, so it is not really safe to have channel creation and channel querying happening in parallel as we do not know which one was just created due to the CreateChannel API exposed by interchain test.

Things that need to happen in follow up PRS

Transition the rest of the tests to use the new method, and phase out single test runs. (we can see how this does re: flakiness before going ahead with that)
Modifying compatibility matrices to use the new method.
Address the test cases that needed to be commented out to make this work correctly.

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

* chore: adding proto files for ics20-v2 * chore: add newline

* add CurrentVersion, EscrowVersion, update tests * update hardcoded transfer channel version from interchaintest

--------- Co-authored-by: Charly <[email protected]>

…ods (#6126)

…shalling/conversion (#6226) * chore: adding proto files for ics20-v2 * chore: add newline * chore: modify MsgTransfer to accept coins instead of coin * chore: reverted unintentional comment changes * chore: reverted unintentional comment changes * chore: adding test for conversion fn * chore: fix e2e linter * chore: adding docs * chore: addressing PR feedback * chore: remove duplicate import * chore: addressing PR feedback * chore: moved extration logic into internal package * chore: commented out failing test * chore: adding link to issue * chore: remove duplicate import * chore: fix linting errors * FungibleTokenPacketData interface methods + tests * linter * wip: token validation * update trace identifier validation in Token + tests * rm Printf * update with pr review * add CurrentVersion, EscrowVersion, update tests * pr review * fix e2e tests * pr review * update e2e test version * linter * update hardcoded transfer channel version from interchaintest * update hardcoded transfer channel version from interchaintest * wip packet unmarshaller * wip * wip testing * update test * linter * rm unnecessary version changes * rm unnecessary artifacts * update callbacks test * update comment * pr review * rename getMultiDenomFungibleTokenPacketData to unmarshalPacketDataBytesToICS20V2 --------- Co-authored-by: chatton <[email protected]> Co-authored-by: Carlos Rodriguez <[email protected]>

…allbacks (#6175) * add SupportedVersions array for different ics20 versions, add version checking on channel handshake application callbacks * add tests * update pr review * pr review * last few pr review nits * linter * add version counter proposing * fix missing app versino * update code + tests to return our proposed version if counterparty version is invalid * remove if statement * address review comments: return ics20-2 if counterparty version is not supported --------- Co-authored-by: Carlos Rodriguez <[email protected]>

…transfers (#6252) * add transfer authz code + tests * linter * address review comments --------- Co-authored-by: Carlos Rodriguez <[email protected]>

…ks (#6189) * chore: adding proto files for ics20-v2 * chore: add newline * chore: modify MsgTransfer to accept coins instead of coin * chore: reverted unintentional comment changes * chore: reverted unintentional comment changes * chore: adding test for conversion fn * chore: fix e2e linter * chore: adding docs * chore: addressing PR feedback * chore: remove duplicate import * chore: addressing PR feedback * chore: moved extration logic into internal package * chore: commented out failing test * chore: adding link to issue * chore: remove duplicate import * chore: fix linting errors * FungibleTokenPacketData interface methods + tests * linter * wip: token validation * update trace identifier validation in Token + tests * rm Printf * update with pr review * pr review * linter * rm unused function: linter * wip pr feedback * fix test * pr review * lintttttt * wip: backwards compatibility for transfer rpc * implement changes for ics20-v2 packet data for onRecvPacket, onAcknowledgePacket and onTimeoutPacket * fix callbacks tests * lint --------- Co-authored-by: chatton <[email protected]> Co-authored-by: Charly <[email protected]>

chatton · 2024-07-03T12:53:02Z

.github/workflows/e2e-test-workflow-call.yml

+          cd e2e
+          make e2e-suite entrypoint=${{ matrix.entrypoint }}


this is a copy paste of our existing workflow that just tweaks the make target.

chatton · 2024-07-03T12:53:24Z

.github/workflows/e2e-test-workflow-call.yml

+        include:
+            # for now we explicitly specify this test suite.
+          - entrypoint: TestTransferTestSuite


we can hard code these in as we transition, and then replace this with the dynamically generated list of tests.

chatton · 2024-07-03T12:53:42Z

.github/workflows/e2e.yaml

@@ -63,4 +63,6 @@ jobs:
      chain-b-tag: '${{ needs.determine-image-tag.outputs.simd-tag }}'
      chain-binary: 'simd'
      # on regular PRs we won't run upgrade tests.
-      test-exclusions: 'TestUpgradeTestSuite,TestGrandpaTestSuite,TestIBCWasmUpgradeTestSuite'
+      # NOTE: we are exluding TestTransferTestSuite as we run this full suite instead of each individual test.
+      test-exclusions: 'TestUpgradeTestSuite,TestGrandpaTestSuite,TestIBCWasmUpgradeTestSuite,TestTransferTestSuite'


we exclude TestTransferTestSuite because we no longer run it as individual tests.

chatton · 2024-07-03T12:54:19Z

e2e/tests/core/03-connection/connection_test.go

@@ -33,8 +33,8 @@ type ConnectionTestSuite struct {
 	testsuite.E2ETestSuite
 }

-func (s *ConnectionTestSuite) SetupTest() {
-	s.SetupPaths(ibc.DefaultClientOpts(), s.TransferChannelOptions())
+func (s *ConnectionTestSuite) SetupConnectionTestPath(testName string) (ibc.Relayer, ibc.ChannelOutput) {


it's no longer a lifecycle method, but one we call at the start of tests. the same thing applies for every change of SetupTest

chatton · 2024-07-03T12:54:55Z

e2e/tests/transfer/base_test.go

-	s.SetupPaths(ibc.DefaultClientOpts(), s.TransferChannelOptions())
+// transferTester defines some helper functions that can be used in various test suites
+// that test transfer functionality.
+type transferTester struct {


this is a construct to share functionality of transfer tests across different test suites.

chatton · 2024-07-03T12:55:19Z

e2e/tests/transfer/base_test.go

@@ -467,180 +499,6 @@ func (s *TransferTestSuite) TestMsgTransfer_Timeout_Nonincentivized() {
 	})
 }

-// TestSendEnabledParam tests changing ics20 SendEnabled parameter


moved into its own test suite since it messes with chain params and breaks other tests.

chatton · 2024-07-03T12:55:44Z

e2e/tests/transfer/localhost_test.go

 	chainA, _ := s.GetChains()
+
+	channelVersion := transfertypes.V2


we don't need a channel to be created before this test is run.

chatton · 2024-07-03T12:56:49Z

e2e/tests/upgrades/genesis_test.go

+// TODO: this configuration was originally being applied to `GetChains` in the test body, but it is not
+// actually being propagated correctly. If we want to apply the configuration, we can uncomment this code
+// however the test actually fails when this is done.
+// func (s *GenesisTestSuite) SetupSuite() {
+//	configFileOverrides := make(map[string]any)
+//	appTomlOverrides := make(test.Toml)
+//
+//	appTomlOverrides["halt-height"] = haltHeight
+//	configFileOverrides["config/app.toml"] = appTomlOverrides
+//
+//	s.SetupChains(context.TODO(), nil, func(options *testsuite.ChainOptions) {
+//		// create chains with specified chain configuration options
+//		options.ChainSpecs[0].ConfigFileOverrides = configFileOverrides
+//	})
+// }
+


This was nasty. Previously, the configuration you see here was not actually applied, and if it is applied the test breaks (chain halts before the ica account was registered)

I will create an issue to track this, as the failure is not related to the current issue.

I might be solving this now in the upgrade tests for v9 because I hit a similar issue when doing a bit more setup before the upgrade test. The hardcoding of haltHeight is being removed. Will just assign myself the 6750 so I can verify it later.

Amazing thank you!

chatton · 2024-07-03T12:57:38Z

e2e/testsuite/testsuite.go

-	s.startRelayerFn = func(relayer ibc.Relayer) {
-		// depending on the test, the path names will be different.
-		// whenever a relayer is started, it should use the paths associated with the test the relayer is running in.
-		pathNames, ok := s.testPaths[s.T().Name()]
-		s.Require().True(ok, "path names not found for test %s", s.T().Name())
-
-		err := relayer.StartRelayer(ctx, s.GetRelayerExecReporter(), pathNames...)
-		s.Require().NoError(err, fmt.Sprintf("failed to start relayer: %s", err))
-
-		var chainHeighters []test.ChainHeighter
-		for _, c := range chains {
-			chainHeighters = append(chainHeighters, c)
-		}
-
-		// wait for every chain to produce some blocks before using the relayer.
-		s.Require().NoError(test.WaitForBlocks(ctx, 10, chainHeighters...), "failed to wait for blocks")
-	}
-


the restructuring of code makes it so that this custom fn is no longer required.

chatton · 2024-07-03T14:27:03Z

e2e/tests/transfer/forwarding_test.go

+	s.Require().Len(s.GetChannelsForTest(chainA, testName), 1, "expected one channel on chain A")
+	s.Require().Len(s.GetChannelsForTest(chainB, testName), 2, "expected two channels on chain B")
+	s.Require().Len(s.GetChannelsForTest(chainC, testName), 1, "expected one channel on chain C")


added some additional checks here since this was failing due to not correctly building up the channel mapping correctly in CreatePath

DimitrisJim · 2024-07-03T15:11:38Z

e2e/testsuite/testsuite.go

 	}

-	s.startRelayerFn(ibcrelayer)
+	// wait for every chain to produce some blocks before using the relayer.
+	s.Require().NoError(test.WaitForBlocks(context.TODO(), 10, chainHeighters...), "failed to wait for blocks")


ah we could now remove a couple of these sprinkled after relayer start/restart?

probably yeah, I think we are not super consistent with the waiting

DimitrisJim · 2024-07-03T15:14:26Z

e2e/testsuite/testsuite.go

-		s.channels[s.T().Name()] = make(map[ibc.Chain][]ibc.ChannelOutput)
-	}
+		if _, ok := s.channels[testName][c]; !ok {
+			s.channels[testName][c] = []ibc.ChannelOutput{}


wouldn't a nil slice be apendable either way? is this check needed?

we initialize the map but it will not have c as a key so I think it would blow up if we don't make the empty list here.

DimitrisJim · 2024-07-03T15:25:38Z

e2e/testsuite/testsuite.go


-	s.testPaths[s.T().Name()] = append(s.testPaths[s.T().Name()], pathName)
+		err = relayer.ApplyPacketFilter(ctx, s.T(), r, c.Config().ChainID, s.channels[testName][c])


DimitrisJim · 2024-07-03T15:26:22Z

e2e/testsuite/testsuite.go

-func (s *E2ETestSuite) GetChainAChannel() ibc.ChannelOutput {
-	chainA := s.GetAllChains()[0]
-	return s.GetChannels(chainA)[0]
+func (s *E2ETestSuite) GetChainAChannelForTest(testName string) ibc.ChannelOutput {


think fortest prefix can be dropped (can do in follow up during overall clean up)

DimitrisJim

this looks amazing to me! fairly happy dropping approval and leaving anything required as follow ups.

bznein

LGTM in general, though I'm not really a fan of functions called Setup returning stuff, especially if it's not clear from the full name. Not blocking on it though

bznein · 2024-07-08T13:03:22Z

e2e/scripts/run-e2e.sh

     cd ..
     ENTRY_POINT="$(go run -mod=readonly cmd/build_test_matrix/main.go | jq  -r '.include[] | .entrypoint' | uniq | fzf)"
     cd - > /dev/null
  fi

  # find the name of the file that has this test in it.
-  test_file="$(grep --recursive --files-with-matches './tests' -e "${ENTRY_POINT}")"
+  test_file="$(grep --recursive --files-with-matches './tests' -e "${ENTRY_POINT}(")"


[q] why the added extra parentheses?

If multiple tests start with ${ENTRYPOINT} things blow up. E.g.

TestFoo()
TestFooBar()
TestFooBarBaz()

Right, it makes sense!

bznein · 2024-07-08T13:05:40Z

e2e/tests/core/03-connection/connection_test.go

@@ -33,8 +33,8 @@ type ConnectionTestSuite struct {
 	testsuite.E2ETestSuite
 }

-func (s *ConnectionTestSuite) SetupTest() {
-	s.SetupPaths(ibc.DefaultClientOpts(), s.TransferChannelOptions())
+func (s *ConnectionTestSuite) SetupConnectionTestPath(testName string) (ibc.Relayer, ibc.ChannelOutput) {


I'd love if the name of the method made it clearer what this function is returning, but naming in hard :)

gjermundgaraba

Love this! The e2e suite is really coming together <3

sonarcloud · 2024-07-09T08:39:07Z

Quality Gate passed for 'ibc-go'

Issues
13 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

* Adding proto files for ics20-v2 (#6110) * chore: adding proto files for ics20-v2 * chore: add newline * update amount -> string (#6120) * Update MsgTransfer to accept sdk.Coins instead of sdk.Coin (#6113) * fix: allow base denom with trailing slash (#6148) * imp: add CurrentVersion, EscrowVersion (#6160) * add CurrentVersion, EscrowVersion, update tests * update hardcoded transfer channel version from interchaintest * chore: add function for converting packet data from v1 to v3 (#6116) --------- Co-authored-by: Charly <[email protected]> * chore: implement required `FungibleTokenPacketData` v3 interface methods (#6126) * imp: `getMultiDenomFungibleTokenPacketData`to be used in packet unmarshalling/conversion (#6226) * chore: adding proto files for ics20-v2 * chore: add newline * chore: modify MsgTransfer to accept coins instead of coin * chore: reverted unintentional comment changes * chore: reverted unintentional comment changes * chore: adding test for conversion fn * chore: fix e2e linter * chore: adding docs * chore: addressing PR feedback * chore: remove duplicate import * chore: addressing PR feedback * chore: moved extration logic into internal package * chore: commented out failing test * chore: adding link to issue * chore: remove duplicate import * chore: fix linting errors * FungibleTokenPacketData interface methods + tests * linter * wip: token validation * update trace identifier validation in Token + tests * rm Printf * update with pr review * add CurrentVersion, EscrowVersion, update tests * pr review * fix e2e tests * pr review * update e2e test version * linter * update hardcoded transfer channel version from interchaintest * update hardcoded transfer channel version from interchaintest * wip packet unmarshaller * wip * wip testing * update test * linter * rm unnecessary version changes * rm unnecessary artifacts * update callbacks test * update comment * pr review * rename getMultiDenomFungibleTokenPacketData to unmarshalPacketDataBytesToICS20V2 --------- Co-authored-by: chatton <[email protected]> Co-authored-by: Carlos Rodriguez <[email protected]> * chore: implement version checking for channel handshake application callbacks (#6175) * add SupportedVersions array for different ics20 versions, add version checking on channel handshake application callbacks * add tests * update pr review * pr review * last few pr review nits * linter * add version counter proposing * fix missing app versino * update code + tests to return our proposed version if counterparty version is invalid * remove if statement * address review comments: return ics20-2 if counterparty version is not supported --------- Co-authored-by: Carlos Rodriguez <[email protected]> * imp: update transfer authz implementation to account for multi denom transfers (#6252) * add transfer authz code + tests * linter * address review comments --------- Co-authored-by: Carlos Rodriguez <[email protected]> * ics20-v2: backwards compatibility for transfer rpc and packet callbacks (#6189) * chore: adding proto files for ics20-v2 * chore: add newline * chore: modify MsgTransfer to accept coins instead of coin * chore: reverted unintentional comment changes * chore: reverted unintentional comment changes * chore: adding test for conversion fn * chore: fix e2e linter * chore: adding docs * chore: addressing PR feedback * chore: remove duplicate import * chore: addressing PR feedback * chore: moved extration logic into internal package * chore: commented out failing test * chore: adding link to issue * chore: remove duplicate import * chore: fix linting errors * FungibleTokenPacketData interface methods + tests * linter * wip: token validation * update trace identifier validation in Token + tests * rm Printf * update with pr review * pr review * linter * rm unused function: linter * wip pr feedback * fix test * pr review * lintttttt * wip: backwards compatibility for transfer rpc * implement changes for ics20-v2 packet data for onRecvPacket, onAcknowledgePacket and onTimeoutPacket * fix callbacks tests * lint --------- Co-authored-by: chatton <[email protected]> Co-authored-by: Charly <[email protected]> * add v3 packet proto * fix protos * fixes * test fixes * add forwardPath keys and memo check in sendTransfer * wip onRecvPacket logic * minor fixes * changes to transfer tx CLI to support multiple denoms * import renaming * onRecv logic completed * add revertInFlights function * add onAck && onTimeout logic * fix interchain accounts test * basic unit test for path forwarding * fix test unsuccessful refund from source * wip test fix * fix mbt test - need more investigation * revert test fix * add assertions * add support for async ack * wip test forwarding happy path * icsv20(path forwarding): use nil as default forwarding path when not set, use sequence in key to store forwarded packet (#6325) * use nil forwarding path as default when not set, use sequence in key to store forwarded packet * lint * add forwarding happy path tests * fix merge * Use type with V2 suffix for package data (#6330) * Adding additional comments and changing version variable names (#6345) * chore: correctly claim capability * lint * imp: change ics20 events to emit token set (#6348) * refactor: quick change to tokens event attribute * fix: various tests * lint * lint:fixeroni * imp: remove events variable in favour of direct event emission --------- Co-authored-by: DimitrisJim <[email protected]> * imp: check length tokens array against maximum allowed (#6349) * check length of tokens array against maximum allowed * chore: add note on arbitrary selection --------- Co-authored-by: Colin Axnér <[email protected]> * Modify UnmarshalPacketData interface to allow additional args (#6341) * api(port)!: Allow passing of context, port and channel identifier to unmarshal packet data interface as disussed. This allows us to grab the app version in transfer and unmarshal the packet based on that instead of a hacky unmarshal v2 then v1 and whatever happens. * lint: as we do * callbacks: fix signature of UnmarshalPacketData as per changes, make refactors to hopefully simplify signatures. * chore: lint and remove some todos. * review: address feedback. * Refactor packet data unmarshalling to use specific version (#6354) * chore: specifically unmarshal v1 or v2 without brute force * chore: fix TestPacketDataUnmarshalerInterface test in transfer module * Pass dest values OnRecv, refactor GetExpectedEvents * chore: fixing TestGetCallbackData test * chore: fixed remaining tests in callbacks module * test: simplify callbacks test, revert back to previous behaviour * chore: fix test case name * chore: addressing PR feedback * chore: added docstring for unmarshalPacketDataBytesToICS20V2 --------- Co-authored-by: DimitrisJim <[email protected]> Co-authored-by: Colin Axnér <[email protected]> * chore: fixing tests * imp: self review comments for ics20-v2 (#6360) * refactor: address various self review comments * revert: unnecessary change * lint * imp: self review on ics20-v2 part 2 (#6364) * refactor: apply review suggestions * imp: additional updates * refactor: make ValidateIBCDenom private * Update modules/apps/transfer/types/msgs.go Co-authored-by: Cian Hatton <[email protected]> * apply review suggestions --------- Co-authored-by: Cian Hatton <[email protected]> * chore: move functions from internal/denom back to trace.go (#6368) * chore: move functions from internal/denom to trace.go * lint * lint: the comeback * imp: ics20 v2 self review part 3 (#6373) * imp: self review items * apply jim's suggestion * Update modules/apps/transfer/keeper/msg_server_test.go * Update modules/apps/transfer/ibc_module.go * Update modules/apps/transfer/ibc_module.go * chore: remove duplicate test case * chore: address minor nits (#6374) * fix lint warning, add extra godocs, and some other small fixes and cleanup * fix finalReceiver address bug * wip - ack test scenario5 * add FungibleTokenPacketDataV2 test for ValidateBasic (#6398) * fix linter complaints * add test - currently faling on middle hop revert * add test comments * fixes * retrieve channel capability only if there is a previous packet in store * add missing parameter * fix: e2e build failures. * Use Transfer instead of sendTransfer when forwarding. (#6564) Pass nil as the next forwarding path if at final hop. Use consistent timeout for all hops. * lint: fix linter issues. * tests(transfer): move forwarding tests to separate file. (#6568) * chore: rename ForwardingInfo to Forwarding * Revert "chore: rename ForwardingInfo to Forwarding" This reverts commit e483b9a. * nit(transfer): Mark hops as non nullable. (#6566) * nit(transfer): Mark hops as non nullable. * lint: fix additional linting issues * feat(transfer): add forwarding info validation to token packet (#6571) * feat(transfer): add forwarding info validation to token packet * Added NewForwardingInfo constructor * Removed redundant check * Clean up tests per cr comments * Back to Validate and use NewDenom * feat(transfer): add validation for forwarding info in msg transfer validate basic (#6583) * Fix and simplify reverts of forwarding state (#6574) * refactor: initial simplification attempt * refactor: further organize * fix: all tests fixed * docs: improved godocs * fix: logic and testing error * style: self suggestion * docs: suggestion * docs: spellcheck * style: suggestions * style: renamed to revertForwardedPacket * style: suggestion * docs: remove docs * docs: godoc suggestion * style: suggestion * docs: colin suggestions * chore: rename ForwardingInfo to Forwarding (#6576) * chore: rename ForwardingInfo to Forwarding * chore: rename forwarding_path field to forwarding * chore: make lint-fix * chore: rename forwarding info -> forwarding * chore: renaming fixes before merge * Refactor packet forward functions (#6575) * feat(transfer): validate forwarding memo in transfer authorization (#6591) * Add func convert token to coin ibc (#6584) * add func convert token to coin ibc * fix command: change func to ToCoin and add godoc * add unit test * Revert using ToCoin on Recv where trace manipulation occurs. Use ToCoin while forwarding. Update tests as per Carlos's review. * rename variable --------- Co-authored-by: Carlos Rodriguez <[email protected]> Co-authored-by: DimitrisJim <[email protected]> * transfer: Disallow a forwarding object specified with zero hops and a memo (#6599) * transfer: Disallow a forwarding object specified with zero hops and a memo. * Apply suggestions from code review Co-authored-by: Carlos Rodriguez <[email protected]> --------- Co-authored-by: Carlos Rodriguez <[email protected]> Co-authored-by: Gjermund Garaba <[email protected]> * feat(transfer): move async decision and handling to the ibc module onrecv callback (#6592) * refactor: initial simplification attempt * refactor: further organize * fix: all tests fixed * chore: refactor packet forward functions * chore: use receiver address directly in msg transfer * feat(transfer): move async to ibc_module for onrecv * chore: fix linter * fix: logic and testing error * style: self suggestion * docs: suggestion * docs: spellcheck * style: suggestions * style: renamed to revertForwardedPacket * style: suggestion * docs: remove docs * Added tests for transfer OnRecv * Use new names and methods from feature branch * Apply suggestions from code review Co-authored-by: colin axnér <[email protected]> * Clean up test from cr feedback * move vars to beginning of function * lint * Update modules/apps/transfer/ibc_module.go Co-authored-by: Carlos Rodriguez <[email protected]> * lint --------- Co-authored-by: srdtrk <[email protected]> Co-authored-by: chatton <[email protected]> Co-authored-by: colin axnér <[email protected]> Co-authored-by: Carlos Rodriguez <[email protected]> Co-authored-by: DimitrisJim <[email protected]> * chore: use NewForwarding instead of direct init (#6605) * Reduce max forwarding to 16 (#6610) * feat(transfer): use single byte ack for successful forward (#6604) Co-authored-by: Nikolas De Giorgis <[email protected]> * chore(transfer/cli): add forwarding flag to tx cli (#6609) Co-authored-by: Carlos Rodriguez <[email protected]> * chore(transfer): make Forwarding non-null (#6618) * chore(transfer): make Forwarding non-null * chore(transfer): always validate forwarding. * chore: restructure functions with logical ordering (#6638) * test: Add tests for OnTimeoutPacket when middle chain times out packet (#6596) * Create ontimeoutpacket test for forwarding --------- Co-authored-by: Carlos Rodriguez <[email protected]> * feat(transfer): add ShouldBeForwarded convenience method to msg transfer (#6595) * feat(transfer): add should-be-forwarded convenience method to msg transfer * Clean up packet and transfer msg validation for forwarding logic --------- Co-authored-by: Carlos Rodriguez <[email protected]> * wip: relayer configuration experimentation * disallow timeout height usage when forwarding packets (#6641) * disallow non-zero timeout height when forwarding tokens * typo * move constant * use time to create default timeout timestamp delta * Apply suggestions from code review * refactor: add GetTimeoutTimestamp helper fn to the testing pkg * lint --------- Co-authored-by: colin axnér <[email protected]> * nit: make set forwarded packet unexported (#6637) Co-authored-by: Gjermund Garaba <[email protected]> Co-authored-by: colin axnér <[email protected]> * feat(transfer): use registered error code for error acks in token forwarding (#6648) * Add typed errors to packet forwarding * Use forward errors in tests * Make ack construction consistent * chore(transfer): emit forwarding information in events. (#6647) * chore(transfer): emit forwarding information in events. * nit: rename as per Carlos's suggestion * Fix e2e test * chore: test passing using relayer packet filter * Refactor forwarding messages for Transfer and Packet (#6655) * feat(transfer): add unwind, refactor proto structure. gen-all * tests(transfer/types): fix test failures in types tests. * tests(transfer/keeper): fix test failures in keeper tests. * cli(transfer): fix cli usage. pending flag for unwind. * tests(callbacks): fix failing tests in callbacks. * tests(transfer/internal): fix failures in internal package. * tests(transfer): fix test failures in top level tranfer package. * tests(ica/host/keeper): fix repr of msg transfer in ica host msg execution. * lint(all): lint this bad boy * chore(transfer/types): amend validation for MsgTransfer's ShouldBeForwarded, add tests for ForwardedPacketData, minor nits. * nit(self): only pass relevant fields to create packet data; minor comment improvement. * Apply suggestions from code review Co-authored-by: Carlos Rodriguez <[email protected]> * chore(merge): fix merge issues. * chore(proto): mention optional nature of fields. * memo: do not drop it * validation: drop requirement on memo being empty on msg transfer. --------- Co-authored-by: Carlos Rodriguez <[email protected]> * feat: allow authz granters to specify forwarding info for token transfer (#6661) * First attempt to modify proto and validation. * Fmt * Make it compile * Proto * Add basic validation * Added tests * Fix nil check and remove redundant code. * Move "forwarding" to non-pointer * Added one test case. * Moved error and changed method name. * PR Feedback. * Add nullable=false * Added test and fixed test names * Run make proto-all * Change pointer to non-pointer * Change Yet Another Pointer * More pointers * Comments and naming * Reintroduce brace removed while merging * Remove accidentally reintroduced tests * feat: delete forwarded packet when it is not needed anymore (#6621) * Create ontimeoutpacket test for forwarding * Propagate ack on A * Refactoring * Minor changes * Added comments * Fix type name. * Gofumpt * Update modules/apps/transfer/keeper/relay_forwarding_test.go Co-authored-by: Carlos Rodriguez <[email protected]> * Update modules/apps/transfer/keeper/relay_forwarding_test.go Co-authored-by: Carlos Rodriguez <[email protected]> * Update modules/apps/transfer/keeper/relay_forwarding_test.go Co-authored-by: Carlos Rodriguez <[email protected]> * Update modules/apps/transfer/keeper/relay_forwarding_test.go Co-authored-by: Carlos Rodriguez <[email protected]> * Add godoc to test. * Changed trace construction * Update modules/apps/transfer/keeper/relay_forwarding_test.go Co-authored-by: Carlos Rodriguez <[email protected]> * remove error msg parameter from helper function * Add test for forwarded packet * Delete packet when not needed anymore. * Move deletion of packet in a single place. * Reintroduce newline * Do not ignore error. * PR feedback. * Construct packet for B ack check. * PR feedback * Pass packet to acknowledgeforwardedpacket * revert unwanted change * Another unwanted change * Better signature and fix source/dest * Added one more test. --------- Co-authored-by: Carlos Rodriguez <[email protected]> Co-authored-by: Gjermund Garaba <[email protected]> * test(transfer): forwarding acknowledgment errors in middle hop (#6659) * test(transfer): forwarding where middle chaind is source for receive and send * Fix errors after merge * Finish up the test * Update some out-of-date comments * test(transfer): multi-hop ack failure with middle chain NOT being source * Fix tests after height change * Fix tests after height change * Fix test after #6586 * Rename tests to not use scenario numbers * Rename test * address self-review comments * use boolean in NewForwarding parameter * some more review comments --------- Co-authored-by: Carlos Rodriguez <[email protected]> * test(transfer): last chain in forwarding packet is ICS20 v1 (#6622) * test(transfer): last chain in forwarding being ICS20 v1 * Finish TestForwarding_WithLastChainBeingICS20v1_Succeeds * Update CreateNewPath signature Co-authored-by: Nikolas De Giorgis <[email protected]> * Fix PR review comments --------- Co-authored-by: Nikolas De Giorgis <[email protected]> * refactor: rename SetupPath to SetupPaths (#6674) * chore: configure the hermes config to watch a specific port and channel ID * chore: add flag for unwind in transfer cli (#6680) * add flag for unwind in transfer cli * update long description of cli --------- Co-authored-by: Carlos Rodriguez <[email protected]> * chore: small refactor for SetupPath * chore: adding link to interchaintest issue and doing some cleanup * chore: adding interchaintest issue link in comment * chore: merge main * chore: fix linter * chore: sync main * feat: impl check reject transfer if len(hops) > 0 and ics20-1 (#6675) * impl check reject transfer if len(hops) > 0 and ics20-1 * add test case hops is not empty with ics20-2 * address review comments * reorder variable declaration --------- Co-authored-by: Carlos Rodriguez <[email protected]> Co-authored-by: Gjermund Garaba <[email protected]> * chore: fix go mod * feat(transfer): add unwinding ability (#6656) * Create ontimeoutpacket test for forwarding * Propagate ack on A * Refactoring * Minor changes * Added comments * Fix type name. * Gofumpt * Update modules/apps/transfer/keeper/relay_forwarding_test.go Co-authored-by: Carlos Rodriguez <[email protected]> * Update modules/apps/transfer/keeper/relay_forwarding_test.go Co-authored-by: Carlos Rodriguez <[email protected]> * Update modules/apps/transfer/keeper/relay_forwarding_test.go Co-authored-by: Carlos Rodriguez <[email protected]> * Update modules/apps/transfer/keeper/relay_forwarding_test.go Co-authored-by: Carlos Rodriguez <[email protected]> * Add godoc to test. * Changed trace construction * Update modules/apps/transfer/keeper/relay_forwarding_test.go Co-authored-by: Carlos Rodriguez <[email protected]> * remove error msg parameter from helper function * Add test for forwarded packet * Construct packet for B ack check. * PR feedback * feat(transfer): add unwind, refactor proto structure. gen-all * tests(transfer/types): fix test failures in types tests. * tests(transfer/keeper): fix test failures in keeper tests. * cli(transfer): fix cli usage. pending flag for unwind. * tests(callbacks): fix failing tests in callbacks. * tests(transfer/internal): fix failures in internal package. * tests(transfer): fix test failures in top level tranfer package. * tests(ica/host/keeper): fix repr of msg transfer in ica host msg execution. * lint(all): lint this bad boy * chore(transfer/types): amend validation for MsgTransfer's ShouldBeForwarded, add tests for ForwardedPacketData, minor nits. * nit(self): only pass relevant fields to create packet data; minor comment improvement. * Apply suggestions from code review Co-authored-by: Carlos Rodriguez <[email protected]> * chore(merge): fix merge issues. * chore(proto): mention optional nature of fields. * memo: do not drop it * validation: drop requirement on memo being empty on msg transfer. * feat(transfer): add unwinding ability, wip. * Added unwind to allocation forwarding. * Add tests and move some validation * Missing import * Fixed validation and added test * PR Feedback * Return nil when returning an error. * Cleaner comment * Add test case for multiple hos --------- Co-authored-by: bznein <[email protected]> Co-authored-by: Nikolas De Giorgis <[email protected]> Co-authored-by: Carlos Rodriguez <[email protected]> * chore: add support to run full test suite in run-e2e.sh * chore: only store last channel for each test * fix typo * remove unnecessary wrapping of function * Revert "remove unnecessary wrapping of function" This reverts commit c2a6bc6. * fix usage of function * (chore) replace reflect.DeepEqual with slices.Equal (#6697) * Replace reflect.DeepEqual with slices.Equal * Nit formatting * chore: comment hop slicing for clarity (#6702) Co-authored-by: Carlos Rodriguez <[email protected]> * chore: cleanup forwarding tests (#6691) * chore: cleanup forwarding tests * lint * fix --------- Co-authored-by: Nikolas De Giorgis <[email protected]> Co-authored-by: Gjermund Garaba <[email protected]> * chore: pull out hop validation and consolidate for transfer+packet (#6695) * chore: pull out hop validation and consolidate transfer+packet * Update modules/apps/transfer/types/forwarding_test.go Co-authored-by: Carlos Rodriguez <[email protected]> * cr fixes --------- Co-authored-by: Carlos Rodriguez <[email protected]> * Remove unwind field in authz (#6701) * chore: remove unwind field in authz * chore: remove duplicate test * Update modules/apps/transfer/types/transfer_authorization.go Co-authored-by: Carlos Rodriguez <[email protected]> --------- Co-authored-by: Carlos Rodriguez <[email protected]> * chore: add packet data validation back (#6704) * (chore) Refactor code around forwarding validation (#6706) * Refactor validation * Fixed verification logic, added two tests * Fix check for unwind * removed unneeded indirection * Update modules/apps/transfer/types/msgs.go Co-authored-by: DimitrisJim <[email protected]> * Add docstring. --------- Co-authored-by: Gjermund Garaba <[email protected]> Co-authored-by: DimitrisJim <[email protected]> * use setupForwardingPaths in test * feat(transfer): allow non-cosmos-sdk AccAddress in final receiver for forwarded packets (#6709) * allow non-cosmos-sdk AccAddress for forwarded packets * cr fixes * chore: pass only hops to sendTransfer + events rename (#6703) Co-authored-by: Carlos Rodriguez <[email protected]> * test: forwarding test that verifies forwarded memo (#6707) * test: forwarding with memo * fix test * cr fixes * chore: update godoc for relay forwarding tests * chore: use module account instead of custom forward address (#6688) * chore: use module account instead of custom forward address * pull blocked addr checker into keeper method * lint * clean up IsBlockedAddr * chore: replace continue with if/else (#6700) Co-authored-by: Nikolas De Giorgis <[email protected]> * add changelog * add test for invalid receiver address * Update CHANGELOG.md * Update CHANGELOG.md * make getForwardedPacket private * remove auxiliary burn coins function * nit: rename func method recv args in types/forwarding.go * chore: rename ShouldBeForwarded to HasFowarding * e2e: remove template test for three chain setup. * nit: no generics silly * nit: add clarifying comment to validate basic call on msg. * nit: remove unused key. * nit: clean up cli help text. * nit: don't export is blocked address helper. * nit: docustring for e2e test and helper. * nit: improve documentation for transfer's OnRecv callback. Co-authored-by: Damian Nolan <[email protected]> * chore: move setup code into individual tests * chore: use custom setup functions in tests that need them * chore: remove unused function * Apply suggestions from code review Co-authored-by: Aditya <[email protected]> * chore: remove unused function * perf: allocate slice to length of packet data tokens * chore(transfer/authz): wrapf unauthorized forwarding hops * lint * Update modules/apps/transfer/types/forwarding.go Co-authored-by: DimitrisJim <[email protected]> * Preallocate slice but keep len==0 (#6725) * imp: validate allowed forwarding hops * test: unwind fails in Transfer rpc * wip: tests failing after merge conflict * chore: test passing with new config * wip: working on ensuring existing tests pass * wip: working on ensuring existing tests pass * chore: revert changes to ibc genesis test * wip: transfer tests passing in parallel * chore: add comments and max function * chore: fix e2e linting * chore: move param tests into their own files * chore: run transfer test in CI in parallel * chore: tweak run-e2e.sh to not check for fzf * chore: reference correct env var * chore: separate out transfer utility logic from test suite * chore: unexporting transfer tester type * chore: temporarily disable regular E2Es * chore: explicitly run a single test suite * chore: re-enable regular E2Es and remove redundant start relayer fn * chore: adding flag to workflow * chore: correctly store all channels created * chore: renaming based on PR feedback --------- Co-authored-by: Charly <[email protected]> Co-authored-by: Carlos Rodriguez <[email protected]> Co-authored-by: Charly <[email protected]> Co-authored-by: Stefano Angieri <[email protected]> Co-authored-by: sangier <[email protected]> Co-authored-by: Colin Axnér <[email protected]> Co-authored-by: DimitrisJim <[email protected]> Co-authored-by: Hastur <[email protected]> Co-authored-by: Damian Nolan <[email protected]> Co-authored-by: Gjermund Garaba <[email protected]> Co-authored-by: srdtrk <[email protected]> Co-authored-by: Maintain <[email protected]> Co-authored-by: srdtrk <[email protected]> Co-authored-by: Nikolas De Giorgis <[email protected]> Co-authored-by: Gjermund Garaba <[email protected]> Co-authored-by: Dzung Do | Decentrio <[email protected]> Co-authored-by: bznein <[email protected]> Co-authored-by: Aditya <[email protected]>

chatton and others added 30 commits April 8, 2024 13:00

Adding proto files for ics20-v2 (#6110)

c07bca9

* chore: adding proto files for ics20-v2 * chore: add newline

update amount -> string (#6120)

e66bd89

Update MsgTransfer to accept sdk.Coins instead of sdk.Coin (#6113)

034f472

fix: allow base denom with trailing slash (#6148)

4cc6a85

imp: add CurrentVersion, EscrowVersion (#6160)

71f830c

* add CurrentVersion, EscrowVersion, update tests * update hardcoded transfer channel version from interchaintest

chore: add function for converting packet data from v1 to v3 (#6116)

28ff9b6

--------- Co-authored-by: Charly <[email protected]>

chore: implement required FungibleTokenPacketData v3 interface meth…

4e55137

…ods (#6126)

imp: update transfer authz implementation to account for multi denom …

9bbfa1a

…transfers (#6252) * add transfer authz code + tests * linter * address review comments --------- Co-authored-by: Carlos Rodriguez <[email protected]>

add v3 packet proto

e04047e

fix protos

c73d5f6

fixes

03cc2e5

test fixes

65f4476

add forwardPath keys and memo check in sendTransfer

db85a29

wip onRecvPacket logic

999ed80

minor fixes

be72180

changes to transfer tx CLI to support multiple denoms

4c333c5

import renaming

0478cb9

onRecv logic completed

37f5b5e

add revertInFlights function

3fd3345

add onAck && onTimeout logic

24e2407

fix interchain accounts test

7858528

basic unit test for path forwarding

55d7bf9

fix test unsuccessful refund from source

1a95262

wip test fix

44e15cb

fix mbt test - need more investigation

334d5a9

revert test fix

7466b19

add assertions

5c46d79

chatton changed the title ~~[WIP] Allow for relayers to create clients/connections and channels in parallel~~ [WIP] Enable Parallel Transfer Test Suite Jul 3, 2024

chore: adding flag to workflow

e7d0520

chatton commented Jul 3, 2024

View reviewed changes

chatton changed the title ~~[WIP] Enable Parallel Transfer Test Suite~~ Enable Parallel Transfer Test Suite Jul 3, 2024

chatton mentioned this pull request Jul 3, 2024

Update CI workflows to run a single test suite on each host instead of one test per host #6677

Open

8 tasks

chatton added 2 commits July 3, 2024 14:57

Merge branch 'main' into cian/enable-parallel-tests

48482c1

chore: correctly store all channels created

d7fa8dd

chatton commented Jul 3, 2024

View reviewed changes

chatton added the priority PRs that need prompt reviews label Jul 3, 2024

DimitrisJim reviewed Jul 3, 2024

View reviewed changes

chatton added 2 commits July 3, 2024 16:48

Merge branch 'main' into cian/enable-parallel-tests

21c2711

Merge branch 'main' into cian/enable-parallel-tests

61c1a81

chatton mentioned this pull request Jul 4, 2024

Enabling additional tests to run in parallel #6766

Merged

10 tasks

DimitrisJim approved these changes Jul 4, 2024

View reviewed changes

chatton added 6 commits July 4, 2024 12:39

Merge branch 'main' into cian/enable-parallel-tests

c9949d9

Merge branch 'main' into cian/enable-parallel-tests

2ec8c1a

Merge branch 'main' into cian/enable-parallel-tests

6f195b1

Merge branch 'main' into cian/enable-parallel-tests

d9a7093

chore: merge main

c7499fd

Merge branch 'main' into cian/enable-parallel-tests

e0f9257

bznein approved these changes Jul 8, 2024

View reviewed changes

Merge branch 'main' into cian/enable-parallel-tests

cf54298

gjermundgaraba approved these changes Jul 9, 2024

View reviewed changes

chore: renaming based on PR feedback

1a385b6

chatton added this pull request to the merge queue Jul 9, 2024

Merged via the queue into main with commit d8cae6e Jul 9, 2024
134 of 136 checks passed

chatton deleted the cian/enable-parallel-tests branch July 9, 2024 08:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable Parallel Transfer Test Suite #6721

Enable Parallel Transfer Test Suite #6721

chatton commented Jun 27, 2024 •

edited

Loading

chatton Jul 3, 2024

chatton Jul 3, 2024

chatton Jul 3, 2024

chatton Jul 3, 2024

chatton Jul 3, 2024

chatton Jul 3, 2024

chatton Jul 3, 2024

chatton Jul 3, 2024

chatton Jul 3, 2024

gjermundgaraba Jul 9, 2024

chatton Jul 9, 2024

chatton Jul 3, 2024

chatton Jul 3, 2024

DimitrisJim Jul 3, 2024

chatton Jul 3, 2024

DimitrisJim Jul 3, 2024

chatton Jul 3, 2024

DimitrisJim Jul 3, 2024

DimitrisJim Jul 3, 2024

DimitrisJim left a comment

bznein left a comment

bznein Jul 8, 2024

chatton Jul 8, 2024

bznein Jul 8, 2024

bznein Jul 8, 2024

gjermundgaraba left a comment

sonarcloud bot commented Jul 9, 2024

		chainA, _ := s.GetChains()

		channelVersion := transfertypes.V2


		s.testPaths[s.T().Name()] = append(s.testPaths[s.T().Name()], pathName)
		err = relayer.ApplyPacketFilter(ctx, s.T(), r, c.Config().ChainID, s.channels[testName][c])

Enable Parallel Transfer Test Suite #6721

Enable Parallel Transfer Test Suite #6721

Conversation

chatton commented Jun 27, 2024 • edited Loading

Description

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

DimitrisJim left a comment

Choose a reason for hiding this comment

bznein left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gjermundgaraba left a comment

Choose a reason for hiding this comment

sonarcloud bot commented Jul 9, 2024

Quality Gate passed for 'ibc-go'

chatton commented Jun 27, 2024 •

edited

Loading