Skip to content

Commit

Permalink
backport: e2e tests (#1193) (#1338)
Browse files Browse the repository at this point in the history
* Setup e2e tests on a single chain; add balances query test (#1193)

* create e2e image and a makefile step to build

* progress

* e2e tests in ci

* use root distroless image and correct volume path

* remove chain b references

* implement query balances

* implement TestQueryBalances

* trigger worflow

* trigger

* test-e2e Makefile step

* fmt and sleep if service unavailable

* README

* restore branches

* add changelog entry

* exclude e2e from regular tests

* -E flag for grep exclusion

* grep

* go mod tidy --compat=1.17

* manually tidy go.mod

* second e2e chain with expanded test (#1206)

* e2e with IBC tx and test (#1216)

* second chain with tests

* add hermes, comment out balance query for now

* eventually

* Update Makefile

Co-authored-by: Roman <[email protected]>

* remove gas fees and unused functions

* readded check

* Nicco changes to own hermes image

* Remove unused hermes.Dockerfile

* Use a single Dockerfile for both debug and official image

* readd build hermes in makefile

* remove hermes

* Set correct golang image and use correct debug image tag

Co-authored-by: Adam Tucker <[email protected]>
Co-authored-by: Roman <[email protected]>
Co-authored-by: Niccolo Raspa <[email protected]>

* missing e2e ci job

* cleanup Makefile and ci workflows (#1203)

* cleanup makefile and ci workflows

* update changelog

* fix sim test

* fix makefile and rename docker repo

* fix make test-cover (#1219)

* refactor: begin modularizing e2e tests in preparation for upgrade (#1293)

* begin modularizing e2e test in preparation for upgrade

* rename common package to util

* move chain related constants from util to chain package

* fix genesis.go

* exctract initNodes into the genesis package

* remove genesis package, move all logic to chain

* continue cleaning up chain package and refactoring e2e

* store chains in a slice

* reuse common cdc from util package

* lexicographical reorder of functions in config.go of chain package

* clean up names

* refactor: implement Dockerized chain initialization in e2e tests (#1330)

Closes: #XXX

## What is the purpose of the change

This is a follow-up PR to #1293 and builds upon its work. It is part of the e2e test chain upgrade epic #1235 

This PR introduces the ability to run chain initialization in a Docker container by running the following:
```
docker run -v < path >:/tmp/osmo-test osmosis-e2e-chain-init:debug --data-dir=/tmp/osmo-test
```
All chain data is placed at the given `< path >` that is mounted as a volume on the container. In addition, this PR introduces documentation about the current state of the e2e tests. 

## Brief change log

- [pull chain temp folder creation our of chain to e2e package](c175289)
- [create image and makefile steps to initialize chain state](de89119)
- [allow for running the upgrade initialization in Docker by providing a data dir](dabb683)
- [improve abstractions for chain initialization and add README](cf0d8a3)
- improve README

## Testing and Verifying

- ran e2e tests locally a few times
- `make build-e2e-chain-init`
- `make docker-build-e2e-chain-init`
- `docker run -v /home/roman/cosmos/osmosis/tmp:/tmp/osmo-test osmosis-e2e-chain-init:debug --data-dir=/tmp/osmo-test`

All steps worked as desired

## Documentation and Release Note

  - Does this pull request introduce a new feature or user-facing behavior changes? no
  - Is a relevant changelog entry added to the `Unreleased` section in `CHANGELOG.md`? no
  - How is the feature or change documented? improved `tests/e2e/README.md`

## Next Steps

The next step is to switch our chain initialization logic in `func (s *IntegrationTestSuite) configureChain(chainId string)` to use Dockertest and the newly introduced container. Then, mount the genesis and configs on the Osmosis containers, against which the e2e tests are executed

* Setup e2e tests on a single chain; add balances query test (#1193)

* create e2e image and a makefile step to build

* progress

* e2e tests in ci

* use root distroless image and correct volume path

* remove chain b references

* implement query balances

* implement TestQueryBalances

* trigger worflow

* trigger

* test-e2e Makefile step

* fmt and sleep if service unavailable

* README

* restore branches

* add changelog entry

* exclude e2e from regular tests

* -E flag for grep exclusion

* grep

* go mod tidy --compat=1.17

* manually tidy go.mod

Co-authored-by: Adam Tucker <[email protected]>
Co-authored-by: Adam Tucker <[email protected]>
Co-authored-by: Niccolo Raspa <[email protected]>
  • Loading branch information
4 people authored Apr 29, 2022
1 parent 445cbc1 commit f82a6fc
Show file tree
Hide file tree
Showing 22 changed files with 1,791 additions and 9 deletions.
3 changes: 2 additions & 1 deletion .github/workflows/sim.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,4 +17,5 @@ jobs:
- name: Display go version
run: go version
- name: Run simulation
run: go test ./simapp
run: |
make test-sim
26 changes: 25 additions & 1 deletion .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,30 @@ jobs:
- name: Display go version
run: go version
- name: Run all tests
run: go test -mod=readonly -timeout 30m -coverprofile=coverage.txt -tags='norace' -covermode=atomic `go list ./... | grep -v simapp`
run: |
make test-cover
- name: Codecov
uses: codecov/[email protected]
test-e2e:
runs-on: ubuntu-latest
timeout-minutes: 25
steps:
- uses: actions/[email protected]
with:
go-version: 1.17
- uses: actions/checkout@v2
- uses: technote-space/[email protected]
with:
PATTERNS: |
**/**.go
go.mod
go.sum
- name: Build Docker Image
run: |
make docker-build-debug
make docker-build-e2e-chain-init
if: env.GIT_DIFF
- name: Test E2E
run: |
make test-e2e
if: env.GIT_DIFF
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
### Minor improvements & Bug Fixes

* [#1177](https://github.com/osmosis-labs/osmosis/pull/1177) upgrade to go 1.18
* [#1193](https://github.com/osmosis-labs/osmosis/pull/1193) Setup e2e tests on a single chain; add balances query test
* [#1061](https://github.com/osmosis-labs/osmosis/pull/1061) upgrade iavl to v0.17.3-osmo-v5 with concurrent map write fix
* [#1071](https://github.com/osmosis-labs/osmosis/pull/1071) improve Dockerfile

Expand Down
7 changes: 5 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# syntax=docker/dockerfile:1

ARG BASE_IMG_TAG=nonroot

## Build Image
FROM golang:1.18-bullseye as build

Expand All @@ -13,7 +15,7 @@ RUN sha256sum /lib/libwasmvm_muslc.a | grep d0152067a5609bfdfb3f0d5d6c0f2760f79d
RUN BUILD_TAGS=muslc make build

## Deploy image
FROM gcr.io/distroless/base-debian11:nonroot
FROM gcr.io/distroless/base-debian11:${BASE_IMG_TAG}

COPY --from=build /osmosis/build/osmosisd /bin/osmosisd

Expand All @@ -22,6 +24,7 @@ WORKDIR $HOME

EXPOSE 26656
EXPOSE 26657
EXPOSE 1317
EXPOSE 1317

ENTRYPOINT ["osmosisd"]
CMD [ "start" ]
28 changes: 24 additions & 4 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,10 @@ build-contract-tests-hooks:
mkdir -p $(BUILDDIR)
go build -mod=readonly $(BUILD_FLAGS) -o $(BUILDDIR)/ ./cmd/contract_tests

build-e2e-chain-init:
mkdir -p $(BUILDDIR)
go build -mod=readonly $(BUILD_FLAGS) -o $(BUILDDIR)/ ./tests/e2e/chain_init

go-mod-cache: go.sum
@echo "--> Download go modules to local cache"
@go mod download
Expand Down Expand Up @@ -210,24 +214,40 @@ sync-docs:
### Tests & Simulation ###
###############################################################################

PACKAGES_UNIT=$(shell go list ./... | grep -E -v 'simapp|e2e')
PACKAGES_E2E=$(shell go list ./... | grep '/e2e')
PACKAGES_SIM=$(shell go list ./... | grep '/simapp')
TEST_PACKAGES=./...

include sims.mk

test: test-unit test-build

test-all: check test-race test-cover

test-unit:
@VERSION=$(VERSION) go test -mod=readonly -tags='ledger test_ledger_mock norace' ./...
@VERSION=$(VERSION) go test -mod=readonly -tags='ledger test_ledger_mock norace' $(PACKAGES_UNIT)

test-race:
@VERSION=$(VERSION) go test -mod=readonly -race -tags='ledger test_ledger_mock' ./...
@VERSION=$(VERSION) go test -mod=readonly -race -tags='ledger test_ledger_mock' $(PACKAGES_UNIT)

test-cover:
@go test -mod=readonly -timeout 30m -coverprofile=coverage.txt -covermode=atomic -tags='ledger test_ledger_mock' ./...
@VERSION=$(VERSION) go test -mod=readonly -timeout 30m -coverprofile=coverage.txt -tags='norace' -covermode=atomic $(PACKAGES_UNIT)

test-sim:
@VERSION=$(VERSION) go test -mod=readonly $(PACKAGES_SIM)

test-e2e:
@VERSION=$(VERSION) go test -mod=readonly -timeout=25m -v $(PACKAGES_E2E)

benchmark:
@go test -mod=readonly -bench=. ./...
@go test -mod=readonly -bench=. $(PACKAGES_UNIT)

docker-build-debug:
@docker build -t osmosis:debug --build-arg BASE_IMG_TAG=debug -f Dockerfile .

docker-build-e2e-chain-init:
@docker build -t osmosis-e2e-chain-init:debug -f tests/e2e/chain_init/chain-init.Dockerfile .

###############################################################################
### Linting ###
Expand Down
21 changes: 20 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,15 @@ require (
github.com/99designs/keyring v1.1.6 // indirect
github.com/Antonboom/errname v0.1.5 // indirect
github.com/Antonboom/nilnil v0.1.0 // indirect
github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect
github.com/BurntSushi/toml v1.1.0 // indirect
github.com/ChainSafe/go-schnorrkel v0.0.0-20200405005733-88cbf1b4c40d // indirect
github.com/CosmWasm/wasmvm v1.0.0-beta5 // indirect
github.com/DataDog/zstd v1.4.5 // indirect
github.com/Djarvur/go-err113 v0.0.0-20210108212216-aea10b59be24 // indirect
github.com/Masterminds/semver v1.5.0 // indirect
github.com/Microsoft/go-winio v0.5.1 // indirect
github.com/Nvveen/Gotty v0.0.0-20120604004816-cd527374f1e5 // indirect
github.com/OpenPeeDeeP/depguard v1.1.0 // indirect
github.com/Workiva/go-datastructures v1.0.53 // indirect
github.com/alexkohler/prealloc v1.0.0 // indirect
Expand All @@ -57,12 +60,14 @@ require (
github.com/breml/errchkjson v0.2.3 // indirect
github.com/btcsuite/btcd v0.22.0-beta // indirect
github.com/butuzov/ireturn v0.1.1 // indirect
github.com/cenkalti/backoff/v4 v4.1.2 // indirect
github.com/cespare/xxhash v1.1.0 // indirect
github.com/cespare/xxhash/v2 v2.1.2 // indirect
github.com/charithe/durationcheck v0.0.9 // indirect
github.com/chavacava/garif v0.0.0-20210405164556-e8a0a408d6af // indirect
github.com/coinbase/rosetta-sdk-go v0.7.0 // indirect
github.com/confio/ics23/go v0.6.6 // indirect
github.com/containerd/continuity v0.2.1 // indirect
github.com/cosmos/btcutil v1.0.4 // indirect
github.com/cosmos/ledger-cosmos-go v0.11.1 // indirect
github.com/cosmos/ledger-go v0.9.2 // indirect
Expand All @@ -74,6 +79,10 @@ require (
github.com/dgraph-io/badger/v2 v2.2007.3 // indirect
github.com/dgraph-io/ristretto v0.0.3 // indirect
github.com/dgryski/go-farm v0.0.0-20200201041132-a6ae2369ad13 // indirect
github.com/docker/cli v20.10.11+incompatible // indirect
github.com/docker/docker v20.10.7+incompatible // indirect
github.com/docker/go-connections v0.4.0 // indirect
github.com/docker/go-units v0.4.0 // indirect
github.com/dustin/go-humanize v1.0.0 // indirect
github.com/dvsekhvalnov/jose2go v0.0.0-20200901110807-248326c1351b // indirect
github.com/esimonov/ifshort v1.0.4 // indirect
Expand Down Expand Up @@ -114,6 +123,7 @@ require (
github.com/google/go-cmp v0.5.7 // indirect
github.com/google/gofuzz v1.2.0 // indirect
github.com/google/orderedcode v0.0.1 // indirect
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 // indirect
github.com/gordonklaus/ineffassign v0.0.0-20210914165742-4cc7213b9bc8 // indirect
github.com/gorilla/handlers v1.5.1 // indirect
github.com/gorilla/websocket v1.5.0 // indirect
Expand All @@ -133,6 +143,7 @@ require (
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/hdevalence/ed25519consensus v0.0.0-20210204194344-59a8610d2b87 // indirect
github.com/hexops/gotextdiff v1.0.3 // indirect
github.com/imdario/mergo v0.3.12 // indirect
github.com/improbable-eng/grpc-web v0.14.1 // indirect
github.com/inconshreveable/mousetrap v1.0.0 // indirect
github.com/jgautheron/goconst v1.5.1 // indirect
Expand Down Expand Up @@ -165,13 +176,18 @@ require (
github.com/minio/highwayhash v1.0.2 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/mapstructure v1.4.3 // indirect
github.com/moby/term v0.0.0-20201216013528-df9cb8a40635 // indirect
github.com/moricho/tparallel v0.2.1 // indirect
github.com/mtibben/percent v0.2.1 // indirect
github.com/nakabonne/nestif v0.3.1 // indirect
github.com/nbutton23/zxcvbn-go v0.0.0-20210217022336-fa2cb2858354 // indirect
github.com/nishanths/exhaustive v0.7.11 // indirect
github.com/nishanths/predeclared v0.2.1 // indirect
github.com/olekukonko/tablewriter v0.0.5 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.0.2 // indirect
github.com/opencontainers/runc v1.0.3 // indirect
github.com/ory/dockertest/v3 v3.8.1
github.com/pelletier/go-toml v1.9.4 // indirect
github.com/petermattis/goid v0.0.0-20180202154549-b0b1615b78e5 // indirect
github.com/phayes/checkstyle v0.0.0-20170904204023-bfd46e6a821d // indirect
Expand Down Expand Up @@ -200,7 +216,7 @@ require (
github.com/sourcegraph/go-diff v0.6.1 // indirect
github.com/spf13/afero v1.6.0 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/spf13/viper v1.10.1 // indirect
github.com/spf13/viper v1.10.1
github.com/ssgreg/nlreturn/v2 v2.2.1 // indirect
github.com/stretchr/objx v0.3.0 // indirect
github.com/subosito/gotenv v1.2.0 // indirect
Expand All @@ -217,6 +233,9 @@ require (
github.com/ultraware/funlen v0.0.3 // indirect
github.com/ultraware/whitespace v0.0.5 // indirect
github.com/uudashr/gocognit v1.0.5 // indirect
github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f // indirect
github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
github.com/xeipuuv/gojsonschema v1.2.0 // indirect
github.com/yagipy/maintidx v1.0.0 // indirect
github.com/yeya24/promlinter v0.1.1-0.20210918184747-d757024714a1 // indirect
github.com/zondax/hid v0.9.0 // indirect
Expand Down
Loading

0 comments on commit f82a6fc

Please sign in to comment.