Skip to content

Commit

Permalink
test_: Code Migration from status-cli-tests
Browse files Browse the repository at this point in the history
author shashankshampi <[email protected]> 1729780155 +0530
committer shashankshampi <[email protected]> 1730274350 +0530

test: Code Migration from status-cli-tests
fix_: functional tests (#5979)

* fix_: generate on test-functional

* chore(test)_: fix functional test assertion

---------

Co-authored-by: Siddarth Kumar <[email protected]>

feat(accounts)_: cherry-pick Persist acceptance of Terms of Use & Privacy policy (#5766) (#5977)

* feat(accounts)_: Persist acceptance of Terms of Use & Privacy policy (#5766)

The original GH issue status-im/status-mobile#21113
came from a request from the Legal team. We must show to Status v1 users the new
terms (Terms of Use & Privacy Policy) right after they upgrade to Status v2
from the stores.

The solution we use is to create a flag in the accounts table, named
hasAcceptedTerms. The flag will be set to true on the first account ever
created in v2 and we provide a native call in mobile/status.go#AcceptTerms,
which allows the client to persist the user's choice in case they are upgrading
(from v1 -> v2, or from a v2 older than this PR).

This solution is not the best because we should store the setting in a separate
table, not in the accounts table.

Related Mobile PR status-im/status-mobile#21124

* fix(test)_: Compare addresses using uppercased strings

---------

Co-authored-by: Icaro Motta <[email protected]>

test_: restore account (#5960)

feat_: `LogOnPanic` linter (#5969)

* feat_: LogOnPanic linter

* fix_: add missing defer LogOnPanic

* chore_: make vendor

* fix_: tests, address pr comments

* fix_: address pr comments

fix(ci)_: remove workspace and tmp dir

This ensures we do not encounter weird errors like:
```
+ ln -s /home/jenkins/workspace/go_prs_linux_x86_64_main_PR-5907 /home/jenkins/workspace/go_prs_linux_x86_64_main_PR-5907@tmp/go/src/github.com/status-im/status-go
ln: failed to create symbolic link '/home/jenkins/workspace/go_prs_linux_x86_64_main_PR-5907@tmp/go/src/github.com/status-im/status-go': File exists
script returned exit code 1
```

Signed-off-by: Jakub Sokołowski <[email protected]>

chore_: enable windows and macos CI build (#5840)

- Added support for Windows and macOS in CI pipelines
- Added missing dependencies for Windows and x86-64-darwin
- Resolved macOS SDK version compatibility for darwin-x86_64

The `mkShell` override was necessary to ensure compatibility with the newer
macOS SDK (version 11.0) for x86_64. The default SDK (10.12) was causing build failures
because of the missing libs and frameworks. OverrideSDK creates a mapping from
the default SDK in all package categories to the requested SDK (11.0).

fix(contacts)_: fix trust status not being saved to cache when changed (#5965)

Fixes status-im/status-desktop#16392

cleanup

added logger and cleanup

review comments changes

fix_: functional tests (#5979)

* fix_: generate on test-functional

* chore(test)_: fix functional test assertion

---------

Co-authored-by: Siddarth Kumar <[email protected]>

feat(accounts)_: cherry-pick Persist acceptance of Terms of Use & Privacy policy (#5766) (#5977)

* feat(accounts)_: Persist acceptance of Terms of Use & Privacy policy (#5766)

The original GH issue status-im/status-mobile#21113
came from a request from the Legal team. We must show to Status v1 users the new
terms (Terms of Use & Privacy Policy) right after they upgrade to Status v2
from the stores.

The solution we use is to create a flag in the accounts table, named
hasAcceptedTerms. The flag will be set to true on the first account ever
created in v2 and we provide a native call in mobile/status.go#AcceptTerms,
which allows the client to persist the user's choice in case they are upgrading
(from v1 -> v2, or from a v2 older than this PR).

This solution is not the best because we should store the setting in a separate
table, not in the accounts table.

Related Mobile PR status-im/status-mobile#21124

* fix(test)_: Compare addresses using uppercased strings

---------

Co-authored-by: Icaro Motta <[email protected]>

test_: restore account (#5960)

feat_: `LogOnPanic` linter (#5969)

* feat_: LogOnPanic linter

* fix_: add missing defer LogOnPanic

* chore_: make vendor

* fix_: tests, address pr comments

* fix_: address pr comments

chore_: enable windows and macos CI build (#5840)

- Added support for Windows and macOS in CI pipelines
- Added missing dependencies for Windows and x86-64-darwin
- Resolved macOS SDK version compatibility for darwin-x86_64

The `mkShell` override was necessary to ensure compatibility with the newer
macOS SDK (version 11.0) for x86_64. The default SDK (10.12) was causing build failures
because of the missing libs and frameworks. OverrideSDK creates a mapping from
the default SDK in all package categories to the requested SDK (11.0).

fix(contacts)_: fix trust status not being saved to cache when changed (#5965)

Fixes status-im/status-desktop#16392

test_: remove port bind

chore(wallet)_: move route execution code to separate module

chore_: replace geth logger with zap logger (#5962)

closes: #6002

feat(telemetry)_: add metrics for message reliability (#5899)

* feat(telemetry)_: track message reliability

Add metrics for dial errors, missed messages,
missed relevant messages, and confirmed delivery.

* fix_: handle error from json marshal

chore_: use zap logger as request logger

iterates: status-im/status-desktop#16536

test_: unique project per run

test_: use docker compose v2, more concrete project name

fix(codecov)_: ignore folders without tests

Otherwise Codecov reports incorrect numbers when making changes.
https://docs.codecov.com/docs/ignoring-paths

Signed-off-by: Jakub Sokołowski <[email protected]>

test_: verify schema of signals during init; fix schema verification warnings (#5947)

fix_: update defaultGorushURL (#6011)

fix(tests)_: use non-standard port to avoid conflicts

We have observed `nimbus-eth2` build failures reporting this port:
```json
{
  "lvl": "NTC",
  "ts": "2024-10-28 13:51:32.308+00:00",
  "msg": "REST HTTP server could not be started",
  "topics": "beacnde",
  "address": "127.0.0.1:5432",
  "reason": "(98) Address already in use"
}
```
https://ci.status.im/job/nimbus-eth2/job/platforms/job/linux/job/x86_64/job/main/job/PR-6683/3/

Signed-off-by: Jakub Sokołowski <[email protected]>

fix_: create request logger ad-hoc in tests

Fixes `TestCall` failing when run concurrently.

chore_: configure codecov (#6005)

* chore_: configure codecov

* fix_: after_n_builds
  • Loading branch information
shashankshampi authored and fbarbu15 committed Nov 9, 2024
1 parent 4fc9361 commit bcef2d6
Show file tree
Hide file tree
Showing 27 changed files with 820 additions and 119 deletions.
19 changes: 12 additions & 7 deletions _assets/ci/Jenkinsfile.desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,14 +110,18 @@ pipeline {
}
}
}
stage('Cleanup') {
steps {
script {
cleanTmp()
}
}
}
} // stages
post {
success { script { github.notifyPR(true) } }
failure { script { github.notifyPR(false) } }
cleanup {
cleanWs()
cleanTmp()
}
cleanup { cleanWs() }
} // post
} // pipeline

Expand Down Expand Up @@ -154,8 +158,9 @@ def shell(cmd) {
}

def cleanTmp() {
/* Fails on windows due to Durable Task plugin failure. */
if (env.PLATFORM != 'windows') {
if (env.PLATFORM == 'windows') {
sh "rm -rf ${env.WORKSPACE}@tmp"
} else {
dir("${env.WORKSPACE}@tmp") { deleteDir() }
}
}
}
2 changes: 1 addition & 1 deletion _assets/ci/Jenkinsfile.tests
Original file line number Diff line number Diff line change
Expand Up @@ -254,4 +254,4 @@ def getDefaultUnitTestCount() { isNightlyJob() ? '20' : '1' }

def getDefaultTimeout() { isNightlyJob() ? 5*60 : 50 }

def getAmountToKeep() { isNightlyJob() ? '14' : isDevelopJob() ? '30' : '5' }
def getAmountToKeep() { isNightlyJob() ? '14' : isDevelopJob() ? '10' : '5' }
8 changes: 3 additions & 5 deletions api/geth_backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -113,8 +113,8 @@ func NewGethStatusBackend(logger *zap.Logger) *GethStatusBackend {
backend.initialize()

logger.Info("Status backend initialized",
zap.String("backend geth version", version.Version()),
zap.String("commit", version.GitCommit()),
zap.String("backend geth version", params.Version),
zap.String("commit", params.GitCommit),
zap.String("IpfsGatewayURL", params.IpfsGatewayURL))

return backend
Expand Down Expand Up @@ -2112,9 +2112,7 @@ func (b *GethStatusBackend) startNode(config *params.NodeConfig) (err error) {
}
}()

b.logger.Info("status-go version details",
zap.String("version", version.Version()),
zap.String("commit", version.GitCommit()))
b.logger.Info("status-go version details", zap.String("version", params.Version), zap.String("commit", params.GitCommit))
b.logger.Debug("starting node with config", zap.Stringer("config", config))
// Update config with some defaults.
if err := config.UpdateWithDefaults(); err != nil {
Expand Down
1 change: 0 additions & 1 deletion params/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ import (

"github.com/status-im/status-go/eth-node/crypto"
"github.com/status-im/status-go/eth-node/types"
"github.com/status-im/status-go/internal/version"
"github.com/status-im/status-go/logutils"
"github.com/status-im/status-go/static"
wakucommon "github.com/status-im/status-go/waku/common"
Expand Down
1 change: 0 additions & 1 deletion rpc/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ import (
"github.com/ethereum/go-ethereum/event"
appCommon "github.com/status-im/status-go/common"
"github.com/status-im/status-go/healthmanager"
"github.com/status-im/status-go/internal/version"
"github.com/status-im/status-go/logutils"
"github.com/status-im/status-go/params"
"github.com/status-im/status-go/rpc/chain"
Expand Down
1 change: 0 additions & 1 deletion services/ext/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@ import (
"github.com/status-im/status-go/eth-node/crypto"
"github.com/status-im/status-go/eth-node/types"
"github.com/status-im/status-go/images"
"github.com/status-im/status-go/internal/version"
"github.com/status-im/status-go/logutils"
"github.com/status-im/status-go/multiaccounts"
"github.com/status-im/status-go/multiaccounts/accounts"
Expand Down
47 changes: 6 additions & 41 deletions services/wallet/routeexecution/manager.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,11 @@ package routeexecution

import (
"context"
"database/sql"
"time"

"go.uber.org/zap"

"github.com/ethereum/go-ethereum/common"
"github.com/ethereum/go-ethereum/log"

"github.com/status-im/status-go/eth-node/types"
"github.com/status-im/status-go/logutils"

status_common "github.com/status-im/status-go/common"
statusErrors "github.com/status-im/status-go/errors"
Expand All @@ -28,26 +23,16 @@ type Manager struct {
router *router.Router
transactionManager *transfer.TransactionManager
transferController *transfer.Controller
db *DB

// Local data used for storage purposes
buildInputParams *requests.RouterBuildTransactionsParams
}

func NewManager(walletDB *sql.DB, router *router.Router, transactionManager *transfer.TransactionManager, transferController *transfer.Controller) *Manager {
func NewManager(router *router.Router, transactionManager *transfer.TransactionManager, transferController *transfer.Controller) *Manager {
return &Manager{
router: router,
transactionManager: transactionManager,
transferController: transferController,
db: NewDB(walletDB),
}
}

func (m *Manager) clearLocalRouteData() {
m.buildInputParams = nil
m.transactionManager.ClearLocalRouterTransactionsData()
}

func (m *Manager) BuildTransactionsFromRoute(ctx context.Context, buildInputParams *requests.RouterBuildTransactionsParams) {
go func() {
defer status_common.LogOnPanic()
Expand All @@ -63,7 +48,7 @@ func (m *Manager) BuildTransactionsFromRoute(ctx context.Context, buildInputPara

defer func() {
if err != nil {
m.clearLocalRouteData()
m.transactionManager.ClearLocalRouterTransactionsData()
err = statusErrors.CreateErrorResponseFromError(err)
response.SendDetails.ErrorResponse = err.(*statusErrors.ErrorResponse)
}
Expand All @@ -77,8 +62,6 @@ func (m *Manager) BuildTransactionsFromRoute(ctx context.Context, buildInputPara
return
}

m.buildInputParams = buildInputParams

updateFields(response.SendDetails, routeInputParams)

// notify client that sending transactions started (has 3 steps, building txs, signing txs, sending txs)
Expand Down Expand Up @@ -125,7 +108,7 @@ func (m *Manager) SendRouterTransactionsWithSignatures(ctx context.Context, send
}

if clearLocalData {
m.clearLocalRouteData()
m.transactionManager.ClearLocalRouterTransactionsData()
}

if err != nil {
Expand Down Expand Up @@ -180,20 +163,6 @@ func (m *Manager) SendRouterTransactionsWithSignatures(ctx context.Context, send
//////////////////////////////////////////////////////////////////////////////

response.SentTransactions, err = m.transactionManager.SendRouterTransactions(ctx, multiTx)
if err != nil {
log.Error("Error sending router transactions", "error", err)
// TODO #16556: Handle partially successful Tx sends?
// Don't return, store whichever transactions were successfully sent
}

// don't overwrite err since we want to process it in the deferred function
var tmpErr error
routerTransactions := m.transactionManager.GetRouterTransactions()
routeData := NewRouteData(&routeInputParams, m.buildInputParams, routerTransactions)
tmpErr = m.db.PutRouteData(routeData)
if tmpErr != nil {
log.Error("Error storing route data", "error", tmpErr)
}

var (
chainIDs []uint64
Expand All @@ -204,17 +173,13 @@ func (m *Manager) SendRouterTransactionsWithSignatures(ctx context.Context, send
addresses = append(addresses, common.Address(tx.FromAddress))
go func(chainId uint64, txHash common.Hash) {
defer status_common.LogOnPanic()
tmpErr = m.transactionManager.WatchTransaction(context.Background(), chainId, txHash)
if tmpErr != nil {
logutils.ZapLogger().Error("Error watching transaction", zap.Error(tmpErr))
err = m.transactionManager.WatchTransaction(context.Background(), chainId, txHash)
if err != nil {
return
}
}(tx.FromChain, common.Hash(tx.Hash))
}
tmpErr = m.transferController.CheckRecentHistory(chainIDs, addresses)
if tmpErr != nil {
logutils.ZapLogger().Error("Error checking recent history", zap.Error(tmpErr))
}
err = m.transferController.CheckRecentHistory(chainIDs, addresses)
}()
}

Expand Down
13 changes: 4 additions & 9 deletions services/wallet/service.go
Original file line number Diff line number Diff line change
Expand Up @@ -190,13 +190,13 @@ func NewService(
}

router := router.NewRouter(rpcClient, transactor, tokenManager, marketManager, collectibles,
collectiblesManager, ens)
pathProcessors := buildPathProcessors(rpcClient, transactor, tokenManager, ens, featureFlags)
collectiblesManager, ens, stickers)
pathProcessors := buildPathProcessors(rpcClient, transactor, tokenManager, ens, stickers, featureFlags)
for _, processor := range pathProcessors {
router.AddPathProcessor(processor)
}

routeExecutionManager := routeexecution.NewManager(db, router, transactionManager, transferController)
routeExecutionManager := routeexecution.NewManager(router, transactionManager, transferController)

return &Service{
db: db,
Expand Down Expand Up @@ -236,6 +236,7 @@ func buildPathProcessors(
transactor *transactions.Transactor,
tokenManager *token.Manager,
ens *ens.Service,
stickers *stickers.Service,
featureFlags *protocolCommon.FeatureFlags,
) []pathprocessor.PathProcessor {
ret := make([]pathprocessor.PathProcessor, 0)
Expand Down Expand Up @@ -267,12 +268,6 @@ func buildPathProcessors(
ensRelease := pathprocessor.NewENSReleaseProcessor(rpcClient, transactor, ens)
ret = append(ret, ensRelease)

ensPublicKey := pathprocessor.NewENSPublicKeyProcessor(rpcClient, transactor, ens)
ret = append(ret, ensPublicKey)

buyStickers := pathprocessor.NewStickersBuyProcessor(rpcClient, transactor)
ret = append(ret, buyStickers)

return ret
}

Expand Down
7 changes: 7 additions & 0 deletions services/wallet/transfer/commands.go
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,13 @@ func (c *transfersCommand) Run(ctx context.Context) (err error) {
logutils.ZapLogger().Error("saveAndConfirmPending error", zap.Error(err))
return err
}

// Check if multi transaction needs to be created
err = c.processMultiTransactions(ctx, allTransfers)
if err != nil {
logutils.ZapLogger().Error("processMultiTransactions error", zap.Error(err))
return err
}
} else {
// If no transfers found, that is suspecting, because downloader returned this block as containing transfers
logutils.ZapLogger().Error("no transfers found in block",
Expand Down
3 changes: 1 addition & 2 deletions telemetry/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,9 @@ import (

wps "github.com/waku-org/go-waku/waku/v2/peerstore"

v2protocol "github.com/waku-org/go-waku/waku/v2/protocol"

v1protocol "github.com/status-im/status-go/protocol/v1"
v2common "github.com/status-im/status-go/wakuv2/common"
v2protocol "github.com/waku-org/go-waku/waku/v2/protocol"
)

type TelemetryType string
Expand Down
2 changes: 2 additions & 0 deletions tests-functional/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
.idea/
.local/
106 changes: 84 additions & 22 deletions tests-functional/README.MD
Original file line number Diff line number Diff line change
@@ -1,41 +1,103 @@
Here’s the updated README with the additional prerequisites and instructions:

---

## Overview

Functional tests for status-go
Functional tests for `status-go`

## Table of Contents

- [Overview](#overview)
- [How to Install](#how-to-install)
- [How to Run](#how-to-run)
- [Running Tests](#running-tests)
- [Implementation details](#implementation-details)
- [Implementation Details](#implementation-details)
- [Build Status Backend](#build-status-backend)

## How to Install

* Install [Docker](https://docs.docker.com/engine/install/) and [Docker Compose](https://docs.docker.com/compose/install/)
* Install [Python 3.10.14](https://www.python.org/downloads/)
* In `./tests-functional`, run `pip install -r requirements.txt`
* **Optional (for test development)**: Use Python virtual environment for better dependency management. You can follow the guide [here](https://akrabat.com/creating-virtual-environments-with-pyenv/):
1. Install [Docker](https://docs.docker.com/engine/install/) and [Docker Compose](https://docs.docker.com/compose/install/)
2. Install [Python 3.10.14](https://www.python.org/downloads/)
3. **Set up a virtual environment (recommended):**
- In `./tests-functional`, run:
```bash
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
- **Optional (for test development)**: Use Python virtual environment for better dependency management. You can follow the guide [here](https://akrabat.com/creating-virtual-environments-with-pyenv/)
4. Install pre-commit hooks (optional):
```bash
pre-commit install
```

## How to Run

### Running dev RPC (anvil with contracts)
- In `./tests-functional` run `docker compose -f docker-compose.anvil.yml up --remove-orphans --build`, as result:
* an [anvil](https://book.getfoundry.sh/reference/anvil/) container with ChainID 31337 exposed on `0.0.0.0:8545` will start running
* Status-im contracts will be deployed to the network
### Running dev RPC (Anvil with contracts)

In `./tests-functional`:
```bash
docker compose -f docker-compose.anvil.yml up --remove-orphans --build
```

This command will:
- Start an [Anvil](https://book.getfoundry.sh/reference/anvil/) container with ChainID `31337`, exposed on `0.0.0.0:8545`
- Deploy Status-im contracts to the Anvil network

### Running Tests

To run the tests:

1. In `./tests-functional`, start the testing containers:
```bash
docker compose -f docker-compose.anvil.yml -f docker-compose.test.status-go.yml -f docker-compose.status-go.local.yml up --build --remove-orphans
```

This command will:
- Create a container with [status-go as daemon](https://github.com/status-im/status-go/issues/5175), exposing `APIModules` on `0.0.0.0:3333`
- Configure `status-go` to use [Anvil](https://book.getfoundry.sh/reference/anvil/) as the `RPCURL` with ChainID `31337`
- Deploy all Status-im contracts to the Anvil network

2. To execute tests:
- Run all tests:
```bash
pytest
```
- Run tests marked as `wallet`:
```bash
pytest -m wallet
```
- Run a specific test:
```bash
pytest -k "test_contact_request_baseline"
```

## Implementation Details

- Functional tests are implemented in `./tests-functional/tests` using [pytest](https://docs.pytest.org/en/8.2.x/).
- Each test performs two types of verifications:
- **`verify_is_valid_json_rpc_response()`**: Checks for a status code `200`, a non-empty response, JSON-RPC structure, presence of the `result` field, and the expected ID.
- **`jsonschema.validate()`**: Validates that the response contains expected data, including required fields and types. Schemas are stored in `/schemas/wallet_MethodName`.
- **Schema Generation**:
- New schemas can be generated with `./tests-functional/schema_builder.py` by passing a response to the `CustomSchemaBuilder(schema_name).create_schema(response.json())` method. This should be used only during test creation.
- Search `how to create schema:` in test files for examples.
## Build Status Backend
You can build the binary with the following command in the `status-go` root directory:
```bash
make status-backend
```

### Run tests
- In `./tests-functional` run `docker compose -f docker-compose.anvil.yml -f docker-compose.test.status-go.yml -f docker-compose.status-go.local.yml up --build --remove-orphans`, as result:
* a container with [status-go as daemon](https://github.com/status-im/status-go/issues/5175) will be created with APIModules exposed on `0.0.0.0:3333`
* status-go will use [anvil](https://book.getfoundry.sh/reference/anvil/) as RPCURL with ChainID 31337
* all Status-im contracts will be deployed to the network
For further details on building and setting up `status-go` and `status-backend`, refer to the official documentation:
- [status-backend README](https://github.com/status-im/status-go/blob/develop/cmd/status-backend/README.md)
- [status-go cmd directory](https://github.com/status-im/status-go/tree/develop/cmd/status-backend)

* In `./tests-functional/tests` directory run `pytest -m wallet`
Location of the binary: `cmd/status-backend/status-backend`

## Implementation details
---

- Functional tests are implemented in `./tests-functional/tests` based on [pytest](https://docs.pytest.org/en/8.2.x/)
- Every test has two types of verifications:
- `verify_is_valid_json_rpc_response()` checks for status code 200, non-empty response, JSON-RPC structure, presence of the `result` field, and expected ID.
- `jsonschema.validate()` is used to check that the response contains expected data, including required fields and types. Schemas are stored in `/schemas/wallet_MethodName`
- New schemas can be generated using `./tests-functional/schema_builder.py` by passing a response to the `CustomSchemaBuilder(schema_name).create_schema(response.json())` method, should be used only on test creation phase, please search `how to create schema:` to see an example in a test
This README should cover your additional setup, installation, and testing instructions with clear steps for users. Let me know if there are any further modifications needed!
Loading

0 comments on commit bcef2d6

Please sign in to comment.