Support automated tests for full-node mode and light-client mode

[Keith-CY]

Recently we found some unexpected behaviors of Neuron when it was in the light client mode. Then I realized that Neuron is a very special case in CKB ecosystem because it's the only product that supports full-node and light-client simultaneously. That means, tests should be doubled for the same function.

Due to the increment in workload, we should accelerate the automation of tests.

It would be challenging because we haven't figured out how to run the e2e tests with full-node in the github runner but now there comes another ckb client.

I'll connect to acceptance team to see if they have handled such situations.

Ref:

• Support E2E tests in Neuron's repo by github action: Support E2E tests in Neuron's repo by github action #132

[Keith-CY]

IMO, Neuron's E2E testing can be divided into two categories:

1. UI interaction testing, involving link navigation, form input, and data display.
2. Testing the synchronization of on-chain data, ensuring that data remains consistent and complete for a given wallet when synchronized to a specific height.

The first category of testing is similar to standard software testing processes and can be supported by automating verification by importing test data into Neuron's database. The specific workflow is as follows:

1. Trigger automated testing upon code submission.
2. Before running the tests, repackage Neuron.
3. Before running Neuron, replace its built-in database with a test database, or use a script to write data into Neuron. Using a script is recommended, as it allows for adapting to data format changes, such as adding or removing fields.
4. Run Neuron and use testing tools to verify the interaction of the pages.

The second category of testing is different from standard software testing and is more similar to end-to-end testing for backend services. One challenging aspect is the need to cover both full node and light client modes. However, after isolating the full node and light client databases (#294), it is possible to complete this testing by comparing database snapshots. The specific steps are as follows:

1. Select a wallet with complete test data and export databases for both the full node and the light client, synchronized to the same height, as the testing baseline.
2. Build a persistent testing environment to provide external nodes and light clients.
3. Before running the tests, repackage Neuron.
4. Run Neuron and wait for it to synchronize to the specified height.
5. Use a script to compare Neuron's database with the baseline database.

Since the data volume for light clients is smaller, testing the synchronization of light clients can also be performed by importing them into the testing environment each time, rather than maintaining a persistent environment.

[silySuper]

In order to ensure the continuity of dynamic operations，we should consider how to make data usable.For example,if we want to test send transaction firstly ,then send lock time transaction,sufficient balance in wallet is needed to advoid throw abnormal tips and automated situation are interrupted.

[Keith-CY]

Now that E2E tests are grouped into 3 parts, we can make schedules for each one.

I'm a bit familiar with test frameworks for UI/interactions, so I will lead this part.

There are 2 frameworks for an electron app

• cypress: https://docs.cypress.io/guides/guides/launching-browsers#Electron-Browser
• playwright: https://playwright.dev/docs/api/class-electron

They are almost the same for our project except

1. cypress is somehow heavy because it relies on a local app(CI would fail because it cannot download the app, really bothering);
2. playwright is so fresh that its support for electron is experimental.

Personally prefer playwright for our test framework of UI because failure to download cypress in CI did annoy me, any idea from @yanguoyu @homura @WhiteMinds @devchenyan @zhangyouxin @Daryl-L @PainterPuppets

[Keith-CY]

Now that E2E tests are grouped into 3 parts, we can make schedules for each one.

I'm a bit familiar with test frameworks for UI/interactions, so I will lead this part.

There are 2 frameworks for an electron app

• cypress: https://docs.cypress.io/guides/guides/launching-browsers#Electron-Browser

• playwright: https://playwright.dev/docs/api/class-electron

They are almost the same for our project except

1. cypress is somehow heavy because it relies on a local app(CI would fail because it cannot download the app, really bothering);

2. playwright is so fresh that its support for electron is experimental.

Personally prefer playwright for our test framework of UI because failure to download cypress in CI did annoy me, any idea from @yanguoyu @homura @WhiteMinds @devchenyan @zhangyouxin @Daryl-L @PainterPuppets

Playwright will be used if there's no more feedback

[Danie0918]

@Keith-CY This issue will be taken by @devchenyan, please hand over the relevant work content.

[Keith-CY]

@Keith-CY This issue will be taken by @devchenyan, please hand over the relevant work content.

I'll DM @devchenyan for this task

[silySuper]

Which github code link is related to this ?

[Keith-CY]

Which github code link is related to this ?

The PR is nervosnetwork/neuron#3004 but I found the CI failed, please have a check @devchenyan

[Keith-CY]

Which github code link is related to this ?

The PR is nervosnetwork/neuron#3004 but I found the CI failed, please have a check @devchenyan

Any update on this PR @devchenyan

[silySuper]

Now neuron can launch ,but can not locate element,what is the way to locate electron element ?@devchenyan

[silySuper]

If launch by manual operation，we can write send transaction as first example ,does send page element can be located?can you upload a video for writen process if it is convenient for you ? @devchenyan

[silySuper]

Change to light client is not my main point.I want to use testnet to run cases,but it can not.
bundled-ckb.log has exported above.

[silySuper]

I have found that a Electron helper can not exit automatically when neuron exit.

2024-07-04.10.22.05.mov

when force out this helper ,and open neuron again ,tne sync become normal,maybe my problem is related to https://github.com/orgs/Magickbase/projects/6/views/2?pane=issue&itemId=68796884

[yanguoyu]

I have found that a Electron helper can not exit automatically when neuron exit.

2024-07-04.10.22.05.mov
when force out this helper ,and open neuron again ,tne sync become normal,maybe my problem is related to https://github.com/orgs/Magickbase/projects/6/views/2?pane=issue&itemId=68796884

I will check this, but it's an exception branch, so it may not break the main process.

[yanguoyu]

Change to light client is not my main point.I want to use testnet to run cases,but it can not. bundled-ckb.log has exported above.

Does this log exported from the ckb that you started locally?

[silySuper]

Change to light client is not my main point.I want to use testnet to run cases,but it can not. bundled-ckb.log has exported above.

Does this log exported from the ckb that you started locally?

Yes,only have a little content in this log when network is abnormal.

[yanguoyu]

Change to light client is not my main point.I want to use testnet to run cases,but it can not. bundled-ckb.log has exported above.

Does this log exported from the ckb that you started locally?

Yes,only have a little content in this log when network is abnormal.

So, it may has a problem when you start the local ckb node. It is not a problem with Neuron. Because the locally started ckb is not controlled by Neuron.

[silySuper]

Change to light client is not my main point.I want to use testnet to run cases,but it can not. bundled-ckb.log has exported above.

Does this log exported from the ckb that you started locally?

Yes,only have a little content in this log when network is abnormal.

So, it may has a problem when you start the local ckb node. It is not a problem with Neuron. Because the locally started ckb is not controlled by Neuron.

Ckb node is running normally in background,it happened when neruon exit.I have operated stop ckb node to reproduct sync problem before ,but sync is normal.

[yanguoyu]

sync is normal.

So currently sync is normal?

[silySuper]

sync is normal.

So currently sync is normal?

Now is normal.

[silySuper]

/neuron/packages/e2e/README.md has updated.

[Danie0918]

All test cases have been completed, but they need to be run manually on a regular basis and cannot be fully automated, pending subsequent optimization.

[silySuper]

Maybe CI failed is not related to my testcase.I found that CI never success in history. I will try to check configuration firstly.

[silySuper]

CI environment shows an database error.

[yanguoyu]

CI environment shows an database error.

Which action throws this error or it can be reproduced locally?

[silySuper]

CI environment shows an database error.

Which action throws this error or it can be reproduced locally?

After UI automic create wallet it shows this today，test cases are same，you can rerun test cases to see error.

[yanguoyu]

CI environment shows an database error.

Which action throws this error or it can be reproduced locally?

After UI automic create wallet it shows this today，test cases are same，you can rerun test cases to see error.

I ran with the branch enable-playright, and I had no problem. Please describe how to reproduce the issue.

[silySuper]

I do nothing,just run CI...OK, if it is right now ,I will go on.

[silySuper]

I have balance ,and have no confirming record in history,also sync 100%.Why always show insufficient balance when I send transaction?

[yanguoyu]

I have balance ,and have no confirming record in history,also sync 100%.Why always show insufficient balance when I send transaction?

When you input the amount, the transactions have not synced yet. After the transactions are synced, the amount doesn't change, so the page will not refresh in the screenshot. After I change the amount, it will show correct.

2024-08-13.15.05.55.mov

Besides, after this commit,
And change this line https://github.com/nervosnetwork/neuron/blob/develop/packages/neuron-wallet/src/services/settings.ts#L123 locally

// do not push this change to the remote
- locale: app.getLocale(),
+ locale: 'en',

Running locally e2e test case will be similar to the action running.

[silySuper]