Skip to content

Commit

Permalink
Merge branch 'master' into db-migration-g
Browse files Browse the repository at this point in the history
  • Loading branch information
jasonqiu212 authored Mar 6, 2025
2 parents 6137cf0 + 36da42c commit ffeffa5
Show file tree
Hide file tree
Showing 10 changed files with 320 additions and 156 deletions.
2 changes: 1 addition & 1 deletion docs/_markbind/layouts/default.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,10 @@
* [Emails]({{ baseUrl }}/emails.html)
* [Unit Testing]({{ baseUrl }}/unit-testing.html)
* [End-to-End Testing]({{ baseUrl }}/e2e-testing.html)
* [Snapshot Testing]({{ baseUrl }}/snapshot-testing.html)
* [Performance Testing]({{ baseUrl }}/performance-testing.html)
* [Accessibility Testing]({{ baseUrl }}/axe-testing.html)
* [Search]({{ baseUrl }}/search.html)
* [Snapshot Testing]({{ baseUrl }}/snapshot-testing.html)
* [Static Analysis]({{ baseUrl }}/static-analysis.html)
* [Troubleshooting Guide]({{ baseUrl }}/troubleshooting-guide.html)
* [Glossary]({{ baseUrl }}/glossary.html)
Expand Down
53 changes: 5 additions & 48 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,56 +279,13 @@ If you are using the Cloud SDK method, you can use `Ctrl + C` in the console to

There are two big categories of testing in TEAMMATES:

- **Component tests**: white-box unit and integration tests, i.e. they test the application components with full knowledge of the components' internal workings. This is configured in `src/test/resources/testng-component.xml` (back-end) and `src/web/jest.config.js` (front-end).
- **E2E (end-to-end) tests**: black-box tests, i.e. they test the application as a whole without knowing any internal working. This is configured in `src/e2e/resources/testng-e2e.xml`. To learn more about E2E tests, refer to this [document](e2e-testing.md).
- **Component tests**: White-box unit and integration tests, i.e. they test the application components with full knowledge of the components' internal workings. To learn more about running and writing component tests, refer to [this guide](unit-testing.md).
- **<tooltip content="End-to-end">E2E</tooltip> tests**: Black-box tests, i.e. they test the application as a whole without knowing any internal working. To learn more about running and writing E2E tests, refer to [this guide](e2e-testing.md).

<div id="running-tests">
Other tests in TEAMMATES include:

#### Running tests

##### Frontend tests

To run all front-end component tests in watch mode (i.e. any change to source code will automatically reload the tests), run the following command:

```sh
npm run test
```

To update snapshots, run the following command:
```sh
npm run test
```

Followed by `a` to run all the test cases. Check through the snapshots to make sure that the changes are as expected, and press `u` to update them.

To run all front-end component tests once and generate coverage data afterwards, run the following command:

```sh
npm run coverage
```

To run an individual test in a test file, change `it` in the `*.spec.ts` file to `fit`.

To run all tests in a test file (or all test files matching a pattern), you can use Jest's watch mode and filter by filename pattern.

##### Backend tests

Back-end component tests follow this configuration:

Test suite | Command | Results can be viewed in
---|---|---
`Component tests` | `./gradlew componentTests --continue` | `{project folder}/build/reports/tests/componentTests/index.html`
Any individual component test | `./gradlew componentTests --tests TestClassName` | `{project folder}/build/reports/tests/componentTests/index.html`

You can generate the coverage data with `jacocoReport` task after running tests, e.g.:

```sh
./gradlew componentTests jacocoReport
```

The report can be found in the `build/reports/jacoco/jacocoReport/` directory.

</div>
- **Performance tests**: Evaluate the application's stability and responsiveness under expected, peak, and prolonged loads. See [Performance Testing](performance-testing.md).
- **Accessibility tests**: Ensure the application is accessible to users with disabilities. See [Accessibility Testing](axe-testing.md).

## Deploying to a staging server

Expand Down
19 changes: 9 additions & 10 deletions docs/e2e-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
- It aims to ensure all integrated components of the application work together as expected when it is being used by the end user.
- This is done by simulating user scenarios on the fully built product.

E2E tests in TEAMMATES can be found in the package `teammates.e2e`.
E2E tests in TEAMMATES are located in the package `teammates.e2e` and configured in `src/e2e/resources/testng-e2e.xml`.

## Running E2E tests

Expand Down Expand Up @@ -69,11 +69,12 @@ Only modern Edge (Chromium-based) is supported.
<br>

### Running the tests

E2E tests follow this configuration:

Test suite | Command | Results can be viewed in
---|---|---
`E2E tests` | `./gradlew e2eTests` | `{project folder}/build/reports/e2e-test-try-{n}/index.html`, where `{n}` is the sequence number of the test run
E2E tests | `./gradlew e2eTests` | `{project folder}/build/reports/e2e-test-try-{n}/index.html`, where `{n}` is the sequence number of the test run
Any individual E2E test | `./gradlew e2eTestTry1 --tests TestClassName` | `{project folder}/build/reports/e2e-test-try-1/index.html`

- `E2E tests` will be run in their entirety once and the failed tests will be re-run a few times.
Expand All @@ -86,14 +87,12 @@ If you are testing against a production server (staging server or live server),

1. Edit `src/e2e/resources/test.properties` as instructed is in its comments.
* In particular, you will need a legitimate Gmail account to be used for testing.

1. If you are testing email sending, you need to setup a `Gmail API` as follows:
2. If you are testing email sending, you need to setup a `Gmail API` as follows:
* [Obtain a Gmail API credentials](https://github.com/TEAMMATES/teammates-ops/blob/master/platform-guide.md#setting-up-gmail-api-credentials) and download it.
* Copy the file to `src/e2e/resources/gmail-api` (create the `gmail-api` folder) of your project and rename it to `client_secret.json`.
* It is also possible to use the Gmail API credentials from any other Google Cloud Platform project for this purpose.
* Run `EmailAccountTest` to confirm that the setup works. For the first run, it is expected that you will need to grant access from the test Gmail account to the above API.

1. Run the full test suite or any subset of it as how you would have done it in dev server.
3. Run the full test suite or any subset of it as how you would have done it in dev server.
* Do note that the GAE daily quota is usually not enough to run the full test suite, in particular for accounts with no billing enabled.

## Creating E2E tests
Expand All @@ -103,8 +102,8 @@ As E2E tests should be written from the end user perspective, each test case sho
In TEAMMATES, E2E test cases are organized by page. For each page, we:

1. Identify the important user workflows
1. Simulate the user actions involved in the workflow by interacting with the UI elements.
1. Assert the expected conditions are present after the interaction.
2. Simulate the user actions involved in the workflow by interacting with the UI elements.
3. Assert the expected conditions are present after the interaction.

[Selenium](https://www.selenium.dev/) is used to locate and interact with elements in the UI.

Expand Down Expand Up @@ -138,8 +137,8 @@ All Page Object classes inherit from `AppPage` which contains methods that are c
### Things to avoid when writing E2E tests

1. **Testing based on implementation** - The focus should be on user actions instead of implementation details. Therefore, black box testing should be adopted and test cases should be designed around use cases.
1. **Excessive exception testing** - Testing edge cases with E2E tests should be avoided. This is because E2E tests are expensive to run and not that effective for isolating bugs. Hence we should focus on the happy path and exception paths that are more common. We should leave more exhaustive testing to lower-level unit or integration tests.
1. **Not following "Tell Don't Ask" Principle** - Instead of "asking" for data from the page objects and performing operations on them, "tell" the page object to do the operations. This is mostly seen in the verification methods where assertions are done in the page object instead of in the test case. This improves readability and maintainability as data and behavior are placed together.
2. **Excessive exception testing** - Testing edge cases with E2E tests should be avoided. This is because E2E tests are expensive to run and not that effective for isolating bugs. Hence we should focus on the happy path and exception paths that are more common. We should leave more exhaustive testing to lower-level unit or integration tests.
3. **Not following "Tell Don't Ask" Principle** - Instead of "asking" for data from the page objects and performing operations on them, "tell" the page object to do the operations. This is mostly seen in the verification methods where assertions are done in the page object instead of in the test case. This improves readability and maintainability as data and behavior are placed together.

### FAQ

Expand Down
20 changes: 11 additions & 9 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,22 +25,24 @@ Here are some documents important for TEAMMATES developers.
### Supplementary documents

* **Coding standards** for:
[Java](https://oss-generic.github.io/process/codingStandards/CodingStandard-Java.html),
[CSS](https://oss-generic.github.io/process/codingStandards/CodingStandard-Css.html),
[HTML](https://oss-generic.github.io/process/codingStandards/CodingStandard-Html.html)

* [Java](https://oss-generic.github.io/process/codingStandards/CodingStandard-Java.html),
* [CSS](https://oss-generic.github.io/process/codingStandards/CodingStandard-Css.html),
* [HTML](https://oss-generic.github.io/process/codingStandards/CodingStandard-Html.html)
* **Best practices** for:
* [Coding](best-practices/coding.md)
* [Testing](best-practices/testing.md)
* [Data migration](best-practices/data-migration.md)
* [UI design](best-practices/ui-design.md)
* [Accessibility](best-practices/accessibility.md)
* [Mobile-friendliness](best-practices/mobile-friendliness.md)

* **How-to guides** for:
* [Static analysis](static-analysis.md): Performing code quality checks
* [Setting up third-party email providers](emails.md)
* [Setting up CAPTCHA](captcha.md)
* [Setting up developer documentation](documentation.md)
* [Setting up third-party email providers](emails.md)
* [Unit testing](unit-testing.md)
* [End-to-End testing](e2e-testing.md)
* [Snapshot testing](snapshot-testing.md)
* [E2E testing](e2e-testing.md)
* [Accessibility testing](axe-testing.md)
* [Performance testing](performance-testing.md)
* [Accessibility testing](axe-testing.md)
* [Setting up full-text search](search.md)
* [Static analysis](static-analysis.md): Performing code quality checks
20 changes: 10 additions & 10 deletions docs/performance-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,6 @@ TEAMMATES makes use of [JMeter](https://jmeter.apache.org/) for load and perform

The performance test cases are located in the [`teammates.lnp.cases`](https://github.com/TEAMMATES/teammates/tree/master/src/lnp/java/teammates/lnp/cases) package.

## Creating Performance Tests

Each new test case must inherit the `BaseLNPTestCase` class, and implement the methods required for generating the test data and the JMeter L&P test plan. The L&P test plans are created in Java using the JMeter API.

The inherited test cases can run JMeter test by calling `runJmeter` method. When passing the parameter `shouldCreateJmxFile` as `true`, an equivalent `.jmx` file can be generated from this test plan.

To help with debugging, you can open this `.jmx` file in the JMeter GUI and add Listeners.

To see a sample implementation of a test case, you can refer to `FeedbackSessionSubmitLNPTest`. It is a _simple_ test case which load tests a PUT endpoint (`/webapi/responses`).

## Running Performance Tests

If you want to use your own copy of [JMeter](https://jmeter.apache.org/download_jmeter.cgi), update the `test.jmeter.*` properties in `src/lnp/resources/test.properties` accordingly.
Expand Down Expand Up @@ -68,3 +58,13 @@ However, you should not use the GUI to run large scale tests as it is very resou

Remember to **disable or remove all `Listeners`** in the `.jmx` file, unless you are debugging. Having them enabled can have a negative impact on the test performance.
</box>

## Creating Performance Tests

Each new test case must inherit the `BaseLNPTestCase` class, and implement the methods required for generating the test data and the JMeter L&P test plan. The L&P test plans are created in Java using the JMeter API.

The inherited test cases can run JMeter test by calling `runJmeter` method. When passing the parameter `shouldCreateJmxFile` as `true`, an equivalent `.jmx` file can be generated from this test plan.

To help with debugging, you can open this `.jmx` file in the JMeter GUI and add Listeners.

To see a sample implementation of a test case, you can refer to `FeedbackSessionSubmitLNPTest`. It is a _simple_ test case which load tests a PUT endpoint (`/webapi/responses`).
Loading

0 comments on commit ffeffa5

Please sign in to comment.