Skip to content

Commit

Permalink
docs(contributing): change wording
Browse files Browse the repository at this point in the history
Co-Authored-By: Sarah Dayan <[email protected]>
  • Loading branch information
Haroenv and sarahdayan committed Nov 24, 2022
1 parent 7c0d464 commit 2931f8f
Show file tree
Hide file tree
Showing 2 changed files with 61 additions and 53 deletions.
10 changes: 6 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,21 +173,23 @@ yarn test --watch

### End-to-end tests

End-to-end tests are defined by the [tests/e2e](./tests/e2e/README.md) project.
End-to-end tests are defined in [tests/e2e](./tests/e2e/README.md).

To run them locally:

```sh
yarn test:e2e:local
```

To run them on saucelabs, with credentials as `SAUCE_USERNAME` and `SAUCE_ACCESS_KEY` environment variable:

To run them on Sauce Labs:
```sh
yarn test:e2e:saucelabs
```

More info, including how to write the tests, can be found in its [CONTRIBUTING](./tests/e2e/CONTRIBUTING.md) and [README](./tests/e2e/README.md) files.
> **Note**
>Make sure to set up Sauce Labs credentials with the `SAUCE_USERNAME` and `SAUCE_ACCESS_KEY` environment variables.
For more info, including how to write end-to-end tests, check the `tests/e2e` [CONTRIBUTING](./tests/e2e/CONTRIBUTING.md) and [README](./tests/e2e/README.md) files.

### Type checks

Expand Down
104 changes: 55 additions & 49 deletions tests/e2e/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Contributing

This repository contains the end-to-end (e2e) test suite for InstantSearch. This test suite is meant to be shared across all InstantSearch flavors, this is why it is stored in the monorepo.
This repository contains the end-to-end (e2e) test suite for InstantSearch.

This test suite is meant to be shared across all InstantSearch flavors, which is why it lives in the monorepo.

## Development

Expand All @@ -23,70 +25,74 @@ See [WebdriverIO CLI documentation](https://webdriver.io/docs/clioptions.html) f

### Writing tests

We use [Webdriverio](https://webdriver.io) and [Jasmine](https://jasmine.github.io) to write the tests. Webdriverio is used to control the browser (navigate, select elements, clicks, etc.) while Jasmine is used to assert the results.
We use [WebdriverIO](https://webdriver.io) and [Jasmine](https://jasmine.github.io) to write the tests. WebdriverIO controls the browser (navigate, select elements, click, etc.) while Jasmine is used to make assertions.

The tests must be stored in the [`specs`](specs) directory and follow the `*.spec.ts` naming pattern. Each `*.spec.ts` file will be executed in a new WebDriver session and if possible in parallel (to the limit of the `maxInstances` configuration parameter in `wdio.*.conf.js`).
All tests must go in the [`specs`](specs) directory and follow the `*.spec.ts` naming pattern. Each `*.spec.ts` file runs in a separate WebdriverIO session, and in parallel (to the limit of the `maxInstances` configuration parameter in `wdio.*.conf.js`).

In general, you should try to test a single feature in one spec file. Try to not have too many or too few tests in one file. However, there is no golden rule about that.
The general rule is to test a single feature in one spec file—for example, pagination. If your test suite has a lot less or a lot more tests than the existing ones, you might have incorrectly scoped the feature you're trying to cover.

One spec file represents a scenario to test a behavior from a user point of view.
Each spec file represents a end-to-end scenario which tests a behavior from a user's point of view.

Example of scenario:
For example:

1. Load the `examples/js/e-commerce/` page
2. Click on "Appliances" category
2. Click on the "Appliances" category
3. Click on rating "4 & up"
4. Check if the result list matches the expected one
4. Assert whether or not the returned hits match the expected ones

Which can be translated in a spec file to:
This user scenario translates to the following spec:

```js
describe('InstantSearch - Search on specific brand and query filtering', () => {
it('navigates to the e-commerce demo', async () => {
await browser.url('examples/e-commerce/');
});

it('selects "Appliances" category', async () => {
await browser.setSelectedHierarchicalMenuItem('Appliances');
});

it('selects "4 & up" rating', async () => {
await browser.setRatingMenuValue('4 & up');
});

it('must have the expected results', async () => {
const hitsTitles = await browser.getHitsTitles();

expect(hitsTitles).toEqual([
'Nest - Learning Thermostat - 3rd Generation - Stainless Steel',
'LG - 1.1 Cu. Ft. Mid-Size Microwave - Stainless-Steel',
'Insignia™ - 2.6 Cu. Ft. Compact Refrigerator - Black',
'Keurig - K50 Coffeemaker - Black',
'iRobot - Roomba 650 Vacuuming Robot - Black',
'Shark - Navigator Lift-Away Deluxe Bagless Upright Vacuum - Blue',
'LG - 1.5 Cu. Ft. Mid-Size Microwave - Stainless Steel',
'Insignia™ - 5.0 Cu. Ft. Chest Freezer - White',
'Samsung - activewash 4.8 Cu. Ft. 11-Cycle High-Efficiency Top-Loading Washer - White',
'Samsung - 4.8 Cu. Ft. 11-Cycle High-Efficiency Top-Loading Washer - White',
'LG - 2.0 Cu. Ft. Full-Size Microwave - Stainless Steel',
'Shark - Rotator Professional Lift-Away HEPA Bagless 2-in-1 Upright Vacuum - Red',
'LG - 4.5 Cu. Ft. 8-Cycle High-Efficiency Top-Loading Washer - White',
'Samsung - 4.2 Cu. Ft. 9-Cycle High-Efficiency Steam Front-Loading Washer - Platinum',
'Panasonic - 1.3 Cu. Ft. Mid-Size Microwave - Stainless steel/black/silver',
'LG - 2.0 Cu. Ft. Mid-Size Microwave - Black Stainless',
]);
function createBrandAndQueryFilteringTestSuite(flavor: string) {
const root = `examples/${flavor}/e-commerce/`;

describe('search on a specific category', () => {
it('navigates to the e-commerce demo', async () => {
await browser.url(root);
});

it('selects "Appliances" category', async () => {
await browser.setSelectedHierarchicalMenuItem('Appliances');
});

it('selects "4 & up" rating', async () => {
await browser.setRatingMenuValue('4 & up');
});

it('must have the expected results', async () => {
const hitsTitles = await browser.getHitsTitles();

expect(hitsTitles).toEqual([
'Nest - Learning Thermostat - 3rd Generation - Stainless Steel',
'LG - 1.1 Cu. Ft. Mid-Size Microwave - Stainless-Steel',
'Insignia™ - 2.6 Cu. Ft. Compact Refrigerator - Black',
'Keurig - K50 Coffeemaker - Black',
'iRobot - Roomba 650 Vacuuming Robot - Black',
'Shark - Navigator Lift-Away Deluxe Bagless Upright Vacuum - Blue',
'LG - 1.5 Cu. Ft. Mid-Size Microwave - Stainless Steel',
'Insignia™ - 5.0 Cu. Ft. Chest Freezer - White',
'Samsung - activewash 4.8 Cu. Ft. 11-Cycle High-Efficiency Top-Loading Washer - White',
'Samsung - 4.8 Cu. Ft. 11-Cycle High-Efficiency Top-Loading Washer - White',
'LG - 2.0 Cu. Ft. Full-Size Microwave - Stainless Steel',
'Shark - Rotator Professional Lift-Away HEPA Bagless 2-in-1 Upright Vacuum - Red',
'LG - 4.5 Cu. Ft. 8-Cycle High-Efficiency Top-Loading Washer - White',
'Samsung - 4.2 Cu. Ft. 9-Cycle High-Efficiency Steam Front-Loading Washer - Platinum',
'Panasonic - 1.3 Cu. Ft. Mid-Size Microwave - Stainless steel/black/silver',
'LG - 2.0 Cu. Ft. Mid-Size Microwave - Black Stainless',
]);
});
});
});
}
```

General guidelines when writing tests:
Here are some general guidelines when writing end-to-end tests:

- Each step should be translated as a `it` function, while this is not mandatory it helps to make the scenario more readable and properly [separate each actions in Sauce Labs reports](https://user-images.githubusercontent.com/13209/62311104-56217d80-b48b-11e9-94dc-3c18b9ddc2af.png).
- Each separate step should be in its own `it` function. This isn't a hard rule, but it helps make the scenario more readable and properly [separate each action in Sauce Labs reports](https://user-images.githubusercontent.com/13209/62311104-56217d80-b48b-11e9-94dc-3c18b9ddc2af.png).
- All actions on the browser are asynchronous, so be sure to always `await` them. **Never run multiple asynchronous commands in parallel as it can confuse some browsers (Internet Explorer)**.
- Use helper functions when possible, for readability but also for maintainability (if one widget is updated, we only have to updates its helpers without touching the tests). [More about Helpers](#helpers).
- Do not make assertions to know if an action was correctly performed in the browser. If an action fails (trying to click on an non-existing element for example) then WebdriverIO will automatically throw and fail the test, so asserting on it ourselves is redundant.
- Use [helper](#helpers) functions whenever possible, for readability but also for maintainabilityif one widget is updated, we only have to updates its helpers without touching the tests.
- Only assert what you want to see on the page after an actionfor example, is the checkbox selected, is the result list correct, etc.
- Only assert what you want to see on the page after an action (Is this checkbox selected? Is the result list correct? etc.)
- You may need to add some additional steps compared to the original scenario, to wait for some elements to update for example (this was not done in the example above for simplicity but you can find some examples [here](https://github.com/algolia/instantsearch-e2e-tests/blob/5a2456b8d63afa684931b0447f00df821b02752b/specs/brand-and-query.spec.ts#L14-L16) or [here](https://github.com/algolia/instantsearch-e2e-tests/blob/5a2456b8d63afa684931b0447f00df821b02752b/specs/price-range.spec.ts#L13-L22)).
- You may need to add some additional steps compared to the original scenario, to wait for some elements to update for example (this was not done in the example above for simplicity but you can find some examples [here](./specs/brand-and-query.spec.ts#L17-19) or [here](./specs/price-range.spec.ts#L16-32)).

### Helpers

Expand Down

0 comments on commit 2931f8f

Please sign in to comment.