E2E Coverage Explanation

[ktun95]

The Coverage Reports

The coverage reports provide percentages of the code that is executed during the end-to-end tests compared to the code that is "loaded".

ex: 656 lines of code EXECUTED / 1004 lines of code LOADED = 65.33% Lines covered

As playwright loads pages and interacts with the application, lines of code are executed.

Lines of code loaded in this case means all the code in the dicty-frontpage's /src folder that is imported from the entrypoint file, src/index.ts. However, what is not included is the code from any of our local packages (e.g. @dictybase/editor). I have looked into configuration options for vite-plugin-istanbul but have not figured out how to include the packages in the coverage collected.

The two attached coverage reports measure the % Coverage from with different number of tests

INITIAL: 19 TESTS [51.71% statements, 74.5% branches, 24.37% functions, 57.26% lines]

[firefox] › contentPage.spec.ts:62:9 › Dicty Art page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Gallery page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Learn About Dicty page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Teaching Protocols page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Useful Links page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Techniques page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Anatomy Ontology page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Codon Bias Table page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Nomenclature Guidelines page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Phenotyping page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Axenic Strain History page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Cite Us page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Dicty Annual Conferences page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Dicty Email Forum page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Dicty Labs page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › History page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Jobs page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Community Annotations page is properly rendered
[firefox] › front.spec.ts:3:5 › has title

AFTER: 28 TESTS [62.13% statements, 76.47% branches, 39.92% functions, 65.79% lines]

[firefox] › contentPage.spec.ts:62:9 › Dicty Art page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Gallery page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Learn About Dicty page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Teaching Protocols page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Useful Links page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Techniques page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Anatomy Ontology page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Codon Bias Table page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Nomenclature Guidelines page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Phenotyping page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Axenic Strain History page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Cite Us page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Dicty Annual Conferences page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Dicty Email Forum page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Dicty Labs page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › History page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Jobs page is properly rendered
[firefox] › contentPage.spec.ts:62:9 › Community Annotations page is properly rendered
+ [firefox] › contentPage.spec.ts:78:5 › The base route redirects unauthorized users to ./show
+ [firefox] › downloads.spec.ts:3:5 › Download's page shows expected categories
[firefox] › front.spec.ts:3:5 › has title
+ [firefox] › front.spec.ts:8:5 › Dicty News component is displayed
+ [firefox] › front.spec.ts:15:5 › Dicty News renders link to /news
+ [firefox] › front.spec.ts:21:5 › Latest Papers renders link to /papers
+ [firefox] › news.spec.ts:3:5 › The base route redirects unauthorized users to ./show
+ [firefox] › news.spec.ts:10:5 › Renders News Heading
+ [firefox] › news.spec.ts:17:5 › Renders at least 1 news item that links to a single news page
+ [firefox] › papers.spec.ts:3:5 › Renders Latest Publications heading

Codecov merged reports

When the unit tests and E2E tests are run by github-actions, both reports are uploaded to Codecov are it seems they are merged:

No setup required. Automatic report merging for all CI and languages into a single report.
Suggest Edits

A common practice in the modern developer's workflow is to test multiple build systems, split up tests in different containers, and group tests based on test focus (unittests vs integration tests). Codecov supports all these setups. Therefore, no setup is required.
(https://docs.codecov.com/docs/merging-reports)

I can't find a more in-depth explanation, but what I take this to mean is:

UT = Unit Tests
IT = Integration Tests

UT + IT lines executed / UT + IT lines loaded

Where any lines that both UT and IT executed or loaded are counted only once.

There appears to be a way to separate Unit Test coverage data and Integration Test coverage data on Codecov:

Components allow you to isolate and categorize coverage data from your project with virtual filters

(https://docs.codecov.com/docs/components)

But I have not yet looked into it enough to say for sure what can be done with it.