Enforces equal or increasing code coverage in PRs

[tmarkley]

Description

• Utilizes Codecov yaml configuration: https://docs.codecov.com/docs/codecov-yaml

The purpose of enforcing increasing code coverage over time is to attempt to improve the quality of changes via testing, reduce the need to request tests in PRs, and stay away from enforcing an ambiguous coverage percentage.

Validated the config:

$ curl -X POST --data-binary @.github/.codecov.yml https://codecov.io/validate
Valid!

{
  "coverage": {
    "status": {
      "project": {
        "default": {
          "target": "auto"
        }
      }
    }
  }
}

Issues Resolved

N/A

Check List

• Commits are signed per the DCO using --signoff

[codecov-commenter]

Codecov Report

Merging #1559 (6a2cb50) into main (cba0764) will increase coverage by 0.00%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1559   +/-   ##
=======================================
  Coverage   68.09%   68.09%           
=======================================
  Files        3072     3072           
  Lines       59032    59032           
  Branches     8928     8928           
=======================================
+ Hits        40198    40200    +2     
+ Misses      16647    16646    -1     
+ Partials     2187     2186    -1     

Impacted Files	Coverage Δ
...ic/application/models/sense_editor/sense_editor.ts	64.88% <0.00%> (+0.88%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cba0764...6a2cb50. Read the comment docs.

[CPTNB]

Do we have a testing doctrine or testing goals or metric targets?

[tmarkley]

Do we have a testing doctrine or testing goals or metric targets?

@CPTNB I think our TESTING.md doc covers some of that. We don't currently have a metric goal for tests; I've seen other repositories in the OpenSearch Project target between 70% and 80%, but I'm hesitant to add a target at this point since we have a lot of issues related to testing that we want to address:

https://github.com/opensearch-project/OpenSearch-Dashboards/issues?q=is%3Aopen+is%3Aissue+label%3Atest%3Aunit%2Ctest%3Aintegration%2Ctest%3Afunctional

[ashwin-pc]

Can we still override it if necessary? Like when we merge the mocha test removal PR or if the diff is minor like -.01%. Some readme update PR's usually have that as the coverage diff

[tmarkley]

Can we still override it if necessary? Like when we merge the mocha test removal PR or if the diff is minor like -.01%. Some readme update PR's usually have that as the coverage diff

@ashwin-pc yes, similar to how we can still merge if one of the Build and test checks fail (e.g. a flaky test fails or the version of chromedriver doesn't match GitHub), we maintainers can still choose to merge if the Codecov check fails and there are two approvers.