Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure tests are actually run when some of them are selected #2653

Closed
conradoplg opened this issue Aug 20, 2021 · 3 comments
Closed

Make sure tests are actually run when some of them are selected #2653

conradoplg opened this issue Aug 20, 2021 · 3 comments
Assignees
Labels
A-devops Area: Pipelines, CI/CD and Dockerfiles A-infrastructure Area: Infrastructure changes C-enhancement Category: This is an improvement

Comments

@conradoplg
Copy link
Collaborator

conradoplg commented Aug 20, 2021

Motivation

In some cases we select one or a group of tests to be run. For example, in the .yml files to regenerate the cached state, and in #2633 to run some tests with fake activation heights.

However, it's possible that renaming these tests can cause the tests to be no longer run, and we could not notice that (since cargo test will happily return 0 indicating success even if no tests are run).

Come up with a way to detect if that happens and fail. (For example, we could grep the output and check if something has actually run)

Examples

Here are some example tickets:

@conradoplg conradoplg added C-enhancement Category: This is an improvement S-needs-triage Status: A bug report needs triage labels Aug 20, 2021
@teor2345 teor2345 added A-devops Area: Pipelines, CI/CD and Dockerfiles A-infrastructure Area: Infrastructure changes labels Aug 23, 2021
@mpguerra
Copy link
Contributor

@teor2345
Copy link
Contributor

We don't know how to do this easily, we should fix missing tests when they come up.

We could search for "run 0 tests" in the logs.

@teor2345 teor2345 closed this as not planned Won't fix, can't repro, duplicate, stale May 26, 2022
@teor2345
Copy link
Contributor

PR #4710 fixes part of this ticket, but only for a single workflow

@arya2 arya2 self-assigned this Sep 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-devops Area: Pipelines, CI/CD and Dockerfiles A-infrastructure Area: Infrastructure changes C-enhancement Category: This is an improvement
Projects
None yet
Development

No branches or pull requests

5 participants