Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

140 automated e2e tests #143

Merged
merged 35 commits into from
May 21, 2024
Merged

140 automated e2e tests #143

merged 35 commits into from
May 21, 2024

Conversation

mschwoer
Copy link
Collaborator

@mschwoer mschwoer commented May 16, 2024

First working version of end 2 end tests.
For each test case defined in a yaml file, download the test data, run alphadia, calculate metrics, and push them to neptune.
Can be seen in action here: https://github.com/MannLabs/alphadia/actions/runs/9113110942/job/25053809668

On neptune:
image

Next steps:

  • use custom github runner (follow-up PR)
  • more metrics, neptune dashboard
  • add caching and more metadata to neptune

@mschwoer mschwoer requested a review from GeorgWa May 16, 2024 14:35
@mschwoer mschwoer marked this pull request as ready for review May 16, 2024 14:39
tests/e2e_tests/calc_metrics.py Outdated Show resolved Hide resolved
tests/e2e_tests/calc_metrics.py Outdated Show resolved Hide resolved
"""Calculate metrics."""
df = self._data_store["stat.tsv"]

for col in ["proteins", "precursors", "ms1_accuracy", "fwhm_rt"]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a naming issue on alphaDIAs side. proteins and genes refers to values annotated in the library. pg is the column for proteins after fdr correction and with the correct inference mode.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hm, not sure what you mean .. I read these values off the actual alphaDIA output?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only about the implementation of the metric. AlphaDIA generates the pg column for FDR corrected protein groups. So for counting PGs at 1% FDR you would need to use the pg column.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, I see .. given the metrics are only a placeholder for now, I would leave it like this

tests/e2e_tests/calc_metrics.py Outdated Show resolved Hide resolved
tests/e2e_tests/calc_metrics.py Show resolved Hide resolved
tests/e2e_tests/calc_metrics.py Outdated Show resolved Hide resolved
tests/e2e_tests/e2e_test_cases.yaml Show resolved Hide resolved
CONFIG_SOURCE_PATH = "../../alphadia/constants/default.yaml"


def _download_file(url: str, target_name: str) -> None:
Copy link
Collaborator

@GeorgWa GeorgWa May 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is some overlap with alphadia.testing. I think we could unify this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tests/run_e2e_tests.sh Outdated Show resolved Hide resolved
@GeorgWa GeorgWa merged commit 30567da into development May 21, 2024
5 checks passed
@GeorgWa GeorgWa deleted the 140-automated-e2e-tests branch May 21, 2024 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants