Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor priors #329

Open
wants to merge 3 commits into
base: develop
Choose a base branch
from
Open

Conversation

dweindl
Copy link
Member

@dweindl dweindl commented Dec 4, 2024

Introduces Distribution classes for handling prior distributions, sampling from them, and evaluating negative log-priors (#312). Later on, this can be extended to noise models for measurements and computing loglikelihoods.
This also adds a notebook demonstrating the various prior options which are a common source confusion, and fixes a faulty test.

Closes #311.

👀 notebook: https://petab--329.org.readthedocs.build/projects/libpetab-python/en/329/example/distributions.html

@codecov-commenter
Copy link

codecov-commenter commented Dec 4, 2024

Codecov Report

Attention: Patch coverage is 80.21390% with 37 lines in your changes missing coverage. Please review.

Project coverage is 74.56%. Comparing base (d7f7e3a) to head (047dc46).

Files with missing lines Patch % Lines
petab/v1/distributions.py 83.33% 14 Missing ⚠️
petab/v1/priors.py 84.09% 9 Missing and 5 partials ⚠️
petab/v1/sampling.py 25.00% 8 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #329      +/-   ##
===========================================
+ Coverage    74.51%   74.56%   +0.05%     
===========================================
  Files           53       54       +1     
  Lines         5222     5378     +156     
  Branches       910      923      +13     
===========================================
+ Hits          3891     4010     +119     
- Misses         984     1016      +32     
- Partials       347      352       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dweindl dweindl force-pushed the 311_refactor_priors branch 5 times, most recently from 5313345 to b28e5c1 Compare December 4, 2024 22:38
@dweindl dweindl marked this pull request as ready for review December 4, 2024 22:47
@dweindl dweindl requested review from m-philipps and a team as code owners December 4, 2024 22:48
@dweindl dweindl self-assigned this Dec 4, 2024
@dweindl dweindl force-pushed the 311_refactor_priors branch 4 times, most recently from 721ba31 to d511639 Compare December 7, 2024 10:04
@dweindl dweindl marked this pull request as draft December 8, 2024 11:25
@dweindl dweindl force-pushed the 311_refactor_priors branch from d511639 to 9381d7c Compare December 9, 2024 13:42
@dweindl
Copy link
Member Author

dweindl commented Dec 9, 2024

Requires some clarification related to normalization of the prior density. Related to PEtab-dev/PEtab#402.

EDIT: Needs clarification, but the current implementation matches what was done before.

@dweindl dweindl force-pushed the 311_refactor_priors branch 4 times, most recently from a272255 to 86643ec Compare December 9, 2024 21:43
@dweindl dweindl marked this pull request as ready for review December 9, 2024 21:44
Copy link
Member

@dilpath dilpath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks 👍


"""

_type_to_cls: dict[str, type[Distribution]] = {}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doc/comment

Comment on lines 46 to 54
def __init__(
self,
type_: str,
transformation: str,
parameters: tuple,
bounds: tuple = None,
parameter_scale: bool = False,
):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Support supplying some rng: np.Generator = None and otherwise set self.rng = np.random.default_rng() in __init__?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I'll add that separately. For now, it's handled the same way as before, using the default numpy random state.

Comment on lines 216 to 218
parameter_scale=_parameter_scale,
type_=_type,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why _ both before and after?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type_ to not shadow builtin type(); _type, private, it should not be used explicitly be the user.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then remove from __init__ signature? Or change to be private class attributes?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It required for initialization. Moving it to an attribute is not really helpful. After further refactoring it's gone completely.

@@ -26,84 +26,10 @@ def sample_from_prior(
"""
# unpack info
p_type, p_params, scaling, bounds = prior
prior_cls = Distribution._type_to_cls[p_type]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is _type_to_cls private? Could remove the _ prefix otherwise.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I'd like to decrease the API surface. Unless there is a clear use case, I'd consider things private to simplify refactoring. Here, I don't see a clear use case yet.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, my question was rather because _type_to_cls is used outside of Distribution here, but fine to use _* for things that can be used anywhere in the library but not "outside" the library, e.g. in user code.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's completely gone by now anyways, will update in a sec.


# Kolmogorov-Smirnov test to check if the sample is drawn from the CDF
_, p = kstest(sample, cdf)
assert p > 0.05, (p, distribution)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be higher than 0.05? e.g. p > 0.5 should also always work for 10000 samples, just guessing. Anyway, fine as is.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With p > 0.5 this will fail frequently. Even with p > 0.05 it fails often enough so that I set a seed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's surprising! I would have guessed that 10,000 samples from a "simple" distribution should be enough to say "these distributions look similar"... anyway fine 👍

@dweindl dweindl force-pushed the 311_refactor_priors branch 3 times, most recently from a6429c8 to 9013648 Compare December 10, 2024 08:25
Introduce `Distribution` classes for handling prior distributions and sampling from them.
Later on this can be extended to noise models for measurements.
@dweindl dweindl force-pushed the 311_refactor_priors branch from 9013648 to e25785f Compare December 10, 2024 08:26
@dweindl dweindl force-pushed the 311_refactor_priors branch 2 times, most recently from 0b704ba to cc6a7e7 Compare December 10, 2024 17:30
@dweindl dweindl requested a review from dilpath December 10, 2024 17:31
@dweindl dweindl force-pushed the 311_refactor_priors branch from cc6a7e7 to 047dc46 Compare December 10, 2024 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants