Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

configure_metrics default kwargs causes errors as default with certain metrics #45

Open
X02cinnamondirty opened this issue Feb 26, 2024 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@X02cinnamondirty
Copy link

X02cinnamondirty commented Feb 26, 2024

def configure_metrics(self, metric, metric_kwargs):
    """Configure metrics
    
    Keeping this a function allows for the metric to be reconfigured
    in inherited classes TODO: add support for multiple metrics
    
    Returns:
    ----------
    torchmetrics.Metric:
        metric
    """
    metric_name = DEFAULT_TASK_METRICS[self.task] if metric is None else metric
    metric_kwargs = (
        metric_kwargs
        if metric_kwargs is not None
        else DEFAULT_METRIC_KWARGS[self.task]
    )
    metric = METRIC_REGISTRY[metric_name](
        num_outputs_=self.output_dim, **metric_kwargs
    )
    return metric, metric_kwargs, metric_name

num_outputs may have some fault

@adamklie adamklie added the bug Something isn't working label Mar 4, 2024
@adamklie adamklie self-assigned this Mar 4, 2024
@adamklie
Copy link
Collaborator

adamklie commented Mar 4, 2024

Hi!

Thanks for raising an issue on this. This is a known problem with the way we currently calculate metrics in the SequenceModule class. We are working on a refactor of the LightningModules here (https://github.com/ML4GLand/SeqModels/tree/dev).

For now, you can workaround this by including arguments in the metric_kwargs for the SequenceModule. You can find an example of that here: https://github.com/ML4GLand/use_cases/blob/dev/Basset/train_eugene.ipynb. I'm happy to assist if you have a specific use case that's causing an error.

@adamklie adamklie changed the title _SequenceModule.py configure_metrics default kwargs causes errors as default with certain metrics Mar 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants