Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update metrics to use Enum #5657

Closed
SkafteNicki opened this issue Jan 26, 2021 · 4 comments
Closed

Update metrics to use Enum #5657

SkafteNicki opened this issue Jan 26, 2021 · 4 comments
Labels
feature Is an improvement or enhancement good first issue Good for newcomers help wanted Open to be worked on
Milestone

Comments

@SkafteNicki
Copy link
Member

🚀 Feature

Motivation

Update metrics package to use Enum where it makes sense. For example:
https://github.com/PyTorchLightning/pytorch-lightning/blob/f78223041283450c98fb34bb23bea6706247d937/pytorch_lightning/metrics/classification/helpers.py#L79-L87
the case (data type) could be made an enum type

class DataType(LightningEnum):
    BINARY = 'binary'
    MULTILABEL = 'multi-label'
    MULTICLASS = 'multi-class'
    MDMC = 'multi-dim multi-class'

Pitch

Alternatives

Additional context

@Borda

@SkafteNicki SkafteNicki added feature Is an improvement or enhancement help wanted Open to be worked on good first issue Good for newcomers Metrics labels Jan 26, 2021
@SkafteNicki SkafteNicki added this to the 1.2 milestone Jan 26, 2021
@yuntai
Copy link
Contributor

yuntai commented Jan 27, 2021

I could give it a shot.

something like this
_check_shape_and_type_consistency(preds: torch.Tensor, target: torch.Tensor) -> Tuple[DataType, int]
& typify all string values to DataType in the metric package?
can you point to a file where DataType should be defined?

@SkafteNicki
Copy link
Member Author

The DataType is propably unique to classification problems and I would therefore put it in pytorch-lightning/pytorch_lightning/metrics/classification/helpers.py file.

@yuntai
Copy link
Contributor

yuntai commented Jan 27, 2021

fine with

class DataType(LightningEnum):
    BINARY = 'binary'
    MULTILABEL = 'multilabel'
    MULTICLASS = 'multiclass'
    MDMC = 'mdmc'

?

Otherwise
DataType.from_str('multi-label') == None
DataType.MULTILABEL == 'multi-label'
DataType.MULTILABEL != 'multilabel'

yuntai added a commit to yuntai/pytorch-lightning that referenced this issue Jan 28, 2021
- change data type string to DataType enum
- add enum equality test
- update test_inputs.py to use the defined enum type
yuntai added a commit to yuntai/pytorch-lightning that referenced this issue Jan 28, 2021
- change data type string to DataType enum
- add enum equality test
- update test_inputs.py to use the defined enum type

update doc
Borda pushed a commit to yuntai/pytorch-lightning that referenced this issue Jan 28, 2021
- change data type string to DataType enum
- add enum equality test
- update test_inputs.py to use the defined enum type

update doc
yuntai added a commit to yuntai/pytorch-lightning that referenced this issue Jan 28, 2021
yuntai added a commit to yuntai/pytorch-lightning that referenced this issue Jan 28, 2021
* remove AverageMethods enum from functional/auroc.py
yuntai added a commit to yuntai/pytorch-lightning that referenced this issue Jan 28, 2021
* remove AverageMethods enum from functional/auroc.py
yuntai added a commit to yuntai/pytorch-lightning that referenced this issue Jan 28, 2021
* remove AverageMethods enum from functional/auroc.py
yuntai added a commit to yuntai/pytorch-lightning that referenced this issue Jan 28, 2021
* remove AverageMethods enum from functional/auroc.py
yuntai added a commit to yuntai/pytorch-lightning that referenced this issue Jan 29, 2021
- Add DataType, AverageMethod and MDMCAverageMethod
yuntai added a commit to yuntai/pytorch-lightning that referenced this issue Jan 29, 2021
- Add DataType, AverageMethod and MDMCAverageMethod
yuntai added a commit to yuntai/pytorch-lightning that referenced this issue Jan 31, 2021
- Add DataType, AverageMethod and MDMCAverageMethod
yuntai added a commit to yuntai/pytorch-lightning that referenced this issue Jan 31, 2021
- Add DataType, AverageMethod and MDMCAverageMethod
@SkafteNicki
Copy link
Member Author

Closed by PR #5689

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Is an improvement or enhancement good first issue Good for newcomers help wanted Open to be worked on
Projects
None yet
Development

No branches or pull requests

2 participants