Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

training accuracy reported as 0.0 with CUB hierarchical label for the classification #2579

Closed
yunchu opened this issue Oct 26, 2023 · 3 comments · Fixed by #2626
Closed

training accuracy reported as 0.0 with CUB hierarchical label for the classification #2579

yunchu opened this issue Oct 26, 2023 · 3 comments · Fixed by #2626
Assignees
Labels
BUG Something isn't working

Comments

@yunchu
Copy link
Contributor

yunchu commented Oct 26, 2023

Describe the bug
Training accuracy reported as 0.0 with CUB hierarchical label for the classification
here is the part of regression_config.json for this testing
image

Steps to Reproduce

  1. run classification regression TC
$ CI_DATA_ROOT=/home/yunchule/data/ci_datasets/ tox -vvv -e tests-cls-py310-pt1 -- tests/regression/classification/test_classification.py::TestRegressionHierarchicalLabelClassification::test_otx_train[Custom_Image_Classification_EfficientNet-V2-S]
  1. check the log
>       assert test_result["passed"] is True, test_result["log"]
E       AssertionError: [EfficientNet-V2-S] Performance: (0.0) < Criteria: (0.5418), threshold: 0.1.
E       assert False is True

Environment:

  • OS: Ubuntu 20.04
  • Framework version: Pytorch
  • Python version: 3.10
  • OpenVINO version: 2023.0
  • CUDA/cuDNN version: 11.7
  • GPU model and memory: 3090/24G
@sungmanc sungmanc self-assigned this Oct 27, 2023
@sungmanc sungmanc added BUG Something isn't working RELEASE and removed RELEASE labels Oct 27, 2023
@wonjuleee wonjuleee assigned wonjuleee and unassigned sungmanc Nov 2, 2023
@wonjuleee
Copy link
Contributor

The problem was because of the hierarchical classification dataset generation from the original CUB data. Since CUB categories are composed of proper noun of birds, it is not appropriate to split the category names into hierarchies.

@yunchu
Copy link
Contributor Author

yunchu commented Nov 2, 2023

Thanks for your update @wonjuleee
I will close this issue.

@goodsong81
Copy link
Contributor

Found root cause, and fixed in #2626

@goodsong81 goodsong81 reopened this Nov 10, 2023
@goodsong81 goodsong81 self-assigned this Nov 10, 2023
@goodsong81 goodsong81 linked a pull request Nov 10, 2023 that will close this issue
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
BUG Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants