Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

🎨 On the road to 0.3.0: Adding shift evaluation & more #117

Merged
merged 73 commits into from
Oct 22, 2024
Merged

🎨 On the road to 0.3.0: Adding shift evaluation & more #117

merged 73 commits into from
Oct 22, 2024

Conversation

alafage
Copy link
Contributor

@alafage alafage commented Oct 22, 2024 •
edited by o-laurent
Loading

  • ✨ Add evaluation of the metrics on a shifted dataset using the new --eval_shift argument
  • ✨ Add a corrupted dataset wrapper
  • ✨ Add new Packed layers
  • ✨ Add the focal loss
  • 👕 Plot segmentation results in the SegmentationRoutine logger.
  • 👕 Rework reliability diagrams for Calibration Error plot utils.
  • ➖ Make some dependencies optional
  • 🐛 Fix corruption transforms

o-laurent and others added 30 commits October 3, 2024 16:05
@o-laurent
🐛 Always use TUTrainer
b61dcd1
@o-laurent
💻 Sync all epoch level metric logs
f91c3d9
@o-laurent
👕 AURC & AUGRC in %
5888b75
@alafage
🐛 Add correct classes of CamVid
034bb41
@o-laurent
👕 Improve datamodules
031c041
@o-laurent
🐛 Fix Camvid img size
47eaac3
@o-laurent
✨ Add CamVid class grouping & deeplab config
372a702
@o-laurent
👕 Finish standardizing WideResNet defaults
ab61752
@o-laurent
✨ Add Focal Loss
b07a954
@o-laurent
🐎 Avoid computing some metrics twice
34b79d6
@o-laurent
✨ Show RMSE at the end of the validation in regression
12db3a3
@o-laurent
👕 Start transforms by ToTensor
6f4b9dd
@o-laurent
👕 Cutout full torch implementation
b9ab4ea
@o-laurent
✅ Update tests
3469031
@o-laurent
⚡ Update dependencies
4c2b858
@o-laurent
➖ Make glest optional
a96eba9
@o-laurent
➖ Make scipy and cv2 optional
00ca077
@o-laurent
🐛 Small img transforms fixes
3bf8069
@o-laurent
✨ Add shift transforms
58ec315
@o-laurent
➕ Add Wand as optionnal dependency
d25fef0
@o-laurent
🐛 Fix RNGs
75ff336
@alafage
🔨 Add other implementations for PackedLinear
c1c0272
@o-laurent
✨ Start adding --eval-shift
95c1a54
@o-laurent
Merge branch 'dev' of github.com:ENSTA-U2IS-AI/torch-uncertainty into…
74b72c2 
… dev
@o-laurent
🔥 Update TinyImagenet exp
327e092
@o-laurent
👕 Small tutorial change
539cf60
@o-laurent
🐛 Make parsers inherit methods
2110367
@o-laurent
:shirtt: Rename severity to shift severity
9df886a
@o-laurent
🚧 Add that eval shift is implemented in classification only
585a0b1
@o-laurent
👕 Print shift severity
bbd9081
alafage and others added 13 commits October 22, 2024 14:10
@alafage
🐛 Fix tests for the calibration error plot method
063ba49
@o-laurent
✔️ Add corruption tests
1181ab3
@o-laurent
Merge branch 'dev' of github.com:ENSTA-U2IS-AI/torch-uncertainty into…
7a7ddcf 
… dev
@o-laurent
🐛 Fix & add Zoomblur
972a01a
@o-laurent
✔️ Add focal loss tests
8e5f2e8
@o-laurent
✔️ Add eval shift tests
db17969
@o-laurent
Merge branch 'dev' of github.com:ENSTA-U2IS-AI/torch-uncertainty into…
c22c258 
… dev
@o-laurent
✔️ Improve datamodule coverage
ce93a5c
@alafage
🐛 Fix PackedLinear implementation feature
42d2c90 
- Add corresponding tests
@alafage
✔️ Test bias=False in PackedLinear
0026c80
@alafage
Merge branch 'dev' of github.com:ENSTA-U2IS-AI/torch-uncertainty into…
81f3fb7 
… dev
@o-laurent
Merge branch 'dev' of github.com:ENSTA-U2IS-AI/torch-uncertainty into…
48df99f 
… dev
@o-laurent
🐛 Use fake ds for MNIST OOD
4ed1c29
@alafage alafage marked this pull request as ready for review October 22, 2024 13:02
o-laurent
o-laurent reviewed Oct 22, 2024
View reviewed changes
torch_uncertainty/datasets/corrupted.py
@@ -0,0 +1,105 @@
from copy import deepcopy
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are no tests for this right now, I'll make them in post0

@o-laurent o-laurent changed the title 🎨 On the road to 0.3.0 🎨 On the road to 0.3.0: Adding shift evaluation & more Oct 22, 2024
@o-laurent o-laurent self-requested a review October 22, 2024 13:43
@o-laurent o-laurent merged commit e0b7a54 into main Oct 22, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@o-laurent o-laurent o-laurent approved these changes

Assignees

@alafage alafage

@o-laurent o-laurent

Labels
dependencies Pull requests that update a dependency file documentation Improvements or additions to documentation enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

👕 Add segmentation output plots 👕 Improve the calibration plots
2 participants
@alafage @o-laurent