Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix checkMatrix usage in some tests #4732

Merged
merged 7 commits into from
Sep 19, 2023

Conversation

camelto2
Copy link
Contributor

Please review the developer documentation
on the wiki of this project that contains help and requirements.

Proposed changes

While trying to get a new feature tested that I'm working on, I noticed that a few unit tests have incorrect usage of the "checkMatrix". Those tests were just calling checkMatrix(matA, matB) and not getting the result and checking to see if the matrices agree or not.

This means that some of those features could actually be failing and we have been missing it since the checks weren't correct. I think I caught all instances here.

What type(s) of changes does this code introduce?

Delete the items that do not apply

  • Bugfix
  • Testing changes (e.g. new unit/integration/performance tests)

Does this introduce a breaking change?

  • No

What systems has this change been tested on?

macOS

Checklist

Update the following with a yes where the items apply. If you're unsure about any of them, don't hesitate to ask. This is
simply a reminder of what we are going to look for before merging your code.

  • Yes. This PR is up to date with current the current state of 'develop'
  • Yes. Code added or changed in the PR has been clang-formatted
  • Yes. This PR adds tests to cover any new code, or to catch a bug that is being fixed (Kind of, it is fixing existing tests)
  • No. Documentation has been added (if appropriate)

@camelto2 camelto2 requested a review from ye-luo September 15, 2023 23:11
@camelto2
Copy link
Contributor Author

camelto2 commented Sep 15, 2023

Note that there are failing tests only from the files where I fixed the checkMatrix call to actually see if the matrices agree or not. Seems like there could be lingering bugs, but haven't looked too closely yet

@ye-luo
Copy link
Contributor

ye-luo commented Sep 17, 2023

Test this please

@ye-luo
Copy link
Contributor

ye-luo commented Sep 18, 2023

@camelto2 remaining failure are from the added code?

@camelto2
Copy link
Contributor Author

@camelto2 remaining failure are from the added code?

It looks like the CGG11-NoMPI-Real-Mixed is failing with what looks like a tolerance issue, e.g.

  CHECKED_ELSE( check.result )
with expansion:
  false

/__w/qmcpack/qmcpack/src/QMCWaveFunctions/tests/test_spline_applyrotation.cpp:672: FAILED:
explicitly with message:
  checkMatrix found bad element at 1:3  2.85123 != 2.8513

but the GCC11-NoMPI-Complex-Mixed looks like something could actually be wrong

   CHECKED_ELSE( check.result )
with expansion:
  false

/__w/qmcpack/qmcpack/src/QMCWaveFunctions/tests/test_spline_applyrotation.cpp:370: FAILED:
explicitly with message:
  checkMatrix found bad element at 1:1  (-0.0107095,0.377086) != (-0.0107111,0.
  377104)

-------------------------------------------------------------------------------
Spline applyRotation complex rotation
-------------------------------------------------------------------------------
/__w/qmcpack/qmcpack/src/QMCWaveFunctions/tests/test_spline_applyrotation.cpp:678
...............................................................................

/__w/qmcpack/qmcpack/src/QMCWaveFunctions/tests/test_spline_applyrotation.cpp:815: FAILED:
  CHECKED_ELSE( check.result )
with expansion:
  false

/__w/qmcpack/qmcpack/src/QMCWaveFunctions/tests/test_spline_applyrotation.cpp:815: FAILED:
explicitly with message:
  checkMatrix found bad element at 0:2  (-3.64174e-06,7.43674e-06) != (4.
  06802e-06,-1.0252e-05)

@camelto2
Copy link
Contributor Author

Looks like it would still be tolerance actually. I think to add a tolerance argument into checkMatrix and that would solve the mixed precision issue

@camelto2
Copy link
Contributor Author

Ok, so the failures were due to the fact that the checkMatrix function uses Catch2 Approx, and was always using the default epsilon. For mixed precision applyRotation in the splines, the laplacian was failing tests due to that default tolerance.

I added some functionality which allows us to pass in a desired tolerance for the element by element comparison in checkMatrix. Also added some testing of that inside the unit test for checkMatrix.

CheckMatrixResult checkMatrix(M1& a_mat,
M2& b_mat,
const bool check_all = false,
const double eps = std::numeric_limits<float>::epsilon() * 100)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't feel right with this hidden choice. Could you use std::optional and default to std::nullopt_t

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think I have this working with optional.

@ye-luo ye-luo enabled auto-merge September 19, 2023 01:11
@ye-luo
Copy link
Contributor

ye-luo commented Sep 19, 2023

Test this please

@ye-luo ye-luo merged commit 2c68369 into QMCPACK:develop Sep 19, 2023
23 checks passed
@camelto2 camelto2 deleted the fix_checkMatrix_usage branch September 19, 2023 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants