Reduce test flakiness #202
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… in a test EDDTestDataset.get{id}() request wait for file copies/data caching to finish instead of throwing an error due to no files (or using whatever subset of files were copied). There's also a fix for metadata in EDDTable where the max value was incorrect in some circumstances when loading a dataset without a cache.
In some error cases table writers were not getting closed. This possible leak of file handles is fixed.
…A/erddap into ReduceTestFlakiness
…t due to external server)
…A/erddap into ReduceTestFlakiness
…le will come later).
srstsavage
pushed a commit
to srstsavage/erddap
that referenced
this pull request
Sep 29, 2024
* Make tests less flaky. The biggest change for this is having datasets in a test EDDTestDataset.get{id}() request wait for file copies/data caching to finish instead of throwing an error due to no files (or using whatever subset of files were copied). There's also a fix for metadata in EDDTable where the max value was incorrect in some circumstances when loading a dataset without a cache.
In some error cases table writers were not getting closed. This possible leak of file handles is fixed.
* Fix zarr files
* Format code, set SAX parser to true for tests.
* Have tests handle differences on different machines better.
* fix check for testPmelTaoAirt
* Deflake intetgration tests on linux (and turn off the saveopendap test due to external server)
* Clean up changes relating to testing XInclude (XInclude testing/example will come later).
---------
Co-authored-by: Chris John <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This reduces flakiness of tests. The majority of fixes are focused on making some tests more lenient, particularly around how results can differ on different machines (mainly Windows vs. Linux which have multiple differences).
There is also a change to force full dataset loading during tests (EDDTestDataset.getdatasetId) to help ensure consistent behavior (and reduce differences between tests with and without a data cache).
There was a difference in how EDDTable.java set max value for datasets with/without cached data. This is fixed now to be consistent.
Zarr file loading logic was improved.
A filehandle leak (in certain circumstances when an error was thrown) was found and fixed.
Type of change
Checklist before requesting a review