Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add additional dolfyn functionality #102

Open
wants to merge 15 commits into
base: develop
Choose a base branch
from
Open

Add additional dolfyn functionality #102

wants to merge 15 commits into from

Conversation

zur-quin
Copy link

@zur-quin zur-quin commented Mar 1, 2023

Starting with fillgaps_time() and val_exceeds_thresh() functions as well as some tests/examples for both.

@zur-quin
Add function fillgaps_time() to dolfyn
fa3baae
@zur-quin
More clean functions added
e0f1857 
-fillgaps_depth and val_exceeds_thresh added to clean folder
-formatting changes for fillgaps_time for consistency
-remove numpy from nan_beyond_surface to fix error
-notice that fillgaps_time and fillgaps_depth do the same thing, need to change to interpolate over just time/depth respectively
@zur-quin
Update fillgaps_time and fillgaps_depth
2cd4004 
-update these functions to interpolate over time and depth respectively
-using fillmissing function in matlab now to specify an interpolation dimension
-include maxgap option for both above functions
-update comments for above functions
@zur-quin
Add minimalistic worksheet to use new functions
6939b7a 
New worksheet to demo val_exceeds_thresh and fill_gaps_time functions.
Calls the same functions/uses same file for testing as python tests for these functions.
@zur-quin zur-quin marked this pull request as ready for review March 27, 2023 18:39
zur-quin added 11 commits April 17, 2023 16:07
@zur-quin
Update fillgaps_time
ffd4b59 
-fillgaps_time now checks var's type a little, and also uses correct error syntax
-Note: MATLAB's fillmissing function does not have a default option for MaxGap, so if statement is still used for MaxGap argument
@zur-quin
Update fillgaps_depth
c980a41 
-update fillgaps_depth function to cover discussed changes. now checking if var is from dolfyn structure, using better syntax for throwing errors
-Note: fillmissing function doesn't offer default value for MaxGap in documentation, if statement needed to call correct function from fillgaps_depth
@zur-quin
Update val_exceeds_thresh error syntax
b4d6495 
-val_exceeds_thresh now checks types and throws an error if structure passed to function is not from dolfyn
@zur-quin
Add proper Test file for functions
6b3adfd 
Add tests/Dolfyn_Test_Clean.m to mimic test_clean.py from python version of MHKiT. Currently compares cleaned data to itself in order to pass tests. Next step: Move cleaned data files into repo to compare with tests here.
@zur-quin
Remove test_clean_adcp_example.mlx
001e4d4 
Since actual test file added, test_clean_adcp_example.mlx not needed
@zur-quin
Update test clean with tests for future clean functions
08ec7f4 
Dolfyn_Test_Clean.m updated to have functions for future tests.
Dolfyn_Test_Clean.m now has all same tests as test_clean.py
Commented out sections for "TODO" component/future clean functions to be added
@zur-quin
Add clean data in test_data subdirectory
581f044 
move required cleaned data from python version to test_data folder
start using cleaned data to compare in the tests (instead of comparing to self)
@zur-quin
Update read funcitons in tests
7a8f751
@zur-quin
Fix x* vs x_star fieldname from read_netcdf.m
d5c6b2c 
There was a bug in read_netcd.m where it wasn't reading from the fixed field x_star, and was trying to read the nonexistant x* field.
@zur-quin
WIP add find_surface function
bbac4c8 
Add find_surface function, not quite finished still has a few errors from translating from python.
@zur-quin
Update comments with TODO notes
f052b98 
Cleaned up these files with additional comments and clarifications on items.
@zur-quin
Copy link
Author

Finished Tasks:

  • added .nc files from MHKiT python for cleaning and comparing
  • fillgaps_depth.m function written
  • fillgaps_time.m function written
  • val_exceeds_thresh.m function written
  • Dolfyn_Test_Clean.m mimics test_clean.py with some function calls commented out (if they haven't been written yet), use this as a secondary TODO list to finish this PR

TO-DO:

  • Check the files uploaded are the files we want to use. In python .nc files are used, but many tests in Matlab use the .ad2cp files.
  • (In-progress) find_surface.m function: this function has been started, but still has some errors. See notes in the file for more details.
  • (In-progress) Dolfyn_Test_Clean.compare_structures helper function to compare the structures at the end of tests. This exists at the bottom of the Dolfyn_Test_Clean.m file, but is mostly just copied from the Dolfyn_Test_Rotate.m test and needs adjusting to compare the correct parts of the data.
  • The following are functions called in the test_clean.py that, as far as I'm aware, do not exist yet on the Matlab side. These are called in Dolfyn_Test_Clean.m but have been commented out. There are "TODO" comments in Dolfyn_Test_Clean.m listing these, but the list is reproduced here:
    • GN2002.m (File does not exist yet)
    • clean_fill.m (File does not exist yet)
    • fill_nan_ensemble_mean.m (File does not exist yet)
    • spike_tresh.m (File does not exist yet)
    • range_limit.m (File does not exist yet)
    • medfilt_orient.m (File does not exist yet)

Final notes:

  • As far as I'm aware the completed functions work as intended, but have not been verified.
  • MHKiT-MATLAB\mhkit\dolfyn\tools\create_dataset.m is a good function to be aware of, it creates the structure in Matlab that is attempting to mimic the xarray dataset used in python.
  • The best things to do first are probably the compare_structures function and finish find_surface.m function...then you can run test_clean_downADCP and test_clean_upADCP tests in Dolfyn_Test_Clean.m. After that, pick whichever test function you want to work on in DOlfyn_Test_Clean and finish the required functions in there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@zur-quin