Add Delft3D Module #124
Merged
Add Delft3D Module #124
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Adressing issue #65 tidal ebb and flood plots for Matlab * adding the project file * updating the function descriptions * WPTO for Matlab (#61) * wpto playground * format changes * upload temporary test script * wpto extract with REST API * fix directional wave spectrum bug * upload hindcast example * auto format time_index into datetime array * add tests for hindcast * remove temp file * bug fix for available buoy data vs hindcast * upload data * moving test files into hindcast folder * updating toolbox paths * updating hindcast example * Setup CI, v16.2, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.3, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.4, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.5, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.6, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.7, list installed toolboxes and addons * Setup CI, v17, uninstall all pre-installed toolboxes * Setup CI, v17.1, uninstall all pre-installed toolboxes * Setup CI, v17.2, toggle off all pre-installed toolboxes * Setup CI, v17.3, toggle off all pre-installed toolboxes * Setup CI, v17.4, toggle off all pre-installed toolboxes * Setup CI, v17.5, toggle off all pre-installed toolboxes * Setup CI, v17.6, toggle off all pre-installed toolboxes * Setup CI, v17.7, toggle off all pre-installed toolboxes * Setup CI, v17.8, toggle off all pre-installed toolboxes * Setup CI, v17.9, toggle off all pre-installed toolboxes * Setup CI, v17.10, list installed addons and toolboxes * Setup CI, v17.11, start uninstalling toolboxes * Setup CI, v17.12, start uninstalling toolboxes * Setup CI, v17.13, start uninstalling toolboxes * Setup CI, v17.14, start uninstalling toolboxes * Setup CI, v17.15, start uninstalling toolboxes * Setup CI, v17.16, start uninstalling toolboxes * Setup CI, v17.17, start uninstalling toolboxes * Setup CI, v17.18, start uninstalling toolboxes * Setup CI, v17.19, start uninstalling toolboxes * Setup CI, v17.20, start uninstalling toolboxes * Setup CI, v17.21, start uninstalling toolboxes * Setup CI, v17.22, start uninstalling toolboxes * Setup CI, v17.23, start uninstalling toolboxes * Setup CI, v17.24, start uninstalling toolboxes * Setup CI, v17.25, start uninstalling toolboxes * Setup CI, v17.26, start uninstalling toolboxes * Setup CI, v17.27, start uninstalling toolboxes * Setup CI, v17.28, start uninstalling toolboxes * Setup CI, v17.29, start uninstalling toolboxes * Setup CI, v17.30, start uninstalling toolboxes * Setup CI, v17.31, start uninstalling toolboxes * Setup CI, v17.32, start uninstalling toolboxes * Setup CI, v17.33, start uninstalling toolboxes * Setup CI, v17.34, start uninstalling toolboxes * Setup CI, v17.35, start uninstalling toolboxes * Setup CI, v18, simplify to most basic test * Setup CI, v19, install mhkit toolbox * Setup CI, v20, run a single test that uses mhkit * Setup CI, v21, run a single different test that uses mhkit * Setup CI, v21.1, run a single different test that uses mhkit * Setup CI, v21.2, run a single different test that uses mhkit * Add function to reload python in matlab after python code changes * Fix paths to python usgs and noaa functions * Update pierson_moskowitz_spectrum wrapper with additional Hs parameter * Update create_spectra with additional Hs parameter * Update tests fixing mostly broken relative file paths Co-authored-by: rpauly18 <[email protected]> Co-authored-by: Matthew Boyd <[email protected]> * Adding tests to account for the new plot functions plot_tidal_phase_exceedance and plot_tidal_phase_probability * Addressing issue #65 tidal ebb and flood plots for Matlab (#73) * Setup CI, v16.2, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.3, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.4, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.5, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.6, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.7, list installed toolboxes and addons * Setup CI, v17, uninstall all pre-installed toolboxes * Setup CI, v17.1, uninstall all pre-installed toolboxes * Setup CI, v17.2, toggle off all pre-installed toolboxes * Setup CI, v17.3, toggle off all pre-installed toolboxes * Setup CI, v17.4, toggle off all pre-installed toolboxes * Setup CI, v17.5, toggle off all pre-installed toolboxes * Setup CI, v17.6, toggle off all pre-installed toolboxes * Setup CI, v17.7, toggle off all pre-installed toolboxes * Setup CI, v17.8, toggle off all pre-installed toolboxes * Setup CI, v17.9, toggle off all pre-installed toolboxes * Setup CI, v17.10, list installed addons and toolboxes * Setup CI, v17.11, start uninstalling toolboxes * Setup CI, v17.12, start uninstalling toolboxes * Setup CI, v17.13, start uninstalling toolboxes * Setup CI, v17.14, start uninstalling toolboxes * Setup CI, v17.15, start uninstalling toolboxes * Setup CI, v17.16, start uninstalling toolboxes * Setup CI, v17.17, start uninstalling toolboxes * Setup CI, v17.18, start uninstalling toolboxes * Setup CI, v17.19, start uninstalling toolboxes * Setup CI, v17.20, start uninstalling toolboxes * Setup CI, v17.21, start uninstalling toolboxes * Setup CI, v17.22, start uninstalling toolboxes * Setup CI, v17.23, start uninstalling toolboxes * Setup CI, v17.24, start uninstalling toolboxes * Setup CI, v17.25, start uninstalling toolboxes * Setup CI, v17.26, start uninstalling toolboxes * Setup CI, v17.27, start uninstalling toolboxes * Setup CI, v17.28, start uninstalling toolboxes * Setup CI, v17.29, start uninstalling toolboxes * Setup CI, v17.30, start uninstalling toolboxes * Setup CI, v17.31, start uninstalling toolboxes * Setup CI, v17.32, start uninstalling toolboxes * Setup CI, v17.33, start uninstalling toolboxes * Setup CI, v17.34, start uninstalling toolboxes * Setup CI, v17.35, start uninstalling toolboxes * Setup CI, v18, simplify to most basic test * Setup CI, v19, install mhkit toolbox * Setup CI, v20, run a single test that uses mhkit * Setup CI, v21, run a single different test that uses mhkit * Setup CI, v21.1, run a single different test that uses mhkit * Setup CI, v21.2, run a single different test that uses mhkit * Add function to reload python in matlab after python code changes * Fix paths to python usgs and noaa functions * Update pierson_moskowitz_spectrum wrapper with additional Hs parameter * Update create_spectra with additional Hs parameter * Update tests fixing mostly broken relative file paths * Adressing issue #65 tidal ebb and flood plots for Matlab * adding the project file * updating the function descriptions * Merge QC_Test into master (#63) * Add QC_Test.m draft * check incriment working * Update qctest file * test_check_timestep passes * test_check_delta passes * cleanup Co-authored-by: kbrode22 <[email protected]> Co-authored-by: rpauly18 <[email protected]> Co-authored-by: Bhaskar <[email protected]> * Adding tests to account for the new plot functions plot_tidal_phase_exceedance and plot_tidal_phase_probability * Bug fix io request data (#78) * Rewrite usgs data request with no python calls to fix ssl error * Rewrite noaa data request with no python calls to fix ssl error, WIP * Split noaa data calls into max day periods * Rewrite ndbc available data request with no python calls to fix ssl error * Rewrite ndbc data request with no python calls to fix ssl error, WIP * Rewrite ndbc data request with no python calls to fix ssl error, include stdmet option * Handle errors from noaa data query * Update example with noaa data query after change to options argument * Check parameter input types to request_usgs_data.m * minor edits * Add proxy instructions and remove associated deprecated code * Handle more NDBC file formats and corruptions * Filter the NDBC available data by buoy * Resolve mostly relative file path issues in associated live scripts * Add automatic retries to usgs, noaa and ndbc data queries Co-authored-by: rpauly18 <[email protected]> * Move CI Unit Tests to GitHub Actions (#80) * Speed up reading of dataframe datetimes * Add datetime_index_to_ordinal function to pandas_dataframe module to convert the datetime index of dataframes to MATLAB ordinal values * Convert all methods using cell arrays to get the datetimes index to use datetime_index_to_ordinal function * Fix test assert to check for POSIX time rather than string * Skip tests of environmental_contour function which is broken * Run CI tests with all installed from source and OutOfProcess execution mode * Rename run_tests to tests and delete main * Give the workflow a more badge friendly name and add to README * Minor changes * Delete TravisCI build * Remove unused import * Remove duplicated code Co-authored-by: Matthew Boyd <[email protected]> Co-authored-by: McVey <[email protected]> Co-authored-by: rpauly18 <[email protected]> Co-authored-by: Parangat Bhaskar <[email protected]> Co-authored-by: kbrode22 <[email protected]> Co-authored-by: Bhaskar <[email protected]> Co-authored-by: Matthew Boyd <[email protected]> Co-authored-by: Mathew Topper <[email protected]> * Addressing issue #59 magnitude and phase function for Matlab (#76) * Addressing issue #59 magnitude and phase function for Matlab * Adding tests for magnitude_phase function Co-authored-by: McVey <[email protected]> Co-authored-by: rpauly18 <[email protected]> * Revert "Addressing issue #65 tidal ebb and flood plots for Matlab (#73)" (#84) This reverts commit 02e344022f85563ac9db4fe10477ba1bf48474f2. * Revert "Revert "Addressing issue #65 tidal ebb and flood plots for Matlab (#73)" (#84)" (#85) This reverts commit 39f11dcee6b083df8c3260746c4b7fea9776387a. * General updates (#71) * Setup CI, v16.2, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.3, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.4, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.5, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.6, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.7, list installed toolboxes and addons * Setup CI, v17, uninstall all pre-installed toolboxes * Setup CI, v17.1, uninstall all pre-installed toolboxes * Setup CI, v17.2, toggle off all pre-installed toolboxes * Setup CI, v17.3, toggle off all pre-installed toolboxes * Setup CI, v17.4, toggle off all pre-installed toolboxes * Setup CI, v17.5, toggle off all pre-installed toolboxes * Setup CI, v17.6, toggle off all pre-installed toolboxes * Setup CI, v17.7, toggle off all pre-installed toolboxes * Setup CI, v17.8, toggle off all pre-installed toolboxes * Setup CI, v17.9, toggle off all pre-installed toolboxes * Setup CI, v17.10, list installed addons and toolboxes * Setup CI, v17.11, start uninstalling toolboxes * Setup CI, v17.12, start uninstalling toolboxes * Setup CI, v17.13, start uninstalling toolboxes * Setup CI, v17.14, start uninstalling toolboxes * Setup CI, v17.15, start uninstalling toolboxes * Setup CI, v17.16, start uninstalling toolboxes * Setup CI, v17.17, start uninstalling toolboxes * Setup CI, v17.18, start uninstalling toolboxes * Setup CI, v17.19, start uninstalling toolboxes * Setup CI, v17.20, start uninstalling toolboxes * Setup CI, v17.21, start uninstalling toolboxes * Setup CI, v17.22, start uninstalling toolboxes * Setup CI, v17.23, start uninstalling toolboxes * Setup CI, v17.24, start uninstalling toolboxes * Setup CI, v17.25, start uninstalling toolboxes * Setup CI, v17.26, start uninstalling toolboxes * Setup CI, v17.27, start uninstalling toolboxes * Setup CI, v17.28, start uninstalling toolboxes * Setup CI, v17.29, start uninstalling toolboxes * Setup CI, v17.30, start uninstalling toolboxes * Setup CI, v17.31, start uninstalling toolboxes * Setup CI, v17.32, start uninstalling toolboxes * Setup CI, v17.33, start uninstalling toolboxes * Setup CI, v17.34, start uninstalling toolboxes * Setup CI, v17.35, start uninstalling toolboxes * updates to spectrum and elevation functions * updating io paths * Setup CI, v18, simplify to most basic test * Setup CI, v19, install mhkit toolbox * Setup CI, v20, run a single test that uses mhkit * Setup CI, v21, run a single different test that uses mhkit * Setup CI, v21.1, run a single different test that uses mhkit * Setup CI, v21.2, run a single different test that uses mhkit * Add function to reload python in matlab after python code changes * Fix paths to python usgs and noaa functions * Update pierson_moskowitz_spectrum wrapper with additional Hs parameter * Update create_spectra with additional Hs parameter * Update tests fixing mostly broken relative file paths * Merge QC_Test into master (#63) * Add QC_Test.m draft * check incriment working * Update qctest file * test_check_timestep passes * test_check_delta passes * cleanup Co-authored-by: kbrode22 <[email protected]> Co-authored-by: rpauly18 <[email protected]> Co-authored-by: Bhaskar <[email protected]> * Bug fix io request data (#78) * Rewrite usgs data request with no python calls to fix ssl error * Rewrite noaa data request with no python calls to fix ssl error, WIP * Split noaa data calls into max day periods * Rewrite ndbc available data request with no python calls to fix ssl error * Rewrite ndbc data request with no python calls to fix ssl error, WIP * Rewrite ndbc data request with no python calls to fix ssl error, include stdmet option * Handle errors from noaa data query * Update example with noaa data query after change to options argument * Check parameter input types to request_usgs_data.m * minor edits * Add proxy instructions and remove associated deprecated code * Handle more NDBC file formats and corruptions * Filter the NDBC available data by buoy * Resolve mostly relative file path issues in associated live scripts * Add automatic retries to usgs, noaa and ndbc data queries Co-authored-by: rpauly18 <[email protected]> * Move CI Unit Tests to GitHub Actions (#80) * Speed up reading of dataframe datetimes * Add datetime_index_to_ordinal function to pandas_dataframe module to convert the datetime index of dataframes to MATLAB ordinal values * Convert all methods using cell arrays to get the datetimes index to use datetime_index_to_ordinal function * Fix test assert to check for POSIX time rather than string * Skip tests of environmental_contour function which is broken * Run CI tests with all installed from source and OutOfProcess execution mode * Rename run_tests to tests and delete main * Give the workflow a more badge friendly name and add to README * Minor changes * Delete TravisCI build * Remove unused import * Remove duplicated code * resolving test issues * Add Windows MATLAB-Actions Runners (#81) * Put all test output in a single folder and hide from git * Test windows runner using InProcess mode Also removed pinning of matplotlib. See https://github.com/MHKiT-Software/MHKiT-MATLAB/pull/80#issuecomment-1036145739 * Add develop branch to on-push and on-pull_request Co-authored-by: Matthew Boyd <[email protected]> * fixing tests * removing trouble test temporairly * restarting tests * updating python calls Co-authored-by: Matthew Boyd <[email protected]> Co-authored-by: Parangat Bhaskar <[email protected]> Co-authored-by: kbrode22 <[email protected]> Co-authored-by: Bhaskar <[email protected]> Co-authored-by: Matthew Boyd <[email protected]> Co-authored-by: Mathew Topper <[email protected]> Co-authored-by: Mathew Topper <[email protected]> * Wave updates (#70) * adding deep flag and wave_length * adding depth_regime * updates * first few tests working * adding final tests * updating toolbox * Setup CI, v16.2, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.3, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.4, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.5, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.6, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.7, list installed toolboxes and addons * Setup CI, v17, uninstall all pre-installed toolboxes * Setup CI, v17.1, uninstall all pre-installed toolboxes * Setup CI, v17.2, toggle off all pre-installed toolboxes * Setup CI, v17.3, toggle off all pre-installed toolboxes * Setup CI, v17.4, toggle off all pre-installed toolboxes * Setup CI, v17.5, toggle off all pre-installed toolboxes * Setup CI, v17.6, toggle off all pre-installed toolboxes * Setup CI, v17.7, toggle off all pre-installed toolboxes * Setup CI, v17.8, toggle off all pre-installed toolboxes * Setup CI, v17.9, toggle off all pre-installed toolboxes * Setup CI, v17.10, list installed addons and toolboxes * Setup CI, v17.11, start uninstalling toolboxes * Setup CI, v17.12, start uninstalling toolboxes * Setup CI, v17.13, start uninstalling toolboxes * Setup CI, v17.14, start uninstalling toolboxes * Setup CI, v17.15, start uninstalling toolboxes * Setup CI, v17.16, start uninstalling toolboxes * Setup CI, v17.17, start uninstalling toolboxes * Setup CI, v17.18, start uninstalling toolboxes * Setup CI, v17.19, start uninstalling toolboxes * Setup CI, v17.20, start uninstalling toolboxes * Setup CI, v17.21, start uninstalling toolboxes * Setup CI, v17.22, start uninstalling toolboxes * Setup CI, v17.23, start uninstalling toolboxes * Setup CI, v17.24, start uninstalling toolboxes * Setup CI, v17.25, start uninstalling toolboxes * Setup CI, v17.26, start uninstalling toolboxes * Setup CI, v17.27, start uninstalling toolboxes * Setup CI, v17.28, start uninstalling toolboxes * Setup CI, v17.29, start uninstalling toolboxes * Setup CI, v17.30, start uninstalling toolboxes * Setup CI, v17.31, start uninstalling toolboxes * Setup CI, v17.32, start uninstalling toolboxes * Setup CI, v17.33, start uninstalling toolboxes * Setup CI, v17.34, start uninstalling toolboxes * Setup CI, v17.35, start uninstalling toolboxes * Setup CI, v18, simplify to most basic test * Setup CI, v19, install mhkit toolbox * Setup CI, v20, run a single test that uses mhkit * Setup CI, v21, run a single different test that uses mhkit * Setup CI, v21.1, run a single different test that uses mhkit * Setup CI, v21.2, run a single different test that uses mhkit * Add function to reload python in matlab after python code changes * Fix paths to python usgs and noaa functions * Update pierson_moskowitz_spectrum wrapper with additional Hs parameter * Update create_spectra with additional Hs parameter * Update tests fixing mostly broken relative file paths * Merge QC_Test into master (#63) * Add QC_Test.m draft * check incriment working * Update qctest file * test_check_timestep passes * test_check_delta passes * cleanup Co-authored-by: kbrode22 <[email protected]> Co-authored-by: rpauly18 <[email protected]> Co-authored-by: Bhaskar <[email protected]> * Bug fix io request data (#78) * Rewrite usgs data request with no python calls to fix ssl error * Rewrite noaa data request with no python calls to fix ssl error, WIP * Split noaa data calls into max day periods * Rewrite ndbc available data request with no python calls to fix ssl error * Rewrite ndbc data request with no python calls to fix ssl error, WIP * Rewrite ndbc data request with no python calls to fix ssl error, include stdmet option * Handle errors from noaa data query * Update example with noaa data query after change to options argument * Check parameter input types to request_usgs_data.m * minor edits * Add proxy instructions and remove associated deprecated code * Handle more NDBC file formats and corruptions * Filter the NDBC available data by buoy * Resolve mostly relative file path issues in associated live scripts * Add automatic retries to usgs, noaa and ndbc data queries Co-authored-by: rpauly18 <[email protected]> * Move CI Unit Tests to GitHub Actions (#80) * Speed up reading of dataframe datetimes * Add datetime_index_to_ordinal function to pandas_dataframe module to convert the datetime index of dataframes to MATLAB ordinal values * Convert all methods using cell arrays to get the datetimes index to use datetime_index_to_ordinal function * Fix test assert to check for POSIX time rather than string * Skip tests of environmental_contour function which is broken * Run CI tests with all installed from source and OutOfProcess execution mode * Rename run_tests to tests and delete main * Give the workflow a more badge friendly name and add to README * Minor changes * Delete TravisCI build * Remove unused import * Remove duplicated code * Add Windows MATLAB-Actions Runners (#81) * Put all test output in a single folder and hide from git * Test windows runner using InProcess mode Also removed pinning of matplotlib. See https://github.com/MHKiT-Software/MHKiT-MATLAB/pull/80#issuecomment-1036145739 * Add develop branch to on-push and on-pull_request Co-authored-by: Matthew Boyd <[email protected]> * updating test Co-authored-by: Matthew Boyd <[email protected]> Co-authored-by: Parangat Bhaskar <[email protected]> Co-authored-by: kbrode22 <[email protected]> Co-authored-by: Bhaskar <[email protected]> Co-authored-by: Matthew Boyd <[email protected]> Co-authored-by: Mathew Topper <[email protected]> Co-authored-by: Mathew Topper <[email protected]> * CDIP Data IO and Graphics (v2) [develop] (#83) * Setup CI, v16.2, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.3, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.4, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.5, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.6, try to fix HDF5 issue, try fix outlined in Troubleshooting * Setup CI, v16.7, list installed toolboxes and addons * Setup CI, v17, uninstall all pre-installed toolboxes * Setup CI, v17.1, uninstall all pre-installed toolboxes * Setup CI, v17.2, toggle off all pre-installed toolboxes * Setup CI, v17.3, toggle off all pre-installed toolboxes * Setup CI, v17.4, toggle off all pre-installed toolboxes * Setup CI, v17.5, toggle off all pre-installed toolboxes * Setup CI, v17.6, toggle off all pre-installed toolboxes * Setup CI, v17.7, toggle off all pre-installed toolboxes * Setup CI, v17.8, toggle off all pre-installed toolboxes * Setup CI, v17.9, toggle off all pre-installed toolboxes * Setup CI, v17.10, list installed addons and toolboxes * Setup CI, v17.11, start uninstalling toolboxes * Setup CI, v17.12, start uninstalling toolboxes * Setup CI, v17.13, start uninstalling toolboxes * Setup CI, v17.14, start uninstalling toolboxes * Setup CI, v17.15, start uninstalling toolboxes * Setup CI, v17.16, start uninstalling toolboxes * Setup CI, v17.17, start uninstalling toolboxes * Setup CI, v17.18, start uninstalling toolboxes * Setup CI, v17.19, start uninstalling toolboxes * Setup CI, v17.20, start uninstalling toolboxes * Setup CI, v17.21, start uninstalling toolboxes * Setup CI, v17.22, start uninstalling toolboxes * Setup CI, v17.23, start uninstalling toolboxes * Setup CI, v17.24, start uninstalling toolboxes * Setup CI, v17.25, start uninstalling toolboxes * Setup CI, v17.26, start uninstalling toolboxes * Setup CI, v17.27, start uninstalling toolboxes * Setup CI, v17.28, start uninstalling toolboxes * Setup CI, v17.29, start uninstalling toolboxes * Setup CI, v17.30, start uninstalling toolboxes * Setup CI, v17.31, start uninstalling toolboxes * Setup CI, v17.32, start uninstalling toolboxes * Setup CI, v17.33, start uninstalling toolboxes * Setup CI, v17.34, start uninstalling toolboxes * Setup CI, v17.35, start uninstalling toolboxes * Setup CI, v18, simplify to most basic test * Setup CI, v19, install mhkit toolbox * Setup CI, v20, run a single test that uses mhkit * Setup CI, v21, run a single different test that uses mhkit * Setup CI, v21.1, run a single different test that uses mhkit * Setup CI, v21.2, run a single different test that uses mhkit * Add function to reload python in matlab after python code changes * Fix paths to python usgs and noaa functions * Update pierson_moskowitz_spectrum wrapper with additional Hs parameter * Update create_spectra with additional Hs parameter * Update tests fixing mostly broken relative file paths * Merge QC_Test into master (#63) * Add QC_Test.m draft * check incriment working * Update qctest file * test_check_timestep passes * test_check_delta passes * cleanup Co-authored-by: kbrode22 <[email protected]> Co-authored-by: rpauly18 <[email protected]> Co-authored-by: Bhaskar <[email protected]> * Bug fix io request data (#78) * Rewrite usgs data request with no python calls to fix ssl error * Rewrite noaa data request with no python calls to fix ssl error, WIP * Split noaa data calls into max day periods * Rewrite ndbc available data request with no python calls to fix ssl error * Rewrite ndbc data request with no python calls to fix ssl error, WIP * Rewrite ndbc data request with no python calls to fix ssl error, include stdmet option * Handle errors from noaa data query * Update example with noaa data query after change to options argument * Check parameter input types to request_usgs_data.m * minor edits * Add proxy instructions and remove associated deprecated code * Handle more NDBC file formats and corruptions * Filter the NDBC available data by buoy * Resolve mostly relative file path issues in associated live scripts * Add automatic retries to usgs, noaa and ndbc data queries Co-authored-by: rpauly18 <[email protected]> * Move CI Unit Tests to GitHub Actions (#80) * Speed up reading of dataframe datetimes * Add datetime_index_to_ordinal function to pandas_dataframe module to convert the datetime index of dataframes to MATLAB ordinal values * Convert all methods using cell arrays to get the datetimes index to use datetime_index_to_ordinal function * Fix test assert to check for POSIX time rather than string * Skip tests of environmental_contour function which is broken * Run CI tests with all installed from source and OutOfProcess execution mode * Rename run_tests to tests and delete main * Give the workflow a more badge friendly name and add to README * Minor changes * Delete TravisCI build * Remove unused import * Remove duplicated code * Initial commit, WIP * More progress, WIP * Make first complete working version of cdip data collector * Refactor datum_categories function * Rename categories to groups * Alphabetize local functions * Fix a couple issues * Finish adding data tests, minus plot tests * Add autoretry * Refactor autoretry * Add compendium plot and associated tests * Add box plot function, not needing statistical toolbox * Customize boxplot function to look more like Python plot * Refactor compendium plot * Add boxplot and associated test * Handle when no times are given * Make plots not open in new window when in live script * Add example live script for CDIP * Make boxplot look more like Python plot * Make compendium plot look more like Python plot * Change plot time parameters from POSIX to datetime * Remove optional nc object parameter * Fix docstring * updating matlab toolbox * Warn on non-existant parameters * Fix spelling errors * Replace a try-catch block * Add low-level netcdf cdip functions for older matlab versions * Reduce query size of USGS instantaneous data test to avoid query limit * Remove USGS instantaneous data test to stop failed CI runs * Remove Python 3.7 CI jobs as it is no longer supported by hosted runners * Remove boxplot legend dependency on data file Co-authored-by: Parangat Bhaskar <[email protected]> Co-authored-by: kbrode22 <[email protected]> Co-authored-by: rpauly18 <[email protected]> Co-authored-by: Bhaskar <[email protected]> Co-authored-by: Mathew Topper <[email protected]> * updating package * fixing error * fixing errors take 2 * Add MHKit-Python test conditional on test failure * include tests for R2020b * add workflow dispatch for testing * fix typo * workflow_dispatch * add push? * update variable name * update on latest input * switch to ubuntu * finalize workflow * Adressing issue #65 tidal ebb and flood plots for Matlab * adding the project file * updating the function descriptions * Adding tests to account for the new plot functions plot_tidal_phase_exceedance and plot_tidal_phase_probability * Removing python dependencies: read_noaa_json * trying to fix git issue * Removing python dependencies: principal_flow_directions * Removing python dependencies: exceedance_probability. This change will need to be propogated through the entire code as I changed the input class, removed an input variable, and changed the output class * Removing python dependencies: plot_tidal_phase_exceedance * Updating examples and test in accordance with the changes made to remove python dependencies * WDRT functionality: MLER (#96) * initial mler commit * mler_coefficients function * tests and all MLER functions * mler bug fixes * create mler example * updates from review * fixing typo * fixing file path call in tests * conditional run of hindcast tests * Dolfyn Functionality: IO and rotations (#79) * preliminary push for dolfyn IO functionality. This push covers only classic Nortek data format * Completed initial development for signature file read. Also added dolfyn_read() function that will try all three read functions for user * Completed initial development for RDI file read. Outstanding itme: Test functions * initial push for NetCDF read and write functionality for Matlab Dolfyn * Lots of minor and some major fixes from developing testing * Adding changes to conform with most recent Dolfyn-python push * Final stages of testing and minor adjustments. Notably changed all time readings to the dolfyn epoch convention * Finalized version of Dolfyn i/o. No example yet but otherwise ready for review * Adding userdata files that were filtered by gitignore * Adding beginnings of one example and attempts at speeding up performance * Significant speed up for read_signature * Final code optimization for read_signature * bug fix * bug fix in signature and speed optimization for norteck * Speed optimization for read_rdi algorithm * Adding earth2principal rotation, calculate principal heading, and final version of adcp example * ADV Data example for Dolfyn * Adding the orientation tests and accompanying functions * ADV test cases * One bug fix and one change to io_adv_imu test to allow for its use in other tests * Adding rotation test and bug fixes associated with said test * Adding inst2head and beam2inst rotation functions and associated tests * Final ADV rotation tests * Adding two more adp rotation tests and their accompnying files along with bug fixes * final rotation tests * Changing call method of dolfyn_read to be compatable with matlab versions prior to 2021 (ie 'option',value instead of option=value) * First pass at addressing a Matlab bug that didn't allow versions prior to 2021b to read strings in netcdf. To address this I created a new readh5 file and the write_netcdf has changed to auto generate h5 if the version is old. I don't believe that the current test files can be read by older versions though * Updated the h5 schema such that netcdf can now be read by the h5 function so users should get the same answer regardless of matlab version * Further errors associated with Matlab versions before 2021 being addressed * More changes to address reading netcdf with H5 for older versions of Matlab * If you read at netcdf file in h5 and it only has one dimension, that dimension is not specified explicitly from what I can tell. There must be a way to determine what it is but I can not find it * Another change to read_h5 * The set_coords change to dir did not propogate through all the fields the same way that xarray would so I added a manual loop through the fields to make the required changes * Added plotting function for Dolfyn that somewhat resembles the xarray plot functionality. Further testing is needed but the basic concept is in place * Added adp's set_range_offset function * Added the find surface from pressure function for Dolfyn adp * Adding the nan beyond surface function for dolfyn adp * adding the correlation filter for dolfyn adp that removes velocity data below a specified threshold. Additionaly, small change to dolfyn plot tool to accomidate titles with numbers in the title * Interim update for the example * read NetCDF file (#104) * read nc into ds / use finfo.variables * read nc file * read nc file * Rename read_nc_file.m to read_nc_file_var.m read nc file: make file name consistent * read nc file * read netcdf file, test, and test data * read nc file add in doc strings * debug test: check R2020b capatability & why isequaln fails * skip test if r2020; diagnostic output for test * debug test: check why isequaln fails * debug isequaln * debug check isequaln * reduce number of nc files in example_ncfiles/ * update read_nc_file_var * updating mhkit package and matlab version number --------- Co-authored-by: rpauly18 <[email protected]> * typo fix in examples (#103) * add notes * add notes * add notes * add notes * kc edit typo in wave example * kc fix typos in examples/ * kc fix typos in examples * Update unit_tests.yml check if R2021a works for test run --------- Co-authored-by: Chong <[email protected]> * Update power quality flicker assessment (#105) * read nc into ds / use finfo.variables * read nc file * read nc file * Rename read_nc_file.m to read_nc_file_var.m read nc file: make file name consistent * read nc file * read netcdf file, test, and test data * read nc file add in doc strings * debug test: check R2020b capatability & why isequaln fails * skip test if r2020; diagnostic output for test * debug test: check why isequaln fails * debug isequaln * debug check isequaln * calc u0 .5 * power quality update * done calc_idealvoltage * power quality workflow finish * flicker frequency update * add zcd method * update workflow * update calc_alpha0_freq * update flickermeter related * test code finalized * power quality flicker & test * minor changes * PowerQuality: update power example with whole workflow of flicker * add test function for calc_Pst & minor fixes * PQ flicker add workflow & minor changes * PQ flicker minor fix * PQ fix --------- Co-authored-by: Chong <[email protected]> * Workflow: Fix matlab mhkit import * Workflow: Reduce scope for testing * Workflow: Pass python executable to MATLAB * Windows: Fix MATLAB not loading MHKiT-Python * Test Case: Add simple MHKiT Python tests Tests the MHKiT Python installation per: https://mhkit-software.github.io/MHKiT/installation.html#test-mhkit-python-installation * Workflow: Use anaconda for python executable On windows it seems like python is failing to find/use h5py. This uses anaconda to install h5py and netcdf4. * Workflow: Troubleshoot Windows MATLAB not loading python Error: Caught "std::exception" Exception message is: Unable to launch Simple server: Unable to launch C:\Program Files\MATLAB\R2023b\interprocess\bin\win64\pycli\MATLABPyHost.exe because: Peer process exited before transport type handshake. Exit status: 3221226505 ERROR: MATLAB error Exit Status: 0x00000001 exit status 1 Maybe need MATLAB Engine API for python https://www.mathworks.com/help/matlab/matlab_external/install-the-matlab-engine-for-python.html ? MacOS: Brew install hdf5 and netcdf Ubuntu: Set MATLAB pyenv Execution Mode to OutOfProcess Windows: pip failing to install matlab engine, trying only binary flag... # 51 Windows only: Install matlab engine per: https://pypi.org/project/matlabengine/ Ubuntu tests ran to completion! Macos Still failing at pip install mhkit, specifically the hd5py wheel build step. Pip installing numpy 53 54 Actually pip install numpy on macos, only test mhkit python on windows 55 Macos - no change, Following https://stackoverflow.com/questions/66741778/how-to-install-h5py-needed-for-keras-on-macos-with-m1 and exporting HDF5_DIR 56 Macos: Skip brew install entirely Windows: Skip install of matlab engine All: Use OutOfProcess for matlab python 57 Macos: use only-binary flag 58: Uninstall mhkit prior to installing mkhit 59: Explicitly setup python on ubuntu and macos. Seems like macos is using python2. Also explicitly use python3 and pip3 60: Printing mhkit version fails. Only try to import mhkit. 61: MacOS Install mhkit from pip not source 62: Attempt to verify mhkit install 63: try to install from source with pip not pip3 64: Remove only binary flag on pip source install 65: Remove python3/pip3 66: Simplify mhkit version call 67: Macos is working, now ubuntu is not. Outputting pip freeze 68: Install mhkit from pip cloud, not local 69: Use latest versions of actions/checkout@v4 and actions/setup-python@v4 Should fix errors with older github actions that show up at the bottom of the workflow run output 70: Only test on windows, see if this helps with License Manager issues 71: Test MHKIT python import first in MATLAB 72: Correct 71 73: Try not changing execution mode to out of process 74: See if we can import h5py 75: Add output python path to run.m. Matlab does this once per run, and we set in in the last run, so we are seeing matlab use the system python, but we want it to use the conda python version. 76: Output run.m, still seeing same problem as 75 78: Use pyenv to set python version 79: Explicitly set python version, not sure why the same command run through echo outputs a different result. Seeing run.m before using is smart. 80: Echo with newline 81: Now try ExecutionMode OutOfProcess... 82: Got this error again: Unable to launch Simple server: Unable to launch C:\Program Files\MATLAB\R2023b\interprocess\bin\win64\pycli\MATLABPyHost.exe Going to try pip installing matlabengine again... 83: Use matlabengine pip install instructions on github... 84: Use powershell to pass anaconda version to run.m file... 85: Fix powershell cmd... 86: Actually fix powershell cmd 87: Use InProcess for Windows Matlab python, does not work for mhkit, but at least tests run... * Workflow: Separate out Windows unit test into separate workflow * Workflow: Separate out Unix unit test into separate workflow * Workflow: Cleanup Windows unit test workflow * Workflow: Cleanup Unix unit test workflow * Workflow: Delete combined Window/Unix unit test Currently windows always fail due to the way that MATLAB works in the cloud. Previously we separated out unit tests into two files and we are deleting this file. * Workflow: Test multiple versions of python on Unix * Workflow: Specify matlab and python versions * Workflow: Test excludes without python 3.10 string * Workflow: Correct compatible version of python and matlab * Workflow: Use matlab run test action * Workflow: Revert back to run command action The test action has more features for testing, but the we need to specifically set pyenv which is not possible at the moment. * Python: Add function to convert list to series * Tests: Fix projected capture calc to reflect python change This commit: https://github.com/MHKiT-Software/MHKiT-Python/pull/236/commits/13407e80eefcf1661aaed3bd9bfe7c8222be0342 changes the value of this calculation, so here we change the assertion. * Test: Use pm spectrum to match python test Match this test to the equivalent test in MHKiT-Python: https://github.com/MHKiT-Software/MHKiT-Python/blob/2ccc286a65685e5e2f5ab68a467209917f0161d9/mhkit/tests/loads/test_loads.py#L139 * Fix: Convert data to series Currently the timeseries_to_pandas function fails when passed a one dimensional array. This bypasses than function and converts the input list and optional index to a pd.Series * WIP: Tests: Add more python tests * Test: Correctly access structs from exceedance probability * WIP: Fix exceedance probability * WIP: Tests: Ignore failing tests * WIP: Test more recent of versions of h5py in mhkit-python * Revert "WIP: Test more recent of versions of h5py in mhkit-python" More recent versions of h5py do not affect the pip build, it is still failing. This reverts commit b8662928c11372d3a148cd06ec96568c10d996d5. * Workflow: Explicitly install hdf5 and netcdf4 This is an attempt to fix pip installing mhkit. * Workflow: Fix: Correctly use apt * Workflow: Install numpy prior to mhkit build This is an attempt to get hd5py to build, which is now failing with ModuleNotFoundError: No module named 'numpy' * Workflow: Narrow Python and MATLAB versions These are the versions that are currently working. Essentially this is all versions of MATLAB from 2021b and python 3.8-3.10. Note that pip installing mhkit fails on python 3.11. This specifically does not change the exclusions which include provisions for later versions of python and earlier versions of MATLAB. * Workflow: Fix: Use correct netcdf version * Workflow: Try more python versions on windows Attempting to troubleshoot pip install mhkit on python 3.11 * Workflow: Use MHKiT-Python conda installation method Currently the python 3.11 pip install mhkit fails on the netcdf4/hd5py wheel build. This is an attempt to fix that bug, but also build mhkit and python in the same way they are built in the parent repo. * Workflow: Explicitly activate the conda environment shell call Again, following along with the way the mhkit-python workflow executes shell scripts. * Fix: Typo in shell specifier * Workflow: Fix: Test using Python OutOfProcess With the new anaconda setup, calling python from matlab fails with: OMP: Error #15: Initializing libomp.dylib, but found libiomp5.dylib already initialized. OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/ * Workflow: Fix MATLAB/Python 3.11 library path on Ubuntu See: https://www.mathworks.com/matlabcentral/answers/643300-why-do-i-receive-the-error-libstdc-so-6-version-glibcxx_3-4-22-not-found-when-trying-to-star This commit changes the library path, which seems like a better option, but may be difficult to achieve in practice. * Workflow: Continue to Fix MATLAB/Python 3.11 - Install latest c, c++ compilers See: https://www.mathworks.com/matlabcentral/answers/643300-why-do-i-receive-the-error-libstdc-so-6-version-glibcxx_3-4-22-not-found-when-trying-to-star This commit installs the latest versions of gcc and libstdc++ * Workflow: Try build-essential on ubuntu Still trying to fix: https://www.mathworks.com/matlabcentral/answers/643300-why-do-i-receive-the-error-libstdc-so-6-version-glibcxx_3-4-22-not-found-when-trying-to-star I would prefer to not pin the versions of these libraries so I am seeing if there is a way to install the latest versions without specifying the version number. * Workflow: Troubleshoot Ubuntu gcc version for Python 3.11 Still attempting to fix: https://www.mathworks.com/matlabcentral/answers/643300-why-do-i-receive-the-error-libstdc-so-6-version-glibcxx_3-4-22-not-found-when-trying-to-star on Python 3.11 * Workflow: Specify LD_Preload on Ubuntu https://www.mathworks.com/matlabcentral/answers/1907290-how-to-manually-select-the-libstdc-library-to-use-to-resolve-a-version-glibcxx_-not-found Attempting to fix a mismatch between MATLAB and Ubuntu c++ libs * Workflow: Add troubleshooting info * Workflow: Drop export from environment variable assignment * Workflow: Remove ubuntu debugging jobs Python 3.11 is working currently. This is trying to simplify the workflow and using a more official version of anaconda. * Workflow: Use conda activate source activate env-name -> conda activate env-name * Tests: Remove extraneous whitespace * Workflow: Clean up whitespace * Tests: Split python tests into imports and IO `Python_Import.m` contains tests for importing python packages `Python_MHKiT_Import.m` contains tests for testing data transfer to and from python * Style: Remove extraneous whitespace in loads module * Style: Remove extraneous whitespace in dolfyn module * Style: Remove extraneous whitespace in power module * Spelling: Fix common errors in power module * Style: Remove extraneous whitespace in qc module * Style: Remove extraneous whitespace in river module * Style: Remove extraneous whitespace in test module * Style: Remove extraneous whitespace in tidal module * Style: Use consistent format for line endings Convert line endings to unix * Style: Remove extraneous whitespace in utils module * Style: Use consistent format for line endings * Style: Remove extraneous whitespace in wave module * Fix: Add back annotations for wave power matrix * Style: Format workflows with prettier * Workflow: Skip Windows unit tests Windows unit test are failing due to issues with matlab and python communication that are specific to way github actions work. This saves the windows workflow, but does not use it * Docs: Test new status badge * Fix: Match test class to test filename * Fix: Warnings related to importing numpy Remove all MATLAB imports of `numpy`. This fixes warnings related to numpy import conflicting with functions in the global namespace. Warnings all fit this pattern: `[Warning: The name 'char' is already in use as a method name. This will become an error in a future release.]`. All of the test cases are passing, so this seems like it works. This also reduces the amount of noise in the test output makes it easier for the developer/users to read the output. The current tests seem adequate to cover the removal of these imports. It may also be nice to wrap the mhkit/mhkit_python_utils imports into a function to keep the code dry and add an error message if the user does not have either of the above python libraries installed. We could also add a check for the version of mhkit. * Tests: Fix Python serialization bug in QC tests Convert inputs to `timeseries_to_pandas` to Python lists to provide proper types for dataframe conversion. * Test: Fix Tidal_TestResource test_plot_phase_exceedance Fixing this test required a significant rewrite. This version uses a native MATLAB interpolation function `griddedInterpolant` in place of scipy.interpolate.interp1d. griddedInterpolant is preferred in MATLAB as in requires minimal data conversion and provides less cryptic error messages upon failure. Other changes are related to code cleanup and clarification of variable names. * Tests: Fix test_request_noaa_data The function request_noaa_data_restricted_duration was erroneously deleted. This adds back that function unmodified from https://github.com/MHKiT-Software/MHKiT-MATLAB/blob/deedd7e7099de21ef742877f3de07af2ec657745/mhkit/tidal/io/request_noaa_data.m * Tests: Request multiple days and months of noaa tidal data * Fix: Convert spectrum into numpy array This fixes a bug where the conversion from MATLAB to python was not compatible. This follows the same pattern as `energy_period.m` * Tests: Re-enable tests fixed by 40cd75e * Docs: Remove extraneous whitespace in README * Docs: Use correct unit test badge Previous badge pointed to forked repository * Fix: Convert array to compatible python data type * Tests: Re-enable test_plot_elevation_timeseries Fixed in 1b973e4 * Tests: Fix test_surface_elevation_seed The called python function `surface_elevation` includes an assert that checks for 0 as the first index of a spectral density DataFrame. Thin commit modifies the range input to jonswap_spectrum to include 0. * Tests: Fix test_surface_elevation_phasing Uses same strategy as a0cb0ef * Tests: Fix test_surface_elevation_moments Uses same strategy as a0cb0ef * Tests: Add code coverage Use built in MATLAB code coverage tool to generate code coverage report. There are two warnings when this runs for which fixes are not readily apparent: Warning: Coverage result is empty for check_range.m because the file is invalid. Warning: Coverage result is empty for blade_moments.m because the file is invalid. These two warnings indicate that code coverage is not being generated for those files. * Ignore: Remove .DS_Store files from repository * Tests: Fix intermittent failure in test_flicker_ufic_workflow Explicit conversion to int fixes this failure. Seems like a bug in MATLAB. * Fix: Explicitly convert `spectra_to_pandas` input types * Fix: Keep more relevant test case note * Cleanup: Remove unnecessary `disp` functions * Actions: Re-enable Windows unit tests * Actions: Set Windows MATLAB Python `ExecutionMode` to `OutOfProcess` * Actions: Use MATLAB/Python version matrix for Windows Tests * Actions: Explicitly activate conda on each shell call * Actions: Install same python libraries on Windows * Actions: Try to run Windows MATLAB Python `InProcess` * Cleanup: Remove `disp` * Tests: Use mkhit without explicit import * Workflow: Narrow MATLAB versions on Windows MATLAB versions 2021b and 2022a appear to be working. This disables all other MATLAB versions. * Tests: disable test_WPTO_point_multiloc on Windows This test is failing on Windows. More investigation is needed to find and fix the root cause. Seems related to the URL or the API Key. * Feature: Add cache functionality for web requests This adds the `cached_webread` function which checks for files in `mhkit_webread_cache` before downloading files, and saves the downloaded files to this cache. If the cached files are older than one day they are deleted and re downloaded. * Feature: Cache results from hindcast queries Use `cached_webread` as a drop in replacement for `webread`. `cached_webread` checks if files have already been downloaded before attempting to download. In this specific case, the hindcast data does not change frequently. Additionally, the developer.nrel.gov queries are rate limited to 1000 per hour, and running many consecutive tests can lead to false negatives in the form of: Error using matlab.internal.webservices.HTTPConnector/copyContentToByteArray (line 373) The server returned the status 503 with message "Service Unavailable" in response to the request to URL https://developer.nrel.gov/api/hsds/?api_key=3K3JQbjZmWctY0xmIfSYvYgtIcM3CN0cb1Y2w9bf&domain=%2Fnrel%2FUS_wave%2Fvirtual_buoy%2FWest_Coast%2FWest_Coast_virtual_buoy_2010.h5. * Tests: Remove platform specific conditionals on request_wpto tests `test_WPTO_point_multiloc` and `test_WPTO_point_multiparm` should be performed on both platforms. In reality these tests are slow, but the previous two commits that add should help to decrease execution time. * Actions: Use artifacts to save web request cache The previous 3 commits setup a caching mechanism that saves web requests from developer.nrel.gov to `mhkit_webread_cache`. This commit tries to persist this cache across multiple tests. Currently, the tests are intermittently failing due to some tests failing to access developer.nrel.gov. This should help by reducing the amount of web requests. * Actions: Add artifacts to Windows unit tests * Actions: Setup cache_population job cache_population runs an initial test and saves `mhkit_webread_cache` as a github artifact. The main job runs next and should use this cache. This should force tests that use cached_webread to request files from the cache rather than downloading them. * Actions: Run main job after cache_population * Actions: Run cache_population then main on Windows * Actions: Remove artifact saving in main job * Action: Fix nested artifact directory. In previous tests artifact directory was: /home/runner/work/MHKiT-MATLAB/MHKiT-MATLAB/mhkit_webread_cache/mhkit_webread_cache It should be: /home/runner/work/MHKiT-MATLAB/MHKiT-MATLAB/mhkit_webread_cache * Actions: Attempt to fix artifact upload/download path Try to correctly set name and path. This sets the path and name on upload, but only sets the name on download. We need the mhkit_webread_cache directory to be in the root directory of this project and before this commit it was in <root directory>/mhkit_webread_cache/mhkit_webread_cache * Tests: Temporarily use different api key for developer.nrel.gov calls * Troubleshooting: Output cached webread url to verify url integrity * Feature: Add retries to cached_webread * Actions: Set artifact download path to "." * Actions: Update path for artifact download * Actions: Install openssl on unix builds This is an attempt to fix the following MATLAB/Python error * macos-latest * MATLAB 2023b * Python 3.11 All other tests are passing. ``` 'MATLAB:Python:PyException' -------------- Error Details: -------------- Error using __init__><module> Python Error: ImportError: dlopen(/usr/local/miniconda/envs/TEST/lib/python3.11/site-packages/netCDF4/_netCDF4.cpython-311-darwin.so, 0x0002): Symbol not found: (_SSL_get0_group_name) Referenced from: '/usr/local/miniconda/envs/TEST/lib/libcurl.4.dylib' Expected in: '/Applications/MATLAB_R2023b/MATLAB_R2023b.app/bin/maci64/libssl.3.dylib' ``` * Actions: Use conda-forge to install netcdf4 Still troubleshooting the error from 4f4c2f4 * Docs: Add Python/MATLAB compatibility information * Docs: Remove specific Python versions Reference Python/MATLAB Compatibility Table * Actions: MATLAB 2023b -> latest See if we can download a new version of MATLAB * Actions: Add output of MHKiT-Python information * Docs: Try different checkmark to improve dark mode compatibility * Actions: Try beta MATLAB setup action * Actions: Disable macos-latest python 3.11 matlab latest This test is failing with: Python Error: ImportError: dlopen(/usr/local/miniconda/envs/TEST/lib/python3.11/site-packages/netCDF4/_netCDF4.cpython-311-darwin.so, 0x0002): Symbol not found: (_SSL_get0_group_name) Referenced from: '/usr/local/miniconda/envs/TEST/lib/libcurl.4.dylib' Expected in: '/Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app/bin/maci64/libssl.3.dylib' On every python call. This is not a common error so for now this test is disabled. * Actions: Add MHKiT-Python output to Windows Action * Actions: Use artifact cache on cache_population job This is an attempt 404 issues with developer.nrel.gov * Actions: Use artifact from last workflow * Revert "Actions: Use artifact from last workflow" This reverts commit c8e415697a22ad44df594a1cbc391976a2dda97c. This fails if the last workflow failed, which can lead to a cascade of failures. * Tests: Use original API key for developer.nrel.gov * Tests: Temporarily disable hindcast request `developer.nrel.gov` seems to be blocking these requests, which is causing the tests to fail. * Actions: Populate cache folder with dummy files Ensures that cache creation occurs, which is required for cache download * Tests: Cleanup Python_IO * Actions: Standardize conda env name The conda environment name is `MHKIT_CONDA_ENV` everywhere * Docs: Add Windows Unit Test Badge * Docs: Simplify Unit Test names * Docs: Move Python/MATLAB compatibility into Installation section * Docs: Use more common header format * Actions: Compare tests on MATLAB 2023b vs latest MATLAB latest tests are failing with a new segmentation fault as of today: -------------------------------------------------------------------------------- Segmentation violation detected at 2023-12-08 15:21:24 +0000 -------------------------------------------------------------------------------- Configuration: Crash Decoding : Disabled - No sandbox or build area path Crash Mode : continue (default) Default Encoding : UTF-8 Deployed : false Graphics Driver : Uninitialized hardware Java Version : Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode MATLAB Architecture : maci64 MATLAB Entitlement ID : Unknown MATLAB Root : /Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app MATLAB Version : 23.2.0.2428915 (R2023b) Update 4 OpenGL : hardware Operating System : Mac OS Version 12.6.9 (Build 21G726) Process ID : 3854 Processor ID : x86 Family 6 Model 58 Stepping 10, GenuineIntel Session Key : 8876e372-0353-4b44-aa20-5264e3864f38 Window System : Quartz Fault Count: 1 Abnormal termination: Segmentation violation Current Thread: 'Java: AWT-EventQueue-0' id 0x70000c4ca000 Register State (from fault): RAX = 0000000000000000 RBX = 0000000000000000 RCX = 0000000195f9df10 RDX = 0000000000000080 RSP = 000070000c4c6870 RBP = 000070000c4c6930 RSI = 0000000000000000 RDI = 0000000000000000 R8 = 00000000000005a0 R9 = 0000000000000010 R10 = 00000000000007fb R11 = 00000000000003ff R12 = 0000000000000080 R13 = 0000000195ff5280 R14 = 0000000195f9df10 R15 = 0000000000000000 RIP = 00007ffa20850f0c RFL = 0000000000010246 CS = 000000000000002b FS = 0000000000000000 GS = 0000000000000000 Stack Trace (from fault): [ 0] 0x00000001020f5204 /Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app/bin/maci64/libmwfl.dylib+00008708 _ZN10foundation4core4diag15stacktrace_base7captureERKNS1_14thread_contextEm+00000052 [ 1] 0x00000001020f81d2 /Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app/bin/maci64/libmwfl.dylib+00020946 _ZN10foundation4core4test17terminate_handledERKNSt3__112basic_stringIcNS2_11char_traitsIcEENS2_9allocatorIcEEEE+00002210 [ 2] 0x00000001020f76d0 /Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app/bin/maci64/libmwfl.dylib+00018128 _ZN10foundation4core4diag13terminate_logEPKcPK17__darwin_ucontext+00000128 [ 3] 0x000000010b6d55cb /Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app/bin/maci64/libmwmcr.dylib+00521675 _Z19mnPrintErrorMessageRKNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEE+00011451 [ 4] 0x000000010b6d35f6 /Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app/bin/maci64/libmwmcr.dylib+00513526 _Z19mnPrintErrorMessageRKNSt3__112basic_stringIcNS_11char_traitsIcEENS_9allocatorIcEEEE+00003302 [ 5] 0x000000010b6d1005 /Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app/bin/maci64/libmwmcr.dylib+00503813 mnFatalSignalHandler+00000149 [ 6] 0x00007ff807a70dfd /usr/lib/system/libsystem_platform.dylib+00015869 _sigtramp+00000029 [ 7] 0x00007f95e9822800 <unknown-module>+00000000 [ 8] 0x0000000195f382d7 /Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app/bin/maci64/libjogl_desktop.jnilib+00099031 getRendererInfo+00000087 [ 9] 0x0000000195f38b58 /Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app/bin/maci64/libjogl_desktop.jnilib+00101208 createPixelFormat+00000088 [ 10] 0x0000000195f97d46 /Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app/bin/maci64/libjogl_desktop.jnilib+00490822 Java_jogamp_opengl_macosx_cgl_CGL_createPixelFormat0__Ljava_lang_Object_2IILjava_lang_Object_2I+00000166 [ 11] 0x000000012778d3a7 <unknown-module>+00000000 [ 12] 0x000000012777cb10 <unknown-module>+00000000 ** This crash report has been saved to disk as /Users/runner/matlab_crash_dump.3854-1 ** * Tests: Re-enable Wave_TestIO assumeFail tests The API seems to online now... * Actions: Skips testing macos/MATLAB R2023b. This seems to be the most fragile build. Currently it is errorring out with: -------------------------------------------------------------------------------- Segmentation violation detected at 2023-12-08 15:21:24 +0000 -------------------------------------------------------------------------------- Configuration: Crash Decoding : Disabled - No sandbox or build area path Crash Mode : continue (default) Default Encoding : UTF-8 Deployed : false Graphics Driver : Uninitialized hardware Java Version : Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode MATLAB Architecture : maci64 MATLAB Entitlement ID : Unknown MATLAB Root : /Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app MATLAB Version : 23.2.0.2428915 (R2023b) Update 4 OpenGL : hardware Operating System : Mac OS Version 12.6.9 (Build 21G726) Process ID : 3854 Processor ID : x86 Family 6 Model 58 Stepping 10, GenuineIntel Session Key : 8876e372-0353-4b44-aa20-5264e3864f38 Window System : Quartz Fault Count: 1 Abnormal termination: Segmentation violation Current Thread: 'Java: AWT-EventQueue-0' id 0x70000c4ca000 Register State (from fault): RAX = 0000000000000000 RBX = 0000000000000000 RCX = 0000000195f9df10 RDX = 0000000000000080 RSP = 000070000c4c6870 RBP = 000070000c4c6930 RSI = 0000000000000000 RDI = 0000000000000000 R8 = 00000000000005a0 R9 = 0000000000000010 R10 = 00000000000007fb R11 = 00000000000003ff R12 = 0000000000000080 R13 = 0000000195ff5280 R14 = 0000000195f9df10 R15 = 0000000000000000 RIP = 00007ffa20850f0c RFL = 0000000000010246 CS = 000000000000002b FS = 0000000000000000 GS = 0000000000000000 Stack Trace (from fault): [ 0] 0x00000001020f5204 /Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app/bin/maci64/libmwfl.dylib+00008708 _ZN10foundation4core4diag15stacktrace_base7captureERKNS1_14thread_contextEm+00000052 [ 1] 0x00000001020f81d2 /Users/runner/hostedtoolcache/MATLAB/2023.2.999/x64/MATLAB.app/bin/maci64/libmwfl.dylib+00020946 _ZN10foundation4core4test17terminate_handl…
simmsa
force-pushed
the
delft-3d-io
branch
2 times, most recently
from
e1e0741 to
5216a6a
Compare
|
@browniea we are getting close to finishing up this PR. Could you take a look at the example livescript here and verify that everything looks OK? Our main questions are related to the visualizations. There is not a MATLAB equivalent to Also, in "Comparing Face Data to Cell Data Section", would you be opposed to converting the "Replacing negative numbers close to zero with zero" into a function? I would be happy to add this function into MHKiT-Python. |
|
@simmsa This looks good! My only comment is to change the colorbar from gradiated to discrete in the contour plots. I'm not opposed to adding a "Replacing negative numbers close to zero with zero" function. |
|
@browniea, thank you for the feedback. We have updated the colorbar to use discrete steps and added python function Can you take a look at the MATLAB example notebook and see if there are any other fixes we need to make. Can also verify the logic in the |
|
@rpauly18, any objections if we work on ADCP_Delft3D_TRTS_Example in a separate pull request? This notebook involves a combination of DOLfYN and Delft3D functionality, and to maintain the scope of this pull request, it might be better to focus solely on the completed features here. Any objections? |
That is fine with me and makes the most sense. |
This all looks good. Did you add |
|
@browniea, apologies for the lack of clarity. The Do you think this function would be a good addition to MHKiT-Python? |
Found it, that looks good to me. Yes, I think it would fit into MHKiT-Python as well. I'll create an issue, so I'll remember to include in my next example notebook. |
|
@browniea awesome, thank you for taking a look! I will work towards adding the |
simmsa
force-pushed
the
delft-3d-io
branch
4 times, most recently
from
b94a0fb to
e76fa53
Compare
|
@rpauly18, this is ready for review. List of changes:
Before merging we should create a new binary, but I need guidance on a version number. OK if we call this version 0.4.2? Current version is 0.4.1. |
* Feature: Add runner for code compatibility report Documentation: https://www.mathworks.com/help/matlab/ref/codecompatibilityreport.html Output is displayed and parsed. If "Error" is found is the Severity column, exit status 1 is returned. Planning to add a separate GitHub action that runs this report only. * Actions: Initial test of code compatibility report * Actions: Remove unnecessary steps from code compatibility --------- Co-authored-by: Andrew Simms <[email protected]>
`method` allows the user to choose between using the `ifft` or `sum_of_sines` method for calculation of the surface elevation. See <MHKiT-Software/MHKiT-Python#250> where this was implemented in MHKiT-Python.
Actions MATLAB does not currently work on arm images of macos
This is windows default and this could be causing issues for MATLAB trying to find the correct paths
Fix the following warnings: ``` Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: conda-incubator/setup-miniconda@v2, actions/download-artifact@v3, matlab-actions/setup-matlab@v1, matlab-actions/run-command@v1. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/. ```
This is windows default and this could be causing issues for MATLAB trying to find the correct paths
* Install `netcdf4` through conda * Add python to MATLAB path
Lower case avoids issues with file access across different file systems
|
In the API Doc header for each function that requires a delft_3d_py_object be passed in, please add a note directing the user to use the Delft_3d_open_netcdf function to create that object from the Delft3D file. I know it is included in the example, but someone may skip opening the netCDF file using the MHKiT function and run into issues. Similarly in the error catching in these functions, a note should be included in the test checking the Delft_3d_object type directing the user to use the Delft_3d_open_netcdf function. |
Direct the user to the `delft_3d_open_netcdf` function the user inputs an unexpected input netCDF data type.
|
@rpauly18, 8ff57a3 improves the documentation and error messages (screenshot below) for Delft3D by directing the user to read Delft3D netCDF files using the |
rpauly18
approved these changes
May 31, 2024
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add Delft3D functionality to MHKiT-MATLAB:
Following MHKiT-Python:
get_layer_data(115b5b2)create_points(4dd8fbe)variable_interpolation(d791fdd)get_all_data_points(3d67e62)unorm(5f8e322)turbulent_intensity(930f180)get_all_timestamps(7d59c78)convert_time(c9ad7e3)get_layer_dataget_all_time,get_layer_data,create_points,get_all_data_points, andturbulent_intensity