Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Produce test data set #5

Closed
1 of 3 tasks
leewujung opened this issue Aug 27, 2018 · 8 comments
Closed
1 of 3 tasks

Produce test data set #5

leewujung opened this issue Aug 27, 2018 · 8 comments
Assignees
@erinann
Milestone
Set up repo struc...

Comments

@leewujung
Copy link
Member

leewujung commented Aug 27, 2018 •
edited
Loading

Produce small data sets that contains only a couple pings for testing purposes (#4 travis-ci)

Need data from:

  • EK60
  • EK80
  • AZFP
@erinann erinann self-assigned this Aug 28, 2018
@erinann
Copy link
Contributor

erinann commented Aug 28, 2018

This is issue number 8 from OHW18_echopype.

@amarburg
Copy link
Contributor

amarburg commented Sep 4, 2018

And also a plan for how to distribute the test data when running tests. Git submodule? Just check it into the repo? Download from the cloud?

@amarburg amarburg mentioned this issue Sep 4, 2018
Closed
@leewujung
Copy link
Member Author

leewujung commented Sep 4, 2018 •
edited
Loading

I'll push up what i have done for folder restructuring once I get the minimum fake test working :P I think it's convenient to just have sample data under ./echopype/data as recommended by shablona -- currently I have a 1MB EK60 file and a 5MB AZFP file. Planning on finding someone with the echosounders to generate a test file that just have a few pings.

@amarburg
Copy link
Contributor

amarburg commented Sep 5, 2018

It's a good question. I guess test data is in the Git repo but is not necessarily included in the Python package ... I'd make it a priority to keep the python package as small as possible.

@leewujung
Copy link
Member Author

I think since this package is for unpacking binary/hex data, it is a good idea to include minimum original data files and unpacked files so that users can have something to check against. This can be specified in package_data within setup.py.

@amarburg
Copy link
Contributor

amarburg commented Sep 5, 2018

Yeah, I agree. It was just a question between putting the data in the Git repo and maybe storing it somewhere else and needing to pull it in. If the test files are going to be small, (10s MB) then it's a moot point ... I just didn't have an expectation on how big (or small) the test files would be.

@leewujung
Copy link
Member Author

The EK60 converter (#22) is currently tested against /echopype/data/DY1801_EK60-D20180211-T164025.raw

@leewujung
Copy link
Member Author

Seems like it's fine to use the small-ish data set (1-5 MB range) for now.

leewujung pushed a commit that referenced this issue Dec 5, 2020
@ngkavin
Merge pull request #5 from leewujung/class-redesign
50a7b7d 
various small things to fix
leewujung pushed a commit that referenced this issue Mar 26, 2024
@anujsinha3 @anantmittal
Optimize get vend cal params power (#1285)
8600a59 
* Optimize get vend cal params power (#2)

* Refactor function to avoid expand_dims and sortby if required

* test: increase co-ordinates to 1000 for ping_time dimension

* style: upper-casing variables

* style: upper-casing variables

* style: revert

---------

Co-authored-by: Anant Mittal <[email protected]>

* test: update test_get_interp_da() to test on 200 time_coordinates (#3)

* test: refactor test data (#4)

* test: refactor test data (#5)

* test: refactor test data

* test: refactor test data

---------

Co-authored-by: Anant Mittal <[email protected]>
leewujung added a commit that referenced this issue Apr 8, 2024
@leewujung @dependabot
@anantmittal @pre-commit-ci @lsetiawan @anujsinha3
Merge pull request #1297 from OSOceanAcoustics/dev
57ec40d 
* Bump actions/cache from 3.0.11 to 3.2.0 (#18)

Bumps [actions/cache](https://github.com/actions/cache) from 3.0.11 to 3.2.0.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](actions/cache@v3.0.11...v3.2.0)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update compress_pulse to Parallelize Convolution (#1208)

Update compress_pulse to Parallelize Convolution. 
This addresses #1164.

* Optimize `harmonize_env_param_time` (#1235)

* Remove extra elses

* Ensure harmonize_env_param_time works with dask arrays

* Remove unused import

* Optimize Frequency Differencing with Dask (#1198)

* Update frequency_differencing method: add dask optimizations if Sv data is dask array

* Update test_frequency_differencing function and incorporate test cases for dask

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix ci error

* Update echopype/mask/api.py

Co-authored-by: Don Setiawan <[email protected]>

* Update echopype/mask/api.py: assign attrs out of if-else

* Update echopype/mask/api.py: remove unused code

* refactor: Update dask chunks iter to use xr.map_blocks

* refactor: Cleanup code and make more explicit

* Add better comments

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix pre-commit ci errors

* refactor: Update algorithm to use map_blocks

Update the looping algorithm to use dask array
map_block instead. This would remove direct slicing
ambiguity and prepare for future xarray
map_block usage.

* refactor: Modify to have no hardcoded channel slicing

* refactor: Assign Sv data array

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Don Setiawan <[email protected]>

* Optimize get vend cal params power (#1285)

* Optimize get vend cal params power (#2)

* Refactor function to avoid expand_dims and sortby if required

* test: increase co-ordinates to 1000 for ping_time dimension

* style: upper-casing variables

* style: upper-casing variables

* style: revert

---------

Co-authored-by: Anant Mittal <[email protected]>

* test: update test_get_interp_da() to test on 200 time_coordinates (#3)

* test: refactor test data (#4)

* test: refactor test data (#5)

* test: refactor test data

* test: refactor test data

---------

Co-authored-by: Anant Mittal <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Anant Mittal <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Don Setiawan <[email protected]>
Co-authored-by: anujsinha3 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@erinann erinann

Labels
None yet
Projects
None yet
Milestone
Set up repo structure for packaging a...
Development

No branches or pull requests
3 participants
@amarburg @leewujung @erinann