Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add simple plotting functionality #436

Merged
merged 29 commits into from
Dec 10, 2021
Merged

Conversation

lsetiawan
Copy link
Member

@lsetiawan lsetiawan commented Aug 30, 2021

Overview

This PR adds an initial simple plotting functionality by calling ep.visualize.create_echogram.

NOTE: This is still a draft, suggestions are highly welcomed and encouraged.

Example

Screenshot from 2021-08-30 15-42-14

Actions

  • If user passes in EchoData, plot the backscatter_r in beam group.
  • If user passes MVBS / Sv, then plot those directly.
  • Ability to plot EK80
  • Ability to plot EK60
  • Ability to plot AZFP
  • Ability to plot ADCP (Currently not supported at this moment).
  • Add docstring to function
  • Add actual plot comparison test? (Maybe future improvements, not sure how to do this at this moment).
  • Add a custom EK500 colormap along with the cmocean colormaps.

Notes

There are still some quirks and improvements to add, and some questions to be discussed such as:

  1. How to plot the EK80 quadrant dims?
  2. Backscatter_r is not available in ADCP, at least the one on the test data. In this case, what to do?
  3. How do I utilize the water_level to calibrate the range?
  4. How to improve the ease of the function call?

@lsetiawan lsetiawan requested a review from leewujung August 30, 2021 22:48
@lsetiawan lsetiawan added this to the 0.5.4 release milestone Aug 30, 2021
@lsetiawan lsetiawan linked an issue Aug 30, 2021 that may be closed by this pull request
@codecov-commenter
Copy link

codecov-commenter commented Aug 30, 2021

Codecov Report

Merging #436 (fdc31e4) into dev (d996e28) will decrease coverage by 57.18%.
The diff coverage is 100.00%.

Impacted file tree graph

@@             Coverage Diff             @@
##              dev     #436       +/-   ##
===========================================
- Coverage   77.13%   19.94%   -57.19%     
===========================================
  Files          39       40        +1     
  Lines        3415     3420        +5     
===========================================
- Hits         2634      682     -1952     
- Misses        781     2738     +1957     
Flag Coverage Δ
unittests 19.94% <100.00%> (-57.19%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
echopype/__init__.py 100.00% <100.00%> (ø)
echopype/metrics/summary_statistics.py 0.00% <0.00%> (-96.43%) ⬇️
echopype/convert/set_groups_ek80.py 12.05% <0.00%> (-83.66%) ⬇️
echopype/convert/set_groups_ad2cp.py 16.66% <0.00%> (-82.23%) ⬇️
echopype/calibrate/calibrate_ek.py 12.19% <0.00%> (-80.14%) ⬇️
echopype/convert/set_groups_azfp.py 19.64% <0.00%> (-78.54%) ⬇️
echopype/convert/parse_azfp.py 14.46% <0.00%> (-76.73%) ⬇️
echopype/convert/set_groups_ek60.py 16.50% <0.00%> (-74.68%) ⬇️
echopype/preprocess/noise_est.py 22.85% <0.00%> (-71.43%) ⬇️
echopype/utils/io.py 13.23% <0.00%> (-71.33%) ⬇️
... and 21 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d996e28...fdc31e4. Read the comment docs.

@lsetiawan
Copy link
Member Author

@leewujung Do you know if there's a better file for me to develop the visualization with that's pretty? 😃

@emiliom
Copy link
Collaborator

emiliom commented Aug 31, 2021

TODO: Investigate what platform or sonar/beam metadata attribute exists or should be added that clearly specifies the "direction" the echosounder is oriented (eg, downward vs upward looking). This need came about in the context of the OOI EK60/80 sensor @lsetiawan is using in his plotting tests, which is upward looking. This functionality would allow the echogram plots of raw data to be automatically oriented correctly with a depth y axis.

Looking through the SONAR-netCDF4 convention, the Sonar/Beam beam_mode attribute is helpful, but not fully what's needed; its values can be "vertical", "horizontal" and "inspection". Platform variables involving orientation and coordinate system may be usable, too, but currently the convention uses a ship-based frame of reference.

@gavinmacaulay
Copy link
Contributor

The upcoming version of SONAR-netCDF4 has a more complete orientation and coordinate system that allows for specifying absolute beam pointing directions - details here. It was designed for multi-beam bathymetric sonars and might be a bit complicated for just upward-looking echosounders, but was intended to be general enough for all purposes and does not assume a ship-based frame of reference (keep in mind that the platform is not necessary a vessel).

The beam_mode in version 1 of SONAR-netCDF4 is a label for the different groups of beams that fisheries omni-directional sonars have (e.g., a set of horizontal beams, a set of vertical beams, and spot beams - the word inspection comes from the usage in such sonars - a narrow beam for inspecting parts of a fish school)

@leewujung
Copy link
Member

Great! please add EK500 colormap as one of the options. :)

@emiliom
Copy link
Collaborator

emiliom commented Sep 5, 2021

@gavinmacaulay thanks so much for chiming in to elaborate on the new coordinate system framework and the context of the beam_mode attribute! I'll look into the coordinate system description more closely.

@lsetiawan
Copy link
Member Author

lsetiawan commented Sep 20, 2021

EK500 Example

Screenshot from 2021-09-20 16-42-05

Notes

With this improvement, when a user import echopype the ek500 colormap is available at ep.ek500 or ep.ek500_r.
Screenshot from 2021-09-20 16-46-47

@leewujung
Copy link
Member

For quick visualization of EK80 complex data, we can plot the magnitude

np.abs(ed.beam.backscatter_r + 1j * ed.beam.backscatter_i)

of each quadrant separately.

Can have an option doing a coherent sum over all quadrants

(ed.beam.backscatter_r + 1j * ed.beam.backscatter_i).sum(dim="quadrant")

and then do the magnitude.

@leewujung leewujung removed this from the 0.5.4 release milestone Sep 29, 2021
@lsetiawan lsetiawan marked this pull request as ready for review November 1, 2021 16:32
@leewujung leewujung added this to the 0.5.5 release milestone Nov 2, 2021
@lsetiawan lsetiawan self-assigned this Nov 4, 2021
echopype/visualize/api.py Outdated Show resolved Hide resolved
@leewujung
Copy link
Member

@lsetiawan : thanks for the changes, they look great! I put in a couple small comments that should pretty easy to address.

Do you think you could add a test for the water_level options? With all the possibilities it seems a good idea to have a test to catch any potential hiccups.

Copy link
Member

@leewujung leewujung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @lsetiawan : Thanks for the changes. I found a couple bugs in the water_level tests and added a couple suggestions. Once they are fixed this should be good to go!

@leewujung
Copy link
Member

@lsetiawan : thanks, the changes look great! I'll merge this after adding in some small notes to the create_echogram docstring due to the range vs depth stuff below.

One thing I noticed which we'll need to do something about (also requested in #259 ) is to include the addition of water_level in the Sv dataset (since right now even if users put in water_level=True if the input is an Sv dataset no water_level offset is added. A more accurate way to think about it is probably that we are actually plotting "depth" in the case when we have range_meters + water_level.

The more general description is that people are plotting the range offset from some platform. Let's discuss how that coordinate should be named... Hence I am wondering if water_level should be range_offset instead, so that it is more general.

Pinging @emiliom here since this is related to the whole convention and attribute conversation.

@leewujung
Copy link
Member

To check how the EK500 colormap looks like (as in this previous comment), use matplotlib.colormaps["ep.ek500"].

From a recent deprecation warning:

The get_cmap function was deprecated in Matplotlib 3.7 and will be removed two minor releases later. Use matplotlib.colormaps[name] or matplotlib.colormaps.get_cmap(obj) instead.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Basic echogram plots
5 participants