-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use just the frame-id for CLI #92
Conversation
isce2_topsapp/localize_slc.py
Outdated
raise RuntimeError('Region of interest does not overlap with IFG ' | ||
'area (ref and sec overlap)') | ||
if frame_id != -1: | ||
df_frames = gpd.read_file('s3://s1-gunw-frames/s1_frames.geojson', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This file is only accessible using JPL A19 AWS credentials and won't be accessible from other deployments, or to developers locally (without effort). If you're going to keep it in an S3 bucket, it'd be best to make it a public bucket.
However, since this file is only 72 mb, and compresses down to 22 mb with zip, I think you'd be best off just including it in the repo. That way, it's guaranteed to work for local developers, in the container, and in any deployment. Also, that way changes to the frames are explicitly captured in PRs/git history, and documented in the changelog.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adding the enumerator as a dependency and reading from that would also work.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am going to add the enumerator as a dependency. There are issues with the frame size/overlap as it takes too long. Will get clarity soon from @dbekaert.
Relevant discussion (shared this with @asjohnston-asf yesterday) tldr: we need to figure out proper size of frames - this is going to be very fluid implementation for a few weeks. |
@jhkennedy and @asjohnston-asf This is finally ready for review and to be used to update hyp3. Notes:
Here is the GUNW for the first example captured in the readme [link]. For additional context, here are some geojsons of the frame 22438 and its bounding box. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this looks good now; but I wasn't able to open the NetCDF you posed because both QGIS and ncview give me an error like:
ncview: can't recognize format of input file S1-GUNW-D-R-144-tops-20230125_20221220-140045-00121W_00035N-PP-4569-v2_0_6.nc
That is, in terms of testing the hyp3 integration, this looks good to me. But the actual product produced likely needs work. |
Per @asjohnston-asf suggestion, utilize just the frame-id to submit jobs.
I am reading directly from an s3 bucket. We could also add the dependency from s1-frame-enumerator and read it from that. Not sure about egress implications - should be infinitesimal.