-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor SEG-Y export to optimize memory usage. #109
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tasansal
force-pushed
the
feature/refactor_segy_export
branch
2 times, most recently
from
October 18, 2022 14:41
b21e4a7
to
0ced705
Compare
Codecov Report
@@ Coverage Diff @@
## main #109 +/- ##
==========================================
+ Coverage 81.45% 81.72% +0.26%
==========================================
Files 42 43 +1
Lines 1564 1647 +83
Branches 171 178 +7
==========================================
+ Hits 1274 1346 +72
- Misses 256 264 +8
- Partials 34 37 +3
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
tasansal
force-pushed
the
feature/refactor_segy_export
branch
2 times, most recently
from
October 26, 2022 14:24
d3a2ed5
to
820fe0c
Compare
tasansal
force-pushed
the
feature/refactor_segy_export
branch
from
November 2, 2022 22:24
576b89d
to
f10ba40
Compare
This has significant improvements to memory usage: import dask
import distributed
with dask.config.set({"distributed.scheduler.worker-saturation": "1.0"}):
client = distributed.Client(...) |
tasansal
force-pushed
the
feature/refactor_segy_export
branch
from
November 7, 2022 18:58
29580f6
to
b87a1ba
Compare
tasansal
force-pushed
the
feature/refactor_segy_export
branch
from
November 15, 2022 03:58
bfd1b1d
to
3ed44a1
Compare
tasansal
force-pushed
the
feature/refactor_segy_export
branch
from
November 18, 2022 22:00
6d582d3
to
c644890
Compare
tasansal
force-pushed
the
feature/refactor_segy_export
branch
from
November 19, 2022 03:26
47ea835
to
5f7eb40
Compare
@srib can you take a look when you have a chance, the entire export logic changed here, would be good to have a second set of eyes. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
closes #34
ref: Dask Discourse
Proposed graph for 3D below. 2D or 4D is similar, with less or more hierarchy.
There are a few minor changes not related to export.