Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save Job History on Flink #131

Closed
wants to merge 73 commits into from
Closed

Conversation

ranchodeluxe
Copy link
Collaborator

@ranchodeluxe ranchodeluxe commented Nov 15, 2023

Mount EFS to Job Managers so they can archive jobs for historical status lookups

Addresses #122

Related PR: pangeo-forge/pangeo-forge-cloud-federation#6

@ranchodeluxe ranchodeluxe changed the title WIP: Save Job History on Flink Save Job History on Flink Nov 15, 2023
@ranchodeluxe ranchodeluxe changed the title Save Job History on Flink WIP: Save Job History on Flink Nov 15, 2023
Copy link

codecov bot commented Nov 15, 2023

Codecov Report

Attention: 21 lines in your changes are missing coverage. Please review.

Comparison is base (b8571e1) 96.06% compared to head (d6b5454) 74.50%.
Report is 61 commits behind head on main.

Files Patch % Lines
pangeo_forge_runner/feedstock.py 40.00% 9 Missing ⚠️
pangeo_forge_runner/commands/bake.py 61.53% 5 Missing ⚠️
pangeo_forge_runner/meta_yaml.py 86.95% 3 Missing ⚠️
pangeo_forge_runner/bakery/dataflow.py 0.00% 2 Missing ⚠️
pangeo_forge_runner/bakery/flink.py 77.77% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main     #131       +/-   ##
===========================================
- Coverage   96.06%   74.50%   -21.57%     
===========================================
  Files          14       15        +1     
  Lines         458      502       +44     
===========================================
- Hits          440      374       -66     
- Misses         18      128      +110     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ranchodeluxe ranchodeluxe changed the title WIP: Save Job History on Flink Save Job History on Flink Nov 15, 2023
@ranchodeluxe
Copy link
Collaborator Author

I didn't want to make the EFS mounting optional for our deployment but I see now I probably should in case folks have older clusters 😞

@ranchodeluxe
Copy link
Collaborator Author

looks like I need some tests too

cisaacstern and others added 23 commits November 18, 2023 13:41
In meta_yaml recipes section, `dict_object`s should be a list element
Move everything to pyproject.toml
Fix release workflow for pyproject.toml
@ranchodeluxe ranchodeluxe added the test-flink Add this label to PRs to trigger Dataflow integration test. label Nov 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test-flink Add this label to PRs to trigger Dataflow integration test.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants