Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Fetch Migrations] Use custom exceptions and other readability improvements #506

Merged
merged 3 commits into from
Feb 13, 2024

Conversation

kartg
Copy link
Member

@kartg kartg commented Feb 8, 2024

Description

This PR incorporates two pieces of prior feedback from @gregschohn :

  1. The index_operations.py file has been renamed to index_management.py
  2. Custom exceptions have been created for index_management.py and metadata_migration.py to use instead of the generic RuntimeError class to help disambiguate errors in the call stack

This change also includes a minor bugfix to the showFetchMigrationCommand.sh script to move default value declaration so that command-line arguments are parsed correctly.

Issues Resolved

closes #491

Testing

Unit tests pass.

$ python -m coverage run -m unittest
............................................................................................
----------------------------------------------------------------------
Ran 92 tests in 0.458s

OK

$ python -m coverage report --omit "*/tests/*"
Name                           Stmts   Miss  Cover
--------------------------------------------------
component_template_info.py        15      0   100%
endpoint_info.py                  24      0   100%
endpoint_utils.py                106      1    99%
exceptions.py                      9      0   100%
fetch_orchestrator.py             76      0   100%
fetch_orchestrator_params.py      22      0   100%
index_diff.py                     20      0   100%
index_doc_count.py                 5      0   100%
index_management.py              118      0   100%
index_template_info.py             5      0   100%
metadata_migration.py             88      0   100%
metadata_migration_params.py       7      0   100%
metadata_migration_result.py       5      0   100%
migration_monitor.py              90      0   100%
migration_monitor_params.py        6      0   100%
progress_metrics.py               91      0   100%
utils.py                          13      0   100%
--------------------------------------------------
TOTAL                            700      1    99%

Check List

  • New functionality includes testing
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link

codecov bot commented Feb 8, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (ffbb46e) 76.06% compared to head (810a1a4) 76.06%.

Additional details and impacted files
@@            Coverage Diff            @@
##               main     #506   +/-   ##
=========================================
  Coverage     76.06%   76.06%           
  Complexity     1356     1356           
=========================================
  Files           158      158           
  Lines          6011     6011           
  Branches        509      509           
=========================================
  Hits           4572     4572           
  Misses         1088     1088           
  Partials        351      351           
Flag Coverage Δ
unittests 76.06% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…dex_management.py and metadata_migration.py

This commit adds custom exception classes contained within the exceptions.py file to improve readability in code and stack traces, and reduce the use of the generic RuntimeError class. Low-level GET requests now raise a RequestError, while index_management and metadata_migration each have their own higher-level exception classes.

This commit also updates and corrects the unit tests to check for these exception classes, and ensure that the underlying cause is propogated.

Signed-off-by: Kartik Ganesh <[email protected]>
…h to before they are read from the command line

Signed-off-by: Kartik Ganesh <[email protected]>
Copy link
Collaborator

@gregschohn gregschohn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - the new names make the code a lot more explicit!

@kartg kartg merged commit d3820af into opensearch-project:main Feb 13, 2024
8 checks passed
@kartg kartg deleted the fm-improvements branch February 13, 2024 19:25
gregschohn pushed a commit to gregschohn/opensearch-migrations that referenced this pull request Feb 20, 2024
opensearch-project#506)

This PR incorporates two pieces of prior feedback:

1. The "index_operations.py" file has been renamed to "index_management.py" to eliminate confusion on whether the operations in this file involve indexing of data (they don't).

2. Custom exceptions have been created for "index_management.py" and "metadata_migration.py" to use instead of the generic "RuntimeError" class to help disambiguate errors in the call stack.

This change also includes a minor bugfix to the "showFetchMigrationCommand.sh" script to move default value declaration so that command-line arguments are parsed correctly.

---------

Signed-off-by: Kartik Ganesh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ Refactoring / Code Improvements ] Incorporate naming and typing feedback from PR 477
2 participants