Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace plus char with minus in version tag for GCP dev installation of UCX #420

Merged
merged 4 commits into from
Oct 10, 2023

Conversation

larsgeorge-db
Copy link
Contributor

Fixes #419

@larsgeorge-db larsgeorge-db requested review from a team October 10, 2023 08:08
@codecov
Copy link

codecov bot commented Oct 10, 2023

Codecov Report

Merging #420 (d5aca41) into main (bc752e6) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #420   +/-   ##
=======================================
  Coverage   84.22%   84.23%           
=======================================
  Files          31       31           
  Lines        2847     2848    +1     
  Branches      538      538           
=======================================
+ Hits         2398     2399    +1     
  Misses        326      326           
  Partials      123      123           
Files Coverage Δ
src/databricks/labs/ucx/install.py 84.00% <100.00%> (+0.03%) ⬆️

src/databricks/labs/ucx/install.py Outdated Show resolved Hide resolved
@larsgeorge-db larsgeorge-db changed the title Skip extended version strings in dev mode for GCP. Replace plus char with underscore for broader support of dev version numbers Oct 10, 2023
@larsgeorge-db larsgeorge-db requested a review from nfx October 10, 2023 10:35
@larsgeorge-db larsgeorge-db self-assigned this Oct 10, 2023
@larsgeorge-db larsgeorge-db added the pr/do-not-merge this pull request is not ready to merge label Oct 10, 2023
@larsgeorge-db
Copy link
Contributor Author

Nope, that does NOT work @nfx

[+] installing dependencies within ephemeral Virtualenv: /var/folders/qf/rb54syb53992k0qhj_y04_yr0000gp/T/tmp.KzZMkpkNuZ
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/lars.george/projects/work/databricks/ucx/src/databricks/labs/ucx/install.py", line 713, in <module>
    installer.run()
  File "/Users/lars.george/projects/work/databricks/ucx/src/databricks/labs/ucx/install.py", line 110, in run
    logger.info(f"Installing UCX v{self._version}")
                                   ^^^^^^^^^^^^^
  File "/Users/lars.george/projects/work/databricks/ucx/src/databricks/labs/ucx/install.py", line 635, in _version
    raise OSError(msg) from None
OSError: Cannot determine unreleased version. Please report this error message that you see on https://github.com/databrickslabs/ucx/issues/new. Meanwhile, download, unpack, and install the latest released version from https://github.com/databrickslabs/ucx/releases. Original error is: Not a valid SemVer: v0.3.1_2420231010124511

@larsgeorge-db
Copy link
Contributor Author

We could use minus -? https://regex101.com/r/Ly7O1x/2071

@larsgeorge-db larsgeorge-db removed the pr/do-not-merge this pull request is not ready to merge label Oct 10, 2023
@larsgeorge-db larsgeorge-db changed the title Replace plus char with underscore for broader support of dev version numbers Replace plus char with minus in version tag for GCP dev installation of UCX Oct 10, 2023
@@ -490,9 +490,10 @@ def _job_settings(self, step_name: str, dbfs_path: str):
on_success=[self._my_username], on_failure=[self._my_username]
)
tasks = sorted([t for t in _TASKS.values() if t.workflow == step_name], key=lambda _: _.name)
version = self._version if not self._ws.config.is_gcp else self._version.replace("+", "-")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would generalize the code to make sure that all tags are cleaned up, not only the version.

@nfx nfx merged commit 0a12f35 into main Oct 10, 2023
@nfx nfx deleted the fix/issue_419_gcp_tag branch October 10, 2023 14:32
FastLee pushed a commit that referenced this pull request Oct 10, 2023
@nfx nfx mentioned this pull request Oct 12, 2023
nfx added a commit that referenced this pull request Oct 12, 2023
# Version changelog

## 0.4.0

* Added exception handling for secret scope not found.
([#418](#418)).
* Added a crawler for creating an inventory of Azure Service Principals
([#326](#326)).
* Added check if account group already exists during failure recovery
([#446](#446)).
* Added checking for index out of range.
([#429](#429)).
* Added hyperlink to UCX releases in the main readme
([#408](#408)).
* Added integration test to check backup groups get deleted
([#387](#387)).
* Added logging of errors during threadpool operations.
([#376](#376)).
* Added recovery mode for workspace-local groups from temporary groups
([#435](#435)).
* Added support for migrating Legacy Table ACLs from workspace-local to
account-level groups
([#412](#412)).
* Added detection for installations of unreleased versions
([#399](#399)).
* Decoupled `PermissionsManager` from `GroupMigrationToolkit`
([#407](#407)).
* Enabled debug logging for every job task run through a file, which is
accessible from both workspace UI and Databricks CLI
([#426](#426)).
* Ensured that table exists, even when crawlers produce zero records
([#373](#373)).
* Extended test suite for HMS->HMS TACL migration
([#439](#439)).
* Fixed handling of secret scope responses
([#431](#431)).
* Fixed `crawl_permissions` task to respect 'workspace_start_path'
config ([#444](#444)).
* Fixed broken logic in `parallel` module and applied hardened error
handling design for parallel code
([#405](#405)).
* Fixed codecov.io reporting
([#403](#403)).
* Fixed integration tests for crawlers
([#379](#379)).
* Improved README.py and logging messages
([#433](#433)).
* Improved cleanup for workspace backup groups by adding more retries on
errors ([#375](#375)).
* Improved dashboard queries to show unsupported storage types.
([#398](#398)).
* Improved documentation for readme notebook
([#257](#257)).
* Improved test coverage for installer
([#371](#371)).
* Introduced deterministic `env_or_skip` fixture for integration tests
([#396](#396)).
* Made HMS & UC fixtures return `CatalogInfo`, `SchemaInfo`, and
`TableInfo` ([#409](#409)).
* Merge `workspace_access.Crawler` and `workspace_access.Applier`
interfaces to `workspace_access.AclSupport`
([#436](#436)).
* Moved examples to docs
([#404](#404)).
* Properly isolated integration testing for workflows on an existing
shared cluster ([#414](#414)).
* Removed thread pool for any IAM Group removals and additions
([#394](#394)).
* Replace plus char with minus in version tag for GCP dev installation
of UCX ([#420](#420)).
* Run integration tests on shared clusters for a faster devloop
([#397](#397)).
* Show difference between serverless and PRO warehouses during
installation ([#385](#385)).
* Split `migrate-groups` workflow into three different stages for
reliability ([#442](#442)).
* Use groups instead of usernames in code owners file
([#389](#389)).
FastLee pushed a commit that referenced this pull request Oct 25, 2023
FastLee pushed a commit that referenced this pull request Oct 25, 2023
# Version changelog

## 0.4.0

* Added exception handling for secret scope not found.
([#418](#418)).
* Added a crawler for creating an inventory of Azure Service Principals
([#326](#326)).
* Added check if account group already exists during failure recovery
([#446](#446)).
* Added checking for index out of range.
([#429](#429)).
* Added hyperlink to UCX releases in the main readme
([#408](#408)).
* Added integration test to check backup groups get deleted
([#387](#387)).
* Added logging of errors during threadpool operations.
([#376](#376)).
* Added recovery mode for workspace-local groups from temporary groups
([#435](#435)).
* Added support for migrating Legacy Table ACLs from workspace-local to
account-level groups
([#412](#412)).
* Added detection for installations of unreleased versions
([#399](#399)).
* Decoupled `PermissionsManager` from `GroupMigrationToolkit`
([#407](#407)).
* Enabled debug logging for every job task run through a file, which is
accessible from both workspace UI and Databricks CLI
([#426](#426)).
* Ensured that table exists, even when crawlers produce zero records
([#373](#373)).
* Extended test suite for HMS->HMS TACL migration
([#439](#439)).
* Fixed handling of secret scope responses
([#431](#431)).
* Fixed `crawl_permissions` task to respect 'workspace_start_path'
config ([#444](#444)).
* Fixed broken logic in `parallel` module and applied hardened error
handling design for parallel code
([#405](#405)).
* Fixed codecov.io reporting
([#403](#403)).
* Fixed integration tests for crawlers
([#379](#379)).
* Improved README.py and logging messages
([#433](#433)).
* Improved cleanup for workspace backup groups by adding more retries on
errors ([#375](#375)).
* Improved dashboard queries to show unsupported storage types.
([#398](#398)).
* Improved documentation for readme notebook
([#257](#257)).
* Improved test coverage for installer
([#371](#371)).
* Introduced deterministic `env_or_skip` fixture for integration tests
([#396](#396)).
* Made HMS & UC fixtures return `CatalogInfo`, `SchemaInfo`, and
`TableInfo` ([#409](#409)).
* Merge `workspace_access.Crawler` and `workspace_access.Applier`
interfaces to `workspace_access.AclSupport`
([#436](#436)).
* Moved examples to docs
([#404](#404)).
* Properly isolated integration testing for workflows on an existing
shared cluster ([#414](#414)).
* Removed thread pool for any IAM Group removals and additions
([#394](#394)).
* Replace plus char with minus in version tag for GCP dev installation
of UCX ([#420](#420)).
* Run integration tests on shared clusters for a faster devloop
([#397](#397)).
* Show difference between serverless and PRO warehouses during
installation ([#385](#385)).
* Split `migrate-groups` workflow into three different stages for
reliability ([#442](#442)).
* Use groups instead of usernames in code owners file
([#389](#389)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Installation failing for GCP workspace
3 participants