Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ISSUE] Regression in PermissionDefined error handling #755

Closed
asnare opened this issue Sep 10, 2024 · 1 comment · Fixed by #758
Closed

[ISSUE] Regression in PermissionDefined error handling #755

asnare opened this issue Sep 10, 2024 · 1 comment · Fixed by #758
Assignees

Comments

@asnare
Copy link

asnare commented Sep 10, 2024

Description

The 0.32.0 release included some changes (#741) that refactored error handling. There's a regression in the handling for PermissionDenied that results in an AttributeError being raised because the code tries to set message on PermissionDenied, but that attribute isn't present.

Reproduction

On an account where you don't have permission to create a group, raising PermissionDenied fails:

Python 3.10.15 (main, Sep  7 2024, 00:20:06) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> from databricks.sdk import AccountClient, version
>>> version.__version__
'0.32.1'
>>> acc = AccountClient()
DEBUG:databricks.sdk:Loaded from environment
DEBUG:databricks.sdk:Attempting to configure auth: pat
DEBUG:databricks.sdk:Attempting to configure auth: basic
DEBUG:databricks.sdk:Attempting to configure auth: metadata-service
DEBUG:databricks.sdk:Attempting to configure auth: oauth-m2m
DEBUG:databricks.sdk:Attempting to configure auth: azure-client-secret
DEBUG:databricks.sdk:Attempting to configure auth: github-oidc-azure
DEBUG:databricks.sdk:Attempting to configure auth: azure-cli
DEBUG:databricks.sdk:Loading tenant ID from https://**REDACTED**/aad/auth
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): **REDACTED**:443
DEBUG:urllib3.connectionpool:https://**REDACTED**:443 "GET /aad/auth HTTP/11" 302 0
DEBUG:databricks.sdk:Loaded tenant ID: common
INFO:databricks.sdk:Using Azure CLI authentication with AAD tokens
>>> acc.groups.create(display_name='test-group-1234')
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): **REDACTED**:443
DEBUG:urllib3.connectionpool:https://**REDACTED**:443 "POST /api/2.0/accounts/**REDACTED**/scim/v2/Groups HTTP/11" 403 146
DEBUG:databricks.sdk:POST /api/2.0/accounts/**REDACTED**/scim/v2/Groups
> {
>   "displayName": "test-group-1234"
> }
< 403 Forbidden
< {
<   "error_code": "403",
<   "message": "Source IP address: **REDACTED** is blocked by Databricks IP ACL for account: **REDACTED**"
< }
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../.venv/lib/python3.10/site-packages/databricks/sdk/service/iam.py", line 1502, in create
    res = self._api.do('POST',
  File ".../.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 155, in do
    response = retryable(self._perform)(method,
  File ".../.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper
    raise err
  File ".../.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper
    return func(*args, **kwargs)
  File ".../.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 270, in _perform
    error.message = self._cfg.wrap_debug_info(error.message)
AttributeError: 'PermissionDenied' object has no attribute 'message'

Expected behavior

The PermissionDenied error should be raised correctly:

Python 3.10.15 (main, Sep  7 2024, 00:20:06) [Clang 15.0.0 (clang-1500.3.9.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import logging
>>> logging.basicConfig(level=logging.DEBUG)
>>> from databricks.sdk import AccountClient, version
>>> version.__version__
'0.31.1'
>>> acc.groups.create(display_name='test-group-1234')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'acc' is not defined
>>> acc = AccountClient()
DEBUG:databricks.sdk:Loaded from environment
DEBUG:databricks.sdk:Attempting to configure auth: pat
DEBUG:databricks.sdk:Attempting to configure auth: basic
DEBUG:databricks.sdk:Attempting to configure auth: metadata-service
DEBUG:databricks.sdk:Attempting to configure auth: oauth-m2m
DEBUG:databricks.sdk:Attempting to configure auth: azure-client-secret
DEBUG:databricks.sdk:Attempting to configure auth: github-oidc-azure
DEBUG:databricks.sdk:Attempting to configure auth: azure-cli
DEBUG:databricks.sdk:Loading tenant ID from https://**REDACTED**/aad/auth
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): **REDACTED**:443
DEBUG:urllib3.connectionpool:https://**REDACTED**:443 "GET /aad/auth HTTP/11" 302 0
DEBUG:databricks.sdk:Loaded tenant ID: common
INFO:databricks.sdk:Using Azure CLI authentication with AAD tokens
>>> acc.groups.create(display_name='test-group-1234')
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): **REDACTED**:443
DEBUG:urllib3.connectionpool:https://**REDACTED**:443 "POST /api/2.0/accounts/**REDACTED**/scim/v2/Groups HTTP/11" 403 146
DEBUG:databricks.sdk:POST /api/2.0/accounts/**REDACTED**/scim/v2/Groups
> {
>   "displayName": "test-group-1234"
> }
< 403 Forbidden
< {
<   "error_code": "403",
<   "message": "Source IP address: **REDACTED** is blocked by Databricks IP ACL for account: **REDACTED**"
< }
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../.venv/lib/python3.10/site-packages/databricks/sdk/service/iam.py", line 1502, in create
    res = self._api.do('POST',
  File ".../.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 157, in do
    response = retryable(self._perform)(method,
  File ".../.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 54, in wrapper
    raise err
  File ".../.venv/lib/python3.10/site-packages/databricks/sdk/retries.py", line 33, in wrapper
    return func(*args, **kwargs)
  File ".../.venv/lib/python3.10/site-packages/databricks/sdk/core.py", line 271, in _perform
    raise self._make_nicer_error(response=response, **payload) from None
databricks.sdk.errors.platform.PermissionDenied: Source IP address: **REDACTED** is blocked by Databricks IP ACL for account: **REDACTED**. Config: host=https://**REDACTED**, account_id=**REDACTED**, azure_tenant_id=common, auth_type=azure-cli. Env: DATABRICKS_HOST, DATABRICKS_ACCOUNT_ID

Is it a regression?

This is a regression introduced in 0.32.0 against 0.31.1. (It is not fixed in 0.32.1.)

Debug Logs

Logs are included in the examples above.

Other Information

  • OS: macOS
  • Version: 0.32.0, 0.32.1

Additional context

We (UCX) discovered this when users reported difficulty understanding the cause of a (permissions-related) error: databrickslabs/ucx#2542

@sdanielzafar
Copy link

This is a big issue causing issues in UCX that fail assessment runs:
databrickslabs/ucx#2542

@mgyucht mgyucht self-assigned this Sep 12, 2024
github-merge-queue bot pushed a commit that referenced this issue Sep 13, 2024
## Changes
#741 introduced a change to how an error message was modified in
`ApiClient._perform`. Previously, arguments to the DatabricksError
constructor were modified as a dictionary in `_perform`. After that
change, `get_api_error` started to return a `DatabricksError` instance
whose attributes were modified. The `message` attribute referred to in
that change does not exist in the DatabricksError class: there is a
`message` constructor parameter, but it is not set as an attribute.

This PR refactors the error handling logic slightly to restore the
original behavior. In doing this, we decouple all error-parsing and
customizing logic out of ApiClient. This also sets us up to allow for
further extension of error parsing and customization in the future, a
feature that I have seen present in other SDKs.

Fixes #755.

## Tests
<!-- 
How is this tested? Please see the checklist below and also describe any
other relevant tests
-->

- [ ] `make test` run locally
- [ ] `make fmt` applied
- [ ] relevant integration tests applied
aravind-segu pushed a commit to aravind-segu/databricks-sdk-py that referenced this issue Sep 18, 2024
## Changes
databricks#741 introduced a change to how an error message was modified in
`ApiClient._perform`. Previously, arguments to the DatabricksError
constructor were modified as a dictionary in `_perform`. After that
change, `get_api_error` started to return a `DatabricksError` instance
whose attributes were modified. The `message` attribute referred to in
that change does not exist in the DatabricksError class: there is a
`message` constructor parameter, but it is not set as an attribute.

This PR refactors the error handling logic slightly to restore the
original behavior. In doing this, we decouple all error-parsing and
customizing logic out of ApiClient. This also sets us up to allow for
further extension of error parsing and customization in the future, a
feature that I have seen present in other SDKs.

Fixes databricks#755.

## Tests
<!-- 
How is this tested? Please see the checklist below and also describe any
other relevant tests
-->

- [ ] `make test` run locally
- [ ] `make fmt` applied
- [ ] relevant integration tests applied
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants