OAuth doesn't work with import-assets command

[Mast3rSensei]

Hello everyone,

I managed to integrate Google OAuth with Superset, but I have a problem: I need to be able to import my assets (it's something we do often for Version Control reasons), but I'm not able to do it through the superset-cli sub-package inside the preset-cli package.

Now, before activating the OAuth I was not aware that Superset would've forced me to choose only one login method, so the first problem I had to solve was related to that. Considering that so far I always used simply the user name and password of an admin user to login to Superset via API, how the hell can I do the same now that I activated OAuth?
Fortunately I found a solution in this thread: the idea is to keep the auth type as OAuth, but create an admin through the command line that requires a password to login with superset fab create-admin --username ${SUPERSET_ADMIN_USR} --firstname superset --lastname conf --email superset@localhost --password ${SUPERSET_DB_ADMIN_PWD}. This should bypass the need of accessing through OAuth and let me keep my code unchanged and keep accessing the API via username and password.

So far so good, and in fact, I can access the API if I for example call the database endpoint with a GET request. No error is returned. However, if I try instead to use the import-assets command using superset-cli, it returns a RecursionError, due to this exception that is raised ad infinitum on the Superset logs after I call the command:

2024-07-15 09:23:37 superset_app          | 192.168.65.1 - - [15/Jul/2024:08:23:37 +0000] "POST /login/ HTTP/1.1" 405 271 "http://localhost:8088/" "Apache Superset Client (0.2.19)"
2024-07-15 09:23:37 superset_app          | 192.168.65.1 - - [15/Jul/2024:08:23:37 +0000] "GET /api/v1/database/?q=(filters:!(),order_column:changed_on_delta_humanized,order_direction:desc,page:0,page_size:100) HTTP/1.1" 401 44 "http://localhost:8088/" "Apache Superset Client (0.2.19)"
2024-07-15 09:23:37 superset_app          | 192.168.65.1 - - [15/Jul/2024:08:23:37 +0000] "GET /login/ HTTP/1.1" 200 51425 "http://localhost:8088/" "Apache Superset Client (0.2.19)"
2024-07-15 09:23:37 superset_app          | 2024-07-15 08:23:37,227:WARNING:superset.views.base:HTTPException
2024-07-15 09:23:37 superset_app          | Traceback (most recent call last):
2024-07-15 09:23:37 superset_app          |   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
2024-07-15 09:23:37 superset_app          |     rv = self.dispatch_request()
2024-07-15 09:23:37 superset_app          |   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1788, in dispatch_request
2024-07-15 09:23:37 superset_app          |     self.raise_routing_exception(req)
2024-07-15 09:23:37 superset_app          |   File "/usr/local/lib/python3.9/site-packages/flask/app.py", line 1770, in raise_routing_exception
2024-07-15 09:23:37 superset_app          |     raise request.routing_exception  # type: ignore
2024-07-15 09:23:37 superset_app          |   File "/usr/local/lib/python3.9/site-packages/flask/ctx.py", line 351, in match_request
2024-07-15 09:23:37 superset_app          |     result = self.url_adapter.match(return_rule=True)  # type: ignore
2024-07-15 09:23:37 superset_app          |   File "/usr/local/lib/python3.9/site-packages/werkzeug/routing/map.py", line 647, in match
2024-07-15 09:23:37 superset_app          |     raise MethodNotAllowed(valid_methods=list(e.have_match_for)) from None
2024-07-15 09:23:37 superset_app          | werkzeug.exceptions.MethodNotAllowed: 405 Method Not Allowed: The method is not allowed for the requested URL.

My original command is
superset-cli -u ${SUPERSET_ADMIN_USR} -p ${SUPERSET_DB_ADMIN_PWD} http://localhost:8088/ import-assets --disable-jinja-templating --split [destination path]

Any idea how I can make this works?

[Vitor-Avila]

Hey @Mast3rSensei,

Could you try running the CLI command using the jwt authentication?

superset-cli --jwt-token $my_jwt http://localhost:8088/ import-assets --disable-jinja-templating --split [destination path]

This would help confirming that your environment is working properly, and the only issue is the auth piece.

[Mast3rSensei]

You beautiful man @Vitor-Avila, you did it again! The use of the --jwt-token in place of the username and password did the trick! I only had one problem while importing:

Traceback (most recent call last):
  File "/Users/matteo/Library/Caches/pypoetry/virtualenvs/cli-app-e_fqU1FV-py3.9/bin/superset-cli", line 8, in <module>
    sys.exit(superset_cli())
  File "/Users/matteo/Library/Caches/pypoetry/virtualenvs/cli-app-e_fqU1FV-py3.9/lib/python3.9/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/Users/matteo/Library/Caches/pypoetry/virtualenvs/cli-app-e_fqU1FV-py3.9/lib/python3.9/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/matteo/Library/Caches/pypoetry/virtualenvs/cli-app-e_fqU1FV-py3.9/lib/python3.9/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/matteo/Library/Caches/pypoetry/virtualenvs/cli-app-e_fqU1FV-py3.9/lib/python3.9/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/matteo/Library/Caches/pypoetry/virtualenvs/cli-app-e_fqU1FV-py3.9/lib/python3.9/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/matteo/Library/Caches/pypoetry/virtualenvs/cli-app-e_fqU1FV-py3.9/lib/python3.9/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/matteo/Library/Caches/pypoetry/virtualenvs/cli-app-e_fqU1FV-py3.9/lib/python3.9/site-packages/preset_cli/cli/superset/sync/native/command.py", line 245, in native
    import_resources_individually(configs, client, overwrite)
  File "/Users/matteo/Library/Caches/pypoetry/virtualenvs/cli-app-e_fqU1FV-py3.9/lib/python3.9/site-packages/preset_cli/cli/superset/sync/native/command.py", line 284, in import_resources_individually
    for uuid in get_related_uuids(config):
  File "/Users/matteo/Library/Caches/pypoetry/virtualenvs/cli-app-e_fqU1FV-py3.9/lib/python3.9/site-packages/preset_cli/cli/superset/sync/native/command.py", line 306, in get_dashboard_related_uuids
    for uuid in get_dataset_filter_uuids(config):
  File "/Users/matteo/Library/Caches/pypoetry/virtualenvs/cli-app-e_fqU1FV-py3.9/lib/python3.9/site-packages/preset_cli/cli/superset/sync/native/command.py", line 329, in get_dataset_filter_uuids
    for target in filter_config["targets"]:
KeyError: 'targets'

I think I know already where this comes from. We have a dashboard that uses no filters whatsoever, so its native_filter_configuration field is completely missing from the metadata (we're using version 3.0.2 of Superset). I think it would be better to add an additional condition for cases where that field is missing.

Edit1: Sorry, just realized you already had a condition for the native_filter_configuration, but not for the targets. In my case I simply solved by doing:

def get_dataset_filter_uuids(config: AssetConfig) -> Set[str]:
    """
    Extract dataset UUID for datasets that are used in dashboard filters.
    """
    dataset_uuids = set()
    for filter_config in config["metadata"].get("native_filter_configuration", []):
        for target in filter_config.get("targets", []):
            if uuid := target.get("datasetUuid"):
                if uuid not in dataset_uuids:
                    dataset_uuids.add(uuid)
    return dataset_uuids

Edit 2: I actually got another error because I had a dashboard with no metadata section in its yaml file. I solved this last problem by adding another condition for checking if the field is present, however I'm wondering: is it ok or at least expected for a dashboard to have no metadata sometimes?

[Vitor-Avila]

Hey @Mast3rSensei,

I'm glad you managed to get unblocked! Thanks for sharing your findings as well.

In regards to "is it ok or at least expected for a dashboard to have no metadata sometimes?" that's a very tricky question. I believe that it's currently no longer possible/expected that dashboards don't have a minimum metadata. However, for older versions (or perhaps dashboards created in an older version that haven't been updated since) they might be in a "more flexible" configuration.

Would you like to contribute your changes to the repo? We have a CLA now, so we can accept external contributions. The only requirement is that all changes must have test coverage.

[Mast3rSensei]

Hey @Vitor-Avila,

Sure thing! Although this is unknown territory to me, I never contributed to an open source project before, so expect some rookie mistakes with my PR. But sure, as soon as I have some time, I'll push a fix.

[Vitor-Avila]

thanks, and no worries! 🙏 🙌