feat(db): Adding DB_SQLA_URI_VALIDATOR

[craig-rueda]

SUMMARY

Quick PR that adds a new callback hook, DB_SQLA_URI_VALIDATOR which can be optionally set in Superset configs. When set, it should point to a callable that is invoked in order to validate db_engine_spec's URIs.

Example:

def DB_SQLA_URI_VALIDATOR(sqlalchemy_uri: URL):
    if not <some condition>:
        raise Exception("URI invalid")

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.83%. Comparing base (ca47717) to head (8e5db72).

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #27847      +/-   ##
==========================================
+ Coverage   67.41%   69.83%   +2.42%     
==========================================
  Files        1920     1920              
  Lines       75242    75245       +3     
  Branches     8423     8423              
==========================================
+ Hits        50724    52548    +1824     
+ Misses      22457    20636    -1821     
  Partials     2061     2061              

Flag	Coverage Δ
hive	48.93% <66.66%> (?)
mysql	77.88% <66.66%> (-0.01%)	⬇️
postgres	77.99% <66.66%> (-0.01%)	⬇️
presto	53.63% <66.66%> (?)
python	83.15% <100.00%> (+5.01%)	⬆️
sqlite	77.43% <66.66%> (-0.01%)	⬇️
unit	56.78% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[mistercrunch]

2 thoughts:

• about naming DB_SQLA_URI_VALIDATOR might be a slightly better name as it validates a sqlalchemy URI more than an "engine" uri
• I thought we could have a more generic DB_VALIDATOR instead or in complement to this so that someone could look at other DatabaseConnection attributes too, as opposed to the uri only. Password complexity, extra attributes or anything else.

[craig-rueda]

Aren't all of the connection attributes captured in the parsed url? Also, this is intended to be invoked pre-connect, so we likely wouldn't want to perform validations on a "connection".

[mistercrunch]

I think the bigger exception is the way we do BigQuery for instance cramming many things into a blob, but unclear how common/uncommon it is. This probably a good starting point. Can add more hooks if/as needed.

[betodealmeida]

Yeah, in general there are two parts:

engine = create_engine(uri, **params)

Both are needed for a full connection, since in some cases the params are too big to build a URI (like BigQuery credentials).

[dpgaspar]

nit: DB_ENGINE_URI_VALIDATOR: Callable[[URL], None] | None = None

[john-bodley]

@craig-rueda out of interest why is this surfacing as an issue now? Did we not gracefully handle SQLAlchemy URI errors in the past?