Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[python] Re-introduce tiledb_ctx for SOMATileDBContext #3335

Merged
merged 7 commits into from
Nov 26, 2024

Conversation

nguyenv
Copy link
Member

@nguyenv nguyenv commented Nov 18, 2024

Issue and/or context:

[sc-58711]

Changes:

  • Re-introduce the tiledb_ctx argument
  • Correct documentation
  • Update typing

Notes for Reviewer:

We are not reintroducing tiledb-py as a dependency. The tiledb_ctx is only usable when tiledb-py is installed.

@nguyenv nguyenv changed the title [python] Re-introduce tiledb_ctx for SOMATileDBContext [python] Re-introduce tiledb_ctx for SOMATileDBContext Nov 18, 2024
Copy link

codecov bot commented Nov 18, 2024

Codecov Report

Attention: Patch coverage is 80.85106% with 9 lines in your changes missing coverage. Please review.

Project coverage is 85.89%. Comparing base (d1178ae) to head (7cc7245).
Report is 18 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3335      +/-   ##
==========================================
+ Coverage   85.87%   85.89%   +0.02%     
==========================================
  Files          55       55              
  Lines        6143     6176      +33     
==========================================
+ Hits         5275     5305      +30     
- Misses        868      871       +3     
Flag Coverage Δ
python 85.89% <80.85%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
python_api 85.89% <80.85%> (+0.02%) ⬆️
libtiledbsoma ∅ <ø> (∅)

Copy link
Member

@bkmartinjr bkmartinjr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of issues:

  1. The PR is missing the tiledb_ctx property that was present in the API. This is the first point mentioned in the bug report. Previously it was possible to do the following:
context = soma.SOMATileDBContext()
A = tiledb.open(uri, ctx=context.tiledb_ctx)

We did deprecate this property in 1.14.5, so removing it is fair game. I'm just unsure what decision @aaronwolen took - the bug report suggests that he wanted to add both back, but I may be misreading it.

Recommend checking with him to confirm removal.

  1. There are some minor style/pattern comments inline.

Comment on lines 73 to 83
tiledb_config: Dict[str, Union[str, float]] | None = None,
tiledb_ctx: "tiledb.Ctx" | None = None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
tiledb_config: Dict[str, Union[str, float]] | None = None,
tiledb_ctx: "tiledb.Ctx" | None = None,
tiledb_config: Dict[str, Union[str, float]] | "tiledb.Ctx" | None = None,

I would slightly prefer this. Admittedly passing a context to the tiledb_config doesn't sound very good, but it eliminates the error case of passing both a context and a config. You could take a leap and say that a context is part of the "configuration".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did considering having tiledb_config also take in a tiledb.Ctx but decided to revert back to how we previously had it as separate arguments. But I'm open to further discussion about only having a tiledb_config argument now.

# it should be impossible to enter into this block because
# we already check that in the constructor
assert tiledb is not None
cfg = self._tiledb_ctx().config().dict()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think _tiledb_ctx is a method, so I suspect this should drop the parens, e.g.,

cfg = self._tiledb_ctx.config().dict()

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ps. assuming my comment is correct, does this imply a missing test case? (or my comment is wrong!)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right that there should not be parens.

The missing test case brings up a another issue though. We have removed all tiledb-py usage from the unit tests. But should we temporarily re-introduce it for testing with tiledb.Ctx()? We would xskip the test if tiledb is not installed locally. But then do we need to install tiledb for CI runs? (cc @johnkerl)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, tough one. I don't see any great solution - options?

The best I can come up with is:

  1. Code the tests to work both with and without the package installed:
    • if installed, test that it works as expected
    • if not installed, test that it generates the right error
  2. Have two CI's: one with it installed, one with it not installed

That at least mimics real-world conditions.

Copy link
Member

@bkmartinjr bkmartinjr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couple of minor suggestions/issues

@nguyenv nguyenv force-pushed the viviannguyen/optionally-use-tiledb-ctx branch from 257b94b to 85f4a5d Compare November 25, 2024 21:30
Copy link
Member

@bkmartinjr bkmartinjr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, this is much cleaner. TY @nguyenv

@nguyenv nguyenv merged commit cd1e487 into main Nov 26, 2024
11 checks passed
@nguyenv nguyenv deleted the viviannguyen/optionally-use-tiledb-ctx branch November 26, 2024 01:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants