Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[c++] Apply subarrays for dense reads and writes #3263

Merged
merged 7 commits into from
Oct 31, 2024
Merged

Conversation

johnkerl
Copy link
Member

@johnkerl johnkerl commented Oct 30, 2024

Issue and/or context: As tracked on issue #2407 / [sc-51048]. See also #3364.

Note that the intended Python and R API changes are all agreed on and finalized as described in #2407.

Changes:

As found on #3244:

  • The main context of [c++/python] Fixes for dense arrays and core 2.27/dev #3244 is:
    • For dense arrays only
    • If any dim's subarray range isn't specified, core uses the core domain
    • With new-shape feature-flag enabled, and with core 2.27 where dense arrays can have current domain (they couldn't in 2.26), we are indeed using data-sized core current domain and huge core domain
    • Put those together and we get these problems
    • Note there are a few other issues on this PR, and a few not yet included in this PR, other than that -- and those should be split out into separate PRs
  • On this PR I've ben finding callsites, for write and for read, for Python and R, and touching up the ranges which end up being core subarrays on each dim slot
    • This is tedious and error-prone
  • What we should do instead is:
    • In one place -- in libtiledbsoma, in ManagedQuery, before submit_query
    • If any dim's range has not been set up to this point, and if the array has a current domain, simply set the ranges on that dim to be the current domain's (lo, hi)
    • We know (for a fact) that core will go ahead and use the too-big domain anyway so this is safe

Notes for Reviewer:

@johnkerl johnkerl marked this pull request as draft October 30, 2024 18:39
@johnkerl johnkerl changed the title [c++] Apply subarrays for dense reads and writes [c++] Apply subarrays for dense reads and writes [WIP] Oct 30, 2024
Copy link

codecov bot commented Oct 30, 2024

Codecov Report

Attention: Patch coverage is 84.61538% with 2 lines in your changes missing coverage. Please review.

Project coverage is 83.81%. Comparing base (130ccc6) to head (febaa7a).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3263      +/-   ##
==========================================
+ Coverage   83.79%   83.81%   +0.01%     
==========================================
  Files          51       51              
  Lines        5567     5578      +11     
==========================================
+ Hits         4665     4675      +10     
- Misses        902      903       +1     
Flag Coverage Δ
python 83.81% <84.61%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
python_api 83.81% <84.61%> (+0.01%) ⬆️
libtiledbsoma ∅ <ø> (∅)

@johnkerl johnkerl force-pushed the kerl/dense-range-trim branch from c2e9a8c to fc3b726 Compare October 31, 2024 16:15
@johnkerl johnkerl force-pushed the kerl/dense-range-trim branch from fc3b726 to 631ff1b Compare October 31, 2024 16:18
@johnkerl johnkerl changed the title [c++] Apply subarrays for dense reads and writes [WIP] [c++] Apply subarrays for dense reads and writes Oct 31, 2024
@johnkerl johnkerl requested a review from nguyenv October 31, 2024 17:41
@johnkerl johnkerl marked this pull request as ready for review October 31, 2024 17:42
@johnkerl
Copy link
Member Author

@nguyenv ready for review 🙏

libtiledbsoma/src/soma/managed_query.cc Outdated Show resolved Hide resolved
libtiledbsoma/src/soma/managed_query.h Outdated Show resolved Hide resolved
@johnkerl
Copy link
Member Author

Thanks @nguyenv !!! :)

@johnkerl johnkerl merged commit 4373050 into main Oct 31, 2024
@johnkerl johnkerl deleted the kerl/dense-range-trim branch October 31, 2024 20:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants