-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[python/c++] COO to CSX conversion optimization #3304
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3304 +/- ##
==========================================
+ Coverage 85.14% 85.52% +0.37%
==========================================
Files 53 54 +1
Lines 5568 5686 +118
==========================================
+ Hits 4741 4863 +122
+ Misses 827 823 -4
Flags with carried forward coverage won't be shown. Click here to find out more.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm partway through but need to be AFK. Poking the submit button on where I am now, as a checkpoint.
@@ -0,0 +1,406 @@ | |||
/** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe this is simply a copy from core of known-good code -- if so I won't even peek at it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
correct - vendored, as noted in the PR description
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for my less-than-thorough read of the PR description!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please see #3328 (review) for narrative.
Thanks @bkmartinjr !
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Some minor suggestions
Issue and/or context:
Using scipy.sparse.spmatrix for conversion from SOMA Arrow tables to SciPy csr_matrix is single-threaded and requires contiguous COO input arrays. This PR:
All API/modules are for internal use. There is no external API surface change.
Changes:
soma.io.to_anndata
and theblockwise()
reader-O3 -mavx2
to setup.pyLater work will wire into additional code paths.
Issue: #3345 [sc-59595]