[PyROOT][DF] Improve RDataFrame.AsNumpy performance #6496
Conversation
* Fixed broken memory adoption formely triggered by numpy.array. Switched to the numpy.asarray adoption mechanism. * Added tests confirming the adoption of the memory from the C++ objects
|
Starting build on |
|
Build failed on windows10/cxx14. Errors:
|
|
Build failed on ROOT-performance-centos7-multicore/default. Failing tests: |
| @@ -76,7 +76,7 @@ def RDataFrameAsNumpy(df, columns=None, exclude=None): | |||
| for column in columns: | |||
| cpp_reference = result_ptrs[column].GetValue() | |||
| if hasattr(cpp_reference, "__array_interface__"): | |||
| tmp = numpy.array(cpp_reference) # This adopts the memory of the C++ object. | |||
| tmp = numpy.asarray(cpp_reference) # This adopts the memory of the C++ object. | |||
There was a problem hiding this comment.
Looks good @stwunsch ! So only asarray adopts now and array copies?
There was a problem hiding this comment.
I've checked the adoption when we put this into legacy pyroot, but we haven't had a test for it. For some reason it seems that the numpy.array constructor does not adopt anymore. I'm pretty sure it did before! However, np.asarray does and we check this now.
| df = ROOT.ROOT.RDataFrame(1).Define("x", "1.0") | ||
| npy = df.AsNumpy(["x"]) | ||
| pyarr = npy["x"] | ||
| cpparr = pyarr.result_ptr.GetValue() |
There was a problem hiding this comment.
This result_ptr is RDataFrame's result pointer that you store?
numpy.arrayWe should merge first the respective benchmarks in rootbench. The PR is ongoing here: root-project/rootbench#195