-
Notifications
You must be signed in to change notification settings - Fork 919
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
API Doc for Polars GPU Engine #16753
Changes from 18 commits
5dd765e
4f11aaa
7bab15f
311deb3
8bf9cd2
b7dd2d8
6a2ffb5
2925068
afdfe5b
093ce0c
a8ac87e
ac4ecf0
8d27437
1d21c92
87e9810
c95f440
9048a91
f22acc7
f47a035
2a6a844
9b21c66
79ac363
ca3517c
c006acd
c395baa
e39539f
4606930
c4ef847
7e71672
f076bd4
36bb894
71ea75a
bb66967
d403419
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
cuDF-based GPU backend for Polars [Open Beta] | ||
============================================= | ||
|
||
cuDF supports an in-memory, GPU-accelerated execution engine for Python users of the Polars Lazy API. | ||
The engine supports most of the core expressions and data types as well as a growing set of more advanced dataframe manipulations | ||
and data file formats. When using the GPU engine, Polars will convert expressions into an optimized query plan and determine | ||
whether the plan is supported on the GPU. If it is not, the execution will transparently fall back to the standard Polars engine | ||
and run on the CPU. | ||
|
||
Benchmark | ||
--------- | ||
We reproduced the `Polars Decision Support (PDS) <https://github.com/pola-rs/polars-benchmark>`_ benchmark to compare Polars GPU engine with the default CPU settings across several dataset sizes. Here are the results: | ||
bdice marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
.. figure:: ../_static/pds_benchmark_polars.png | ||
:width: 600px | ||
|
||
|
||
The per-query speedups using Polars GPU Engine for the 80 GB dataset are below: | ||
|
||
|
||
.. figure:: ../_static/Polars_GPU_speedup_80GB.png | ||
:width: 1000px | ||
|
||
You can reproduce the results by visiting the `Polars Decision Support (PDS) Github repository. <https://github.com/pola-rs/polars-benchmark>`_ | ||
bdice marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Learn More | ||
---------- | ||
|
||
The GPU backend for Polars is now available in Open Beta and the engine is undergoing rapid development. To learn more, visit the `GPU Support page <https://docs.pola.rs/user-guide/gpu-support/>`_ on the Polars website. | ||
bdice marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
Launch on Google Colab | ||
---------------------- | ||
|
||
.. figure:: ../_static/colab.png | ||
:width: 200px | ||
:target: https://colab.research.google.com/github/rapidsai-community/showcase/blob/main/accelerated_data_processing_examples/polars_gpu_engine_demo.ipynb | ||
|
||
Take the cuDF backend for Polars for a test-drive in a free GPU-enabled notebook environment using your Google account by `launching on Colab <TBD>`_ | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Reminder to fix this before merging! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think I fixed this. I assume it's supposed to point to https://colab.research.google.com/github/rapidsai-community/showcase/blob/main/accelerated_data_processing_examples/polars_gpu_engine_demo.ipynb. If that's incorrect please fix it. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Y axis label should be “Speedup (Polars CPU runtime / Polars GPU runtime)”