Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable unified memory by default in cudf_polars #17375

Merged
merged 27 commits into from
Nov 22, 2024
Merged

Enable unified memory by default in cudf_polars #17375

merged 27 commits into from
Nov 22, 2024

Conversation

galipremsagar
Copy link
Contributor

@galipremsagar galipremsagar commented Nov 19, 2024 •
edited
Loading

Description

This PR enables Unified memory as the default memory resource for cudf_polars

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@github-actions github-actions bot added Python Affects Python cuDF API. cudf.polars Issues specific to cudf.polars labels Nov 19, 2024
@galipremsagar galipremsagar added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Nov 19, 2024
vyasr
vyasr reviewed Nov 20, 2024
View reviewed changes
@galipremsagar galipremsagar marked this pull request as ready for review November 20, 2024 22:37
@galipremsagar galipremsagar requested a review from a team as a code owner November 20, 2024 22:37
@galipremsagar galipremsagar changed the title Config to enable uvm in polars Switch cudf_polars to use Unified Memory by default Nov 20, 2024
Matt711
Matt711 reviewed Nov 20, 2024
View reviewed changes
python/cudf_polars/cudf_polars/callback.py Outdated
@@ -32,6 +33,37 @@
__all__: list[str] = ["execute_with_cudf"]


def enable_uvm(device: int):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could be missing some context, but can you simplify this function given that rmm_mode depends only on managed_memory_is_supported and its never equal to "cuda"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, we don't need to support all the modes like we do in cudf.pandas. In this case we should do a managed pool by default and fall back to the async allocator if not.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done 👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function needs a docstring. Also, the name is misleading, it doesn't "enable uvm", it produces a memory resource.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved everything into default_memory_resource

@vyasr vyasr changed the title Switch cudf_polars to use Unified Memory by default Config to enable uvm in polars Nov 20, 2024
vyasr
vyasr requested changes Nov 21, 2024
View reviewed changes
python/cudf_polars/cudf_polars/callback.py Outdated
@@ -32,6 +33,37 @@
__all__: list[str] = ["execute_with_cudf"]


def enable_uvm(device: int):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, we don't need to support all the modes like we do in cudf.pandas. In this case we should do a managed pool by default and fall back to the async allocator if not.

@galipremsagar galipremsagar requested a review from wence- November 21, 2024 14:26
vyasr
vyasr approved these changes Nov 21, 2024
View reviewed changes
Copy link
Contributor

@vyasr vyasr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two suggestions but assuming they're uncontroversial I don't need to review again. Please ping me if you need another look though. Thanks Prem!

wence-
wence- approved these changes Nov 21, 2024
View reviewed changes
Copy link
Contributor

@wence- wence- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving, but please address remaining comments.

@galipremsagar
Copy link
Contributor Author

/merge

@galipremsagar galipremsagar added the 5 - Ready to Merge Testing and reviews complete, ready to merge label Nov 21, 2024
@galipremsagar
Copy link
Contributor Author

/merge

@galipremsagar galipremsagar changed the title Config to enable uvm in polars Enable unified memory by default in cudf_polars Nov 22, 2024
@raydouglass raydouglass merged commit 305182e into rapidsai:branch-24.12 Nov 22, 2024
120 of 121 checks passed
@vyasr vyasr mentioned this pull request Nov 25, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vyasr vyasr vyasr approved these changes

@Matt711 Matt711 Matt711 left review comments

@wence- wence- wence- approved these changes

Assignees

@vyasr vyasr

@galipremsagar galipremsagar

@Matt711 Matt711

Labels
5 - Ready to Merge Testing and reviews complete, ready to merge cudf.polars Issues specific to cudf.polars improvement Improvement / enhancement to an existing function non-breaking Non-breaking change Python Affects Python cuDF API.
Projects
cuDF Python
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@galipremsagar @wence- @vyasr @Matt711 @raydouglass