Adding feature Truncate to DataFrame and Series #11435

VamsiTallam95 · 2022-08-02T17:47:18Z

Description

This PR closes #9629 by adding truncate feature to DataFrame and Series. Truncates a DataFrame or Series before and after some index value. If the index being truncated contains only datetime values, before and after may be specified as strings instead of Timestamps.

Checklist

I am familiar with the Contributing Guidelines.
New or existing tests cover these changes.
The documentation is up to date with these changes.

bdice

Hi @VamsiTallam95, thanks for the PR! I have some preliminary feedback. Please let me know if you have any questions and I look forward to iterating on this feature with you!

bdice · 2022-08-02T18:53:23Z

python/cudf/cudf/core/dataframe.py

@@ -2886,6 +2886,112 @@ def axes(self):
        """
        return [self._index, self._data.to_pandas_index()]

+    @_cudf_nvtx_annotate
+    def truncate(self, before=None, after=None, axis=0, copy=True):


This method probably belongs in python/cudf/cudf/core/indexed_frame.py if the implementation is identical for DataFrame and Series. Those methods will be inherited by both DataFrame and Series.

I tried to move the code to indexed_frame.py and was not successful due to missing functionalities.

What functionality was missing? Did it attempt to call functions that don't exist in Series? If so, we may just need to move them up from DataFrame.

This is done now.

python/cudf/cudf/core/dataframe.py

python/cudf/cudf/tests/test_dataframe.py

python/cudf/cudf/core/series.py

brandon-b-miller · 2022-08-03T12:48:13Z

In addition to the great suggestions from @bdice , could we add truncate to the relevant rst's so they show up in the docs? Thanks again and great work :)

Co-authored-by: Bradley Dice <[email protected]>

codecov · 2022-08-17T21:25:20Z

Codecov Report

Base: 87.47% // Head: 88.11% // Increases project coverage by +0.63% 🎉

Coverage data is based on head (9127a42) compared to base (f817d96).
Patch has no changes to coverable lines.

❗ Current head 9127a42 differs from pull request most recent head b3bea25. Consider uploading reports for the commit b3bea25 to get more accurate results

Additional details and impacted files

@@               Coverage Diff                @@
##           branch-22.12   #11435      +/-   ##
================================================
+ Coverage         87.47%   88.11%   +0.63%     
================================================
  Files               133      135       +2     
  Lines             21826    22021     +195     
================================================
+ Hits              19093    19404     +311     
+ Misses             2733     2617     -116

Impacted Files	Coverage Δ
python/cudf/cudf/io/text.py	`91.66% <0.00%> (-8.34%)`	⬇️
python/strings_udf/strings_udf/__init__.py	`76.47% <0.00%> (-7.85%)`	⬇️
python/cudf/cudf/core/_base_index.py	`81.28% <0.00%> (-4.27%)`	⬇️
python/cudf/cudf/io/json.py	`92.06% <0.00%> (-2.68%)`	⬇️
python/cudf/cudf/utils/utils.py	`89.91% <0.00%> (-0.69%)`	⬇️
python/dask_cudf/dask_cudf/core.py	`73.72% <0.00%> (-0.41%)`	⬇️
python/cudf/cudf/io/parquet.py	`90.45% <0.00%> (-0.39%)`	⬇️
python/dask_cudf/dask_cudf/backends.py	`84.90% <0.00%> (-0.37%)`	⬇️
python/cudf/cudf/core/column/datetime.py	`89.62% <0.00%> (-0.09%)`	⬇️
python/cudf/cudf/io/orc.py	`92.94% <0.00%> (-0.09%)`	⬇️
... and 37 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

VamsiTallam95 · 2022-08-18T21:56:35Z

rerun tests

VamsiTallam95 · 2022-08-19T16:05:27Z

rerun tests

bdice

I see @vyasr re-requested my review but there are previous comments from my last review that are still open. @VamsiTallam95 Can you take a look at those comments and reply / address them, please?

vyasr · 2022-09-01T19:36:56Z

@bdice apologies, it looked like @VamsiTallam95 was just waiting on reviews to continue. It looks like he is currently blocked, though?

github-actions · 2022-10-01T21:03:21Z

This PR has been labeled inactive-30d due to no recent activity in the past 30 days. Please close this PR if it is no longer required. Otherwise, please respond with a comment indicating any updates. This PR will be labeled inactive-90d if there is no activity in the next 60 days.

GPUtester · 2022-11-04T18:14:25Z

Can one of the admins verify this patch?

Admins can comment ok to test to allow this one PR to run or add to allowlist to allow all future PRs from the same author to run.

vyasr · 2022-11-04T18:15:37Z

ok to test

Whoops I don't think I can do that, will request.

python/cudf/cudf/core/indexed_frame.py

python/cudf/cudf/tests/test_series.py

bdice · 2022-11-04T19:07:21Z

ok to test

python/cudf/cudf/core/indexed_frame.py

bdice

Final request/comment here: #11435 (comment)

Otherwise approved!

vyasr · 2022-11-04T23:01:20Z

@gpucibot merge

In #11435, the `truncate` API was added but I had a review comment(to add it docs) that I forgot to publish. This PR adds `truncate` to the docs page. Authors: - GALI PREM SAGAR (https://github.com/galipremsagar) Approvers: - Bradley Dice (https://github.com/bdice) URL: #12109

VamsiTallam95 added 4 commits August 1, 2022 12:52

Added Truncate Functionality and test cases

9ab9e90

style and docstring fix

1c5082c

Added truncate functionality to dataframe

0b28765

cleaner version

4194427

VamsiTallam95 requested a review from a team as a code owner August 2, 2022 17:47

VamsiTallam95 requested review from bdice and brandon-b-miller August 2, 2022 17:47

github-actions bot added the Python Affects Python cuDF API. label Aug 2, 2022

VamsiTallam95 added feature request New feature or request non-breaking Non-breaking change labels Aug 2, 2022

bdice reviewed Aug 2, 2022

View reviewed changes

bdice changed the base branch from branch-22.08 to branch-22.10 August 2, 2022 21:41

brandon-b-miller reviewed Aug 3, 2022

View reviewed changes

python/cudf/cudf/core/series.py Outdated Show resolved Hide resolved

VamsiTallam95 and others added 9 commits August 17, 2022 08:34

Update python/cudf/cudf/tests/test_dataframe.py

2fd181d

Co-authored-by: Bradley Dice <[email protected]>

Update python/cudf/cudf/core/dataframe.py

b69f306

Co-authored-by: Bradley Dice <[email protected]>

Update python/cudf/cudf/tests/test_dataframe.py

2d6b20e

Co-authored-by: Bradley Dice <[email protected]>

Update python/cudf/cudf/core/series.py

2d2271e

Co-authored-by: Bradley Dice <[email protected]>

Update python/cudf/cudf/core/series.py

2734bd2

Co-authored-by: Bradley Dice <[email protected]>

Update python/cudf/cudf/core/series.py

ddede29

Co-authored-by: Bradley Dice <[email protected]>

Update python/cudf/cudf/core/series.py

31f6696

Co-authored-by: Bradley Dice <[email protected]>

incorporating feedback

42fb830

fixing failed tests

033f27e

VamsiTallam95 added 3 commits August 18, 2022 10:39

incorporating feedback

3f740bc

fixing style

42b4c7f

fixing doctests

900110f

bdice requested changes Aug 30, 2022

View reviewed changes

github-actions bot added the inactive-30d label Oct 1, 2022

bdice changed the base branch from branch-22.10 to branch-22.12 October 17, 2022 21:01

vyasr added 4 commits November 2, 2022 16:36

Merge remote-tracking branch 'origin/branch-22.12' into truncate

7ce214c

Fix Series indexing bug when given a length 1 list of slices.

9a24b15

Move method into IndexedFrame.py.

530b54b

Document that copy=False is not supported.

e6446e8

vyasr requested a review from bdice November 4, 2022 18:33

bdice requested changes Nov 4, 2022

View reviewed changes

bdice reviewed Nov 4, 2022

View reviewed changes

python/cudf/cudf/core/indexed_frame.py Outdated Show resolved Hide resolved

bdice reviewed Nov 4, 2022

View reviewed changes

python/cudf/cudf/core/indexed_frame.py Outdated Show resolved Hide resolved

vyasr added 2 commits November 4, 2022 12:35

Address PR comments.

2e6ba3b

Remove conditional on copy.

515fa22

bdice approved these changes Nov 4, 2022

View reviewed changes

Shrink test.

b3bea25

rapids-bot bot merged commit 9df2eba into rapidsai:branch-22.12 Nov 4, 2022

galipremsagar mentioned this pull request Nov 9, 2022

[REVIEW] Add truncate API to python doc pages #12109

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding feature Truncate to DataFrame and Series #11435

Adding feature Truncate to DataFrame and Series #11435

VamsiTallam95 commented Aug 2, 2022

bdice left a comment

bdice Aug 2, 2022 •

edited

Loading

VamsiTallam95 Aug 30, 2022

vyasr Sep 1, 2022

vyasr Nov 4, 2022

brandon-b-miller commented Aug 3, 2022

codecov bot commented Aug 17, 2022 •

edited

Loading

VamsiTallam95 commented Aug 18, 2022

VamsiTallam95 commented Aug 19, 2022

bdice left a comment

vyasr commented Sep 1, 2022

github-actions bot commented Oct 1, 2022

GPUtester commented Nov 4, 2022

vyasr commented Nov 4, 2022 •

edited

Loading

bdice commented Nov 4, 2022

bdice left a comment •

edited

Loading

vyasr commented Nov 4, 2022

Adding feature Truncate to DataFrame and Series #11435

Adding feature Truncate to DataFrame and Series #11435

Conversation

VamsiTallam95 commented Aug 2, 2022

Description

Checklist

bdice left a comment

Choose a reason for hiding this comment

bdice Aug 2, 2022 • edited Loading

Choose a reason for hiding this comment

VamsiTallam95 Aug 30, 2022

Choose a reason for hiding this comment

vyasr Sep 1, 2022

Choose a reason for hiding this comment

vyasr Nov 4, 2022

Choose a reason for hiding this comment

brandon-b-miller commented Aug 3, 2022

codecov bot commented Aug 17, 2022 • edited Loading

Codecov Report

VamsiTallam95 commented Aug 18, 2022

VamsiTallam95 commented Aug 19, 2022

bdice left a comment

Choose a reason for hiding this comment

vyasr commented Sep 1, 2022

github-actions bot commented Oct 1, 2022

GPUtester commented Nov 4, 2022

vyasr commented Nov 4, 2022 • edited Loading

bdice commented Nov 4, 2022

bdice left a comment • edited Loading

Choose a reason for hiding this comment

vyasr commented Nov 4, 2022

bdice Aug 2, 2022 •

edited

Loading

codecov bot commented Aug 17, 2022 •

edited

Loading

vyasr commented Nov 4, 2022 •

edited

Loading

bdice left a comment •

edited

Loading