Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closes #3870: bug in reshape for bigint type #3907

Merged

Conversation

ajpotts
Copy link
Contributor

@ajpotts ajpotts commented Nov 20, 2024

Fixes a bug where reshape function gives the incorrect result for the bigint type, as shown in the following example:

In [3]: dtype = "bigint"

In [4]: size = 10

In [5]:         x = ak.arange(size, dtype=dtype).reshape((1, size, 1))

In [6]: x
Out[6]: array([array([array([0]) array([1]) array([2]) array([3]) array([4]) array([224251899497]) array([1919968626]) array([78401563153491]) array([127961796865894]) array([111052610564705])])])

Fixes the issue by creating a separate flatten and unflatten function in AryUtil.chpl for the bigint case.

Closes #3870: bug in reshape for bigint type
Closes #3908: flatten for bigint

@ajpotts
Copy link
Contributor Author

ajpotts commented Nov 20, 2024

I believe the cause of the bug is this section the unflatten procedure:

            get(
              c_ptrTo(unflat[low]),
              getAddr(a[flatSlice.low]),
              locInStart,
              c_sizeof(t) * flatSlice.size
            );

I think the fact that BigInt does not have constant size, the data cannot be bulk copied using the address.

The fix was write a separate unflatten procedure for the bigint case.

@ajpotts ajpotts closed this Nov 21, 2024
@ajpotts ajpotts reopened this Nov 21, 2024
@ajpotts ajpotts force-pushed the 3870_bug_in_reshape_for_bigint_type branch 2 times, most recently from d05990e to e3da525 Compare November 21, 2024 15:39
@ajpotts ajpotts marked this pull request as ready for review November 21, 2024 18:25
@ajpotts ajpotts force-pushed the 3870_bug_in_reshape_for_bigint_type branch 2 times, most recently from 6a79ee1 to 604ad95 Compare November 21, 2024 20:31
@ajpotts ajpotts force-pushed the 3870_bug_in_reshape_for_bigint_type branch from 68b0269 to f131cca Compare December 3, 2024 19:34
Copy link
Contributor

@drculhane drculhane left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. You'll want to rename test_reshape_bug_reproducer :)

@ajpotts ajpotts force-pushed the 3870_bug_in_reshape_for_bigint_type branch from cbbe9c4 to c46bf70 Compare December 3, 2024 21:19
@ajpotts ajpotts force-pushed the 3870_bug_in_reshape_for_bigint_type branch from c46bf70 to 16c15a4 Compare December 3, 2024 21:20
Copy link
Contributor

@jaketrookman jaketrookman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reapproving after formatting changes

@ajpotts ajpotts added this pull request to the merge queue Dec 3, 2024
Merged via the queue into Bears-R-Us:master with commit cc1797a Dec 3, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

flatten for bigint bug in reshape for bigint type
3 participants