Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-17733: [C++] Take index_width into account when filling nulls in index buffer #14129

Merged
merged 5 commits into from
Sep 16, 2022
Merged

ARROW-17733: [C++] Take index_width into account when filling nulls in index buffer #14129

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3251535
add failing test
rasnjo Sep 14, 2022
8d5ef87
fix offset bug
rasnjo Sep 14, 2022
204981d
add run of nulls to make test more robust
rasnjo Sep 14, 2022
401622c
fix linting error
rasnjo Sep 14, 2022
8d02222
add JIRA number in comment for test
rasnjo Sep 16, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions cpp/src/arrow/array/concatenate.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -311,8 +311,8 @@ class ConcatenateImpl {
/*dest_offset=*/position, run.length,
transpose_map));
} else {
std::fill(out_data + position,
out_data + position + (run.length * index_width), 0x00);
std::fill(out_data + (position * index_width),
out_data + (position + run.length) * index_width, 0x00);
}

position += run.length;
Expand Down
11 changes: 11 additions & 0 deletions cpp/src/arrow/array/concatenate_test.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -539,4 +539,15 @@ TEST_F(ConcatenateTest, OffsetOverflow) {
ASSERT_RAISES(Invalid, Concatenate({fake_long, fake_long}).status());
}

TEST_F(ConcatenateTest, DictionaryConcatenateWithEmptyUint16) {
// Regression test for ARROW-17733
auto dict_type = dictionary(uint16(), utf8());
Copy link
Member

@pitrou pitrou Sep 15, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a comment here mentioning the JIRA number?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, done now (commit: 8d02222)

auto dict_one = DictArrayFromJSON(dict_type, "[]", "[]");
auto dict_two =
DictArrayFromJSON(dict_type, "[0, 1, null, null, null, null]", "[\"A0\", \"A1\"]");
ASSERT_OK_AND_ASSIGN(auto concat_actual, Concatenate({dict_one, dict_two}));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: it might be worth concatenating with a run of a few nulls?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good suggestion, makes sense. Done in commit: 204981d


AssertArraysEqual(*dict_two, *concat_actual);
}

} // namespace arrow