Skip to content

Commit

Permalink
Fix for appending decimal128 under list and struct types (#10105)
Browse files Browse the repository at this point in the history
I know that this is past the freeze date. This is a fix for a P1 bug that we just found when trying to build Scalar values of Lists and Structs that contain Decimal128 values. We might be able to work around it some other way, but it would take a lot of changes to the existing Spark plugin code to do that so I wanted to try this first.

Authors:
   - Robert (Bobby) Evans (https://github.com/revans2)

Approvers:
   - Kuhu Shukla (https://github.com/kuhushukla)
   - Niranjan Artal (https://github.com/nartal1)
  • Loading branch information
revans2 authored Jan 21, 2022
1 parent 53a31d1 commit 5a4c5f3
Showing 1 changed file with 9 additions and 5 deletions.
14 changes: 9 additions & 5 deletions java/src/main/java/ai/rapids/cudf/HostColumnVector.java
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/*
*
* Copyright (c) 2020-2021, NVIDIA CORPORATION.
* Copyright (c) 2020-2022, NVIDIA CORPORATION.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -1136,6 +1136,8 @@ private void appendChildOrNull(ColumnBuilder childBuilder, Object listElement) {
childBuilder.append((Short) listElement);
} else if (listElement instanceof BigDecimal) {
childBuilder.append((BigDecimal) listElement);
} else if (listElement instanceof BigInteger) {
childBuilder.append((BigInteger) listElement);
} else if (listElement instanceof List) {
childBuilder.append((List<?>) listElement);
} else if (listElement instanceof StructData) {
Expand Down Expand Up @@ -1230,18 +1232,20 @@ public final ColumnBuilder append(boolean value) {
return this;
}

public final ColumnBuilder append(BigDecimal value) {
public ColumnBuilder append(BigDecimal value) {
return append(value.setScale(-type.getScale(), RoundingMode.UNNECESSARY).unscaledValue());
}

public ColumnBuilder append(BigInteger unscaledVal) {
growBuffersAndRows(false, currentIndex * type.getSizeInBytes() + type.getSizeInBytes());
assert currentIndex < rows;
// Rescale input decimal with UNNECESSARY policy, which accepts no precision loss.
BigInteger unscaledVal = value.setScale(-type.getScale(), RoundingMode.UNNECESSARY).unscaledValue();
if (type.typeId == DType.DTypeEnum.DECIMAL32) {
data.setInt(currentIndex * type.getSizeInBytes(), unscaledVal.intValueExact());
} else if (type.typeId == DType.DTypeEnum.DECIMAL64) {
data.setLong(currentIndex * type.getSizeInBytes(), unscaledVal.longValueExact());
} else if (type.typeId == DType.DTypeEnum.DECIMAL128) {
assert currentIndex < rows;
byte[] unscaledValueBytes = value.unscaledValue().toByteArray();
byte[] unscaledValueBytes = unscaledVal.toByteArray();
byte[] result = convertDecimal128FromJavaToCudf(unscaledValueBytes);
data.setBytes(currentIndex*DType.DTypeEnum.DECIMAL128.sizeInBytes, result, 0, result.length);
} else {
Expand Down

0 comments on commit 5a4c5f3

Please sign in to comment.