[FEA] Fixed-point Decimal type support

[jrhemstad]

Is your feature request related to a problem? Please describe.
cuDF should support fixed-point decimal types.

Describe the solution you'd like
Add a new DECIMAL type for columns.

The scale information will be stored per-column in the data_type.

Also requires a new scaled_integer type for encapsulating an integral value and a scale factor that provides arithmetic operators to allow operating on the fixed point values. E.g.,

template <typename Rep>
class scaled_integer{
   // Note that the scale must be runtime info
   scaled_integer(Rep value, int32_t scale);
};

While Arrow only supports 128 bit fixed-point decimals, this design will allow us to have 32bit, 64bit, etc.

Describe alternatives you've considered
The scaled_integer type from CNL won't work because it requires the scale information to be compile time info.

Additional context
In order to support Decimal types with 128 bits of precision (like Arrow), we'll also need a 128 bit integer type.

List of PR Breakdowns:

Major PRS:

• 1. Add fixed_point type ([REVIEW] Add fixed_point class to support DecimalType #3782)
• 2. Initial fixed_point Column Support ([REVIEW] Initial fixed_point Column Support #5704)
  • sort / gather
  • scatter
  • replace
  • upper_bound / lower_bound
  • interleave
  • concatenate
  • adding type decimal32 and decimal64 to relevant type lists
• 3. Move fixed_point::scale to data_type ([REVIEW] fixed_point Column Optimization (store scale in data_type) [skip ci] #5861)
  • Add scale into data_type
  • Customize allocation factory functions (to only allocate for underlying value)
  • Specialize algorithms to that don't need to operate on full fixed_point type (i.e sort)

Minor PRs:

• Enable more unit tests to work with FixedWidthTypes (instead of FixedWidthTypesWithoutFixedPoint) ([REVIEW] Enable more fixed_point unit tests by introducing "scale-less" constructor [skip ci] #5817)
• Enable fixed_point binary operations ([REVIEW] Enable fixed_point binary operations #6528)
  • Add cudf::binary_operation NULL_MIN, NULL_MAX & NULL_EQUALS for decimal32 and decimal64 (Add cudf::binary_operation NULL_MIN, NULL_MAX & NULL_EQUALS for decimal32 and decimal64 #7119)
• Implement cudf::round decimal32 & decimal64 (HALF_UP and HALF_EVEN) ([REVIEW] Implement cudf::round decimal32 & decimal64 (HALF_UP and HALF_EVEN) #6685)
• Casting to/from floats, ints and decimal with other precision/accuracy
  • Implement cudf::cast for decimal32/64 to/from integer and floating point (Implement cudf::cast for decimal32/64 to/from integer and floating point #6711)
  • Implement cudf::cast for decimal32/64 to/from different scale (Implement cudf::cast for decimal32/64 to/from different type_id #6729)
• Unary operators (abs, ceil, floor) (Implement cudf::unary_operation for decimal32 & decimal64 #6777)
• Unary operators MOD, PMOD and TRUE_DIV [FEA] Provide fixed-point support for MOD, PMOD and TRUE_DIV #7132
  • Support for TRUE_DIV (Add support for cudf::binary_operation TRUE_DIV for decimal32 and decimal64 #7198)
  • Support for (P)MOD (Support for MOD, PMOD and PYMOD for decimal32/64/128 #10179)
• Aggregation operators
  • cudf::reduce : MIN, MAX, SUM, PRODUCT, NUNIQUE (Implement cudf::reduce for decimal32 and decimal64 (part 1) #6814)
  • cudf::reduce : others (Implement cudf::reduce for decimal32 and decimal64 (part 2) #6980)
  • cudf::scan (cudf::scan support for decimal32 and decimal64 #7063)
  • cudf::groupby
    • Implement cudf::group_by (sort) for decimal32 and decimal64 (Implement cudf::group_by (sort) for decimal32 and decimal64 #7169)
    • Implement cudf::group_by (hash) for decimal32 and decimal64 (Implement cudf::group_by (hash) for decimal32 and decimal64 #7190)
  • cudf::rolling_window (Implement cudf::rolling for decimal32 and decimal64 #7037)
    • cudf::rolling_window ROW_NUMBER support for decimal32 and decimal64 (cudf::rolling ROW_NUMBER support for decimal32 and decimal64 #7061)
    • cudf::rolling_window SUM support for decimal32 and decimal64 (cudf::rolling_window SUM support for decimal32 and decimal64 #7147)
  • cudf::grouped_rolling_window
• Support for cudf::copy_range algorithm (Implement cudf::copy_range for decimal32 and decimal64 #6843)
• Support for cudf::clamp algorithm (Implement cudf::clamp for decimal32 and decimal64 #6792)
• Support for cudf::detail::copy_if / cudf::apply_boolean_mask ([REVIEW] Enable copy_if for fixed-point decimal columns #6805)
• Support for cudf::copy_if_else algorithm (Implement cudf::copy_if_else for decimal32 and decimal64 #6845)
• Add scale and value methods to fixed_point (Add scale and value methods to fixed_point #7109)
• to_arrow / from_arrow [FEA] Fixed-point support for to_arrow and from_arrow #7116
• Add null mask fixed_point_column_wrapper constructors (Add null mask fixed_point_column_wrapper constructors #6951)
• Make cudf::round for fixed_point when scale = -decimal_places a no-op (Make cudf::round for fixed_point when scale = -decimal_places a no-op #6975)
• Adding unit tests for fixed_point with extremely large scales (Adding unit tests for fixed_point with extremely large scales #7178)
• Casting decimal<-->string [FEA] string conversion to/from decimal values #7285

[codereport]

Summarising some of the design choices that were made offline with @jrhemstad and @harrism:

• Radix type should be a compile time variable
• Radix type should be programmed generically, but cuDF is mostly concerned with base 10
• Constructors should support integer and float to start
• Constructors shouldn't allow implicit conversions (mark explicit)
• Arithmetic operators should only support scaled_integer types
• Want to support exponent types with different values for arithmetic operations unless it proves to be too difficult
• Use arrow for a reference: https://arrow.apache.org/docs/cpp/classarrow_1_1_decimal128.html

Updated class might look something like:

using scale_type = int32_t;

// Rep = representative type
template <typename Rep, typename Radix>
class scaled_integer {
    template <typename T,
              typename std::enable_if_t<std::is_same<T, float>::value>* = nullptr>
    scaled_integer(T value, scale_type scale) {
        // implementation for int
    }

    template <typename T,
              typename std::enable_if_t<std::is_same<T, int>::value>* = nullptr>
    scaled_integer(T value, scale_type scale) {
        // implementation for float
    }
};

and then:

using decimal32 = scaled_integer<int32_t, 10>;

[harrism]

Do you need the SFINAE shown here, or can you just specialize the template constructors (since the specifializations are for specific single types and they are not partial specializations)?

[jrhemstad]

Do you need the SFINAE shown here, or can you just specialize the template constructors (since the specifializations are for specific single types and they are not partial specializations)?

Yeah, if you're only interested in int/float then just use overloads:

   scaled_integer(int value, scale_type scale) {
        // implementation for int
    }

    scaled_integer(float value, scale_type scale) {
        // implementation for float
    }

That would require 6 overloads (int8,16,32,64,float32,64).

I'd do SFINAE on traits like is_integer/is_floating_point

[kkraus14]

Looks like the only thing missing here is the mod / pmod operators and the cudf::grouped_rolling_window support. We already have a separate issue for the mod / pmod operators, so I think we should raise an issue for cudf::grouped_rolling_window and then we can close this out and follow up with any gaps in subsequent issues.

[kkraus14]

Closing as I've opened #8161 for tracking cudf::grouped_rolling_window libcudf support.