[dtypes] FixedIncrements Fails with New Numerical Data Types

[pvk-developer]

Error Description

The FixedIncrements constraint fails to apply to new numerical data types (Int, UInt) due to a TypeError: unsupported operand type(s) for &: 'bool' and 'float'. This error originates from RDT’s FloatFormatter. However it has to do with the following line:

    def _transform(self, table_data):
->        table_data[self.column_name] = table_data[self.column_name] / self.increment_value
        return table_data

This line changes the dtype of the column, therefore this is no longer UInt or Int, it gets converted to a float dtype because of the division being applied.

Steps to reproduce

import pandas as pd
import numpy as np


data = {
    'UInt8': pd.Series([1, 2, 3], dtype='UInt8') * 10,
    'UInt16': pd.Series([1, 2, 3], dtype='UInt16')* 10,
    'UInt32': pd.Series([1, 2, 3], dtype='UInt32')* 10,
    'UInt64': pd.Series([1, 2, 3], dtype='UInt64')* 10,
}
from sdv.metadata import SingleTableMetadata
metadata = SingleTableMetadata()

# Add the fields to the metadata
metadata.add_column('UInt8', sdtype='numerical', computer_representation='UInt8')
metadata.add_column('UInt16', sdtype='numerical', computer_representation='UInt16')
metadata.add_column('UInt32', sdtype='numerical', computer_representation='UInt32')
metadata.add_column('UInt64', sdtype='numerical', computer_representation='UInt64')



df = pd.DataFrame(data)
print(df.dtypes)

from sdv.single_table import GaussianCopulaSynthesizer

gcs = GaussianCopulaSynthesizer(metadata)

my_constraints = [
    {
        'constraint_class': 'FixedIncrements',
        'constraint_parameters': {
            'column_name': column,
            'increment_value':  10
            }
    }
    for column in df.columns
]

gcs.add_constraints(my_constraints)
gcs.fit(df)

File ~/.virtualenvs/SDV/lib/python3.10/site-packages/rdt/transformers/numerical.py:106, in FloatFormatter._validate_values_within_bounds(self, data)
    104 def _validate_values_within_bounds(self, data):
    105     if self.computer_representation != 'Float':
--> 106         fractions = data[~data.isna() & data % 1 != 0]
    107         if not fractions.empty:
    108             raise ValueError(
    109                 f"The column '{data.name}' contains float values {fractions.tolist()}. "
    110                 f"All values represented by '{self.computer_representation}' must be integers."
    111             )

Additional Context

If this can't be fixed directly here, file a new issue pointing to this one in RDT and apply any fixes required there.

[amontanez24]

The code in this issue still seems to crash