When we build the row validation comparison string we include an rstrip expression as described here: https://github.com/GoogleCloudPlatform/professional-services-data-validator/blob/develop/README.md#hash-concat-and-comparison-fields

As it appears we have inconsistencies in which characters are stripped I first need to survey all supported engines to see what characters are removed.

We should also discuss why we have rtrim in there at all.

It appears to me recent Ibis versions attempt to trim all white space:

https://github.com/ibis-project/ibis/blob/decb96de6d6786afe4a23f6b14e3f3053860b9c8/ibis/backends/sql/compilers/base.py#L988

   def visit_RStrip(self, op, *, arg):
        return self.f.rtrim(arg, string.whitespace)

So a lot of our inconsistencies might be due to the version of Ibis we support.

I've surveyed our current support:

System      Characters removed List controllable
----------- ------------------ ---------------------------
BigQuery    All whitespace     Yes
DB2         Space              Yes, but docs are confusing
Hive        Space              No
Impala      Space              Yes
MySQL       Space              No
Oracle      Space              Yes
PostgreSQL  All whitespace     Yes (default is space only)
Snowflake   Space              Yes
Spanner     All whitespace     Yes
SQL Server  Space              Yes
Teradata    All whitespace     Yes (default is space only)

There are maybe two problems here:

1. Our support should be consistent which probably means all RStrip expressions should attempt to remove all whitespace. Except for Hive and MySQL which might be uncontrollable.
2. Is it correct to trim white space before hashing rows? When migrating data, trailing white space is just the sort of awkward data that might be lost/problematic. Perhaps RStrip should only be included under an option and not by default.