[BUG] fastparquet compatibility tests fail with data mismatch if TZ is not set and system timezone is not UTC

[jlowe]

Running the fastparquet compatibility tests will fail if the TZ environment variable is not set. I suspect this is because after #9652 we can tell the JVM to use a timezone that could be out-of-sync with what the native code thinks the timezone is.

Here's an example of how I reproduced the errors:

$ echo $TZ

$ date
Fri 17 Nov 2023 09:36:39 AM CST
$ SPARK_HOME=/home/jlowe/spark-3.4.1-bin-hadoop3/ integration_tests/run_pyspark_from_build.sh -k test_reading_file_written --pyarrow_test
[...]
FAILED ../../src/main/python/fastparquet_compatibility_test.py::test_reading_file_written_by_spark_cpu[Timestamp(not_null)0][DATAGEN_SEED=1700171382, INJECT_OOM] - AssertionError: GPU and CPU timestamp values are different at [0, 'a']
FAILED ../../src/main/python/fastparquet_compatibility_test.py::test_reading_file_written_with_gpu[Timestamp(not_null)0][DATAGEN_SEED=1700171382, INJECT_OOM] - AssertionError: GPU and CPU timestamp values are different at [0, 'a']

The tests pass if I export TZ=UTC or fail due to a fallback to the CPU (which is appropriate for the current state of the plugin) if I explicitly set TZ=America/Chicago.

[revans2]

I just ran into the problem with test_timestamp_{seconds,millis,micros}. I didn't set TZ explicitly to UTC, and python thought we were in one time zone, while java thought we were in UTC. This resulted in them failing with an error that the minimum value generated by the tests for a long we before the year 0.