[BUG] tpcx-bb Q2 1K randomly fails validation

[wmalpica]

No description provided.

[Christian8491]

Using the dgx201 at sf 1K. (10 repeats, Validation summary: 6 False - 4 True)

BlazingContext ready
Blazing Queries
02
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/pandas/util/__init__.py:12: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  import pandas.util.testing
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Error DataFrame.iloc[:, 2] (column name="cnt") are different

DataFrame.iloc[:, 2] (column name="cnt") values are different (10.0 %)
[index]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]
[left]:  [101301, 2287720, 1899456, 2186894, 98571, 99549, 99077, 97410, 99700, 97010, 98790, 97948, 96926, 101235, 103528, 100822, 100508, 102468, 102384, 101024, 101174, 100537, 96858, 99316, 97036, 100933, 97161, 97464, 101371, 98215]
[right]: [101301, 2287721, 1899456, 2186893, 98572, 99549, 99077, 97411, 99700, 97010, 98791, 97948, 96926, 101235, 103528, 100822, 100508, 102468, 102384, 101024, 101174, 100537, 96858, 99316, 97036, 100933, 97161, 97464, 101371, 98215]
Correctness Assertion False
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Error DataFrame.iloc[:, 2] (column name="cnt") are different

DataFrame.iloc[:, 2] (column name="cnt") values are different (3.33333 %)
[index]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]
[left]:  [101301, 2287721, 1899455, 2186893, 98572, 99549, 99077, 97411, 99700, 97010, 98791, 97948, 96926, 101235, 103528, 100822, 100508, 102468, 102384, 101024, 101174, 100537, 96858, 99316, 97036, 100933, 97161, 97464, 101371, 98215]
[right]: [101301, 2287721, 1899456, 2186893, 98572, 99549, 99077, 97411, 99700, 97010, 98791, 97948, 96926, 101235, 103528, 100822, 100508, 102468, 102384, 101024, 101174, 100537, 96858, 99316, 97036, 100933, 97161, 97464, 101371, 98215]
Correctness Assertion False
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Error DataFrame.iloc[:, 2] (column name="cnt") are different

DataFrame.iloc[:, 2] (column name="cnt") values are different (3.33333 %)
[index]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]
[left]:  [101301, 2287721, 1899456, 2186893, 98572, 99549, 99077, 97411, 99699, 97010, 98791, 97948, 96926, 101235, 103528, 100822, 100508, 102468, 102384, 101024, 101174, 100537, 96858, 99316, 97036, 100933, 97161, 97464, 101371, 98215]
[right]: [101301, 2287721, 1899456, 2186893, 98572, 99549, 99077, 97411, 99700, 97010, 98791, 97948, 96926, 101235, 103528, 100822, 100508, 102468, 102384, 101024, 101174, 100537, 96858, 99316, 97036, 100933, 97161, 97464, 101371, 98215]
Correctness Assertion False
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Error DataFrame.iloc[:, 2] (column name="cnt") are different

DataFrame.iloc[:, 2] (column name="cnt") values are different (3.33333 %)
[index]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]
[left]:  [101301, 2287721, 1899456, 2186893, 98572, 99549, 99077, 97411, 99700, 97010, 98791, 97948, 96925, 101235, 103528, 100822, 100508, 102468, 102384, 101024, 101174, 100537, 96858, 99316, 97036, 100933, 97161, 97464, 101371, 98215]
[right]: [101301, 2287721, 1899456, 2186893, 98572, 99549, 99077, 97411, 99700, 97010, 98791, 97948, 96926, 101235, 103528, 100822, 100508, 102468, 102384, 101024, 101174, 100537, 96858, 99316, 97036, 100933, 97161, 97464, 101371, 98215]
Correctness Assertion False
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Error DataFrame.iloc[:, 2] (column name="cnt") are different

DataFrame.iloc[:, 2] (column name="cnt") values are different (3.33333 %)
[index]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]
[left]:  [101301, 2287721, 1899455, 2186893, 98572, 99549, 99077, 97411, 99700, 97010, 98791, 97948, 96926, 101235, 103528, 100822, 100508, 102468, 102384, 101024, 101174, 100537, 96858, 99316, 97036, 100933, 97161, 97464, 101371, 98215]
[right]: [101301, 2287721, 1899456, 2186893, 98572, 99549, 99077, 97411, 99700, 97010, 98791, 97948, 96926, 101235, 103528, 100822, 100508, 102468, 102384, 101024, 101174, 100537, 96858, 99316, 97036, 100933, 97161, 97464, 101371, 98215]
Correctness Assertion False
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Error DataFrame.iloc[:, 2] (column name="cnt") are different

DataFrame.iloc[:, 2] (column name="cnt") values are different (6.66667 %)
[index]: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]
[left]:  [101301, 2287721, 1899456, 2186892, 98572, 99549, 99077, 97410, 99700, 97010, 98791, 97948, 96926, 101235, 103528, 100822, 100508, 102468, 102384, 101024, 101174, 100537, 96858, 99315, 97036, 100933, 97161, 97464, 101371, 98215]
[right]: [101301, 2287721, 1899456, 2186893, 98572, 99549, 99077, 97411, 99700, 97010, 98791, 97948, 96926, 101235, 103528, 100822, 100508, 102468, 102384, 101024, 101174, 100537, 96858, 99316, 97036, 100933, 97161, 97464, 101371, 98215]
Correctness Assertion False

conda list, useful packages

blazingsql                0.16.0a         cuda10.2_py37_52    blazingsql-nightly/label/cuda10.2
cudf                      0.16.0a200917   cuda_10.2_py37_g858035497_1856    rapidsai-nightly
cuml                      0.16.0a200918   cuda10.2_py37_gfe7fd990a_812    rapidsai-nightly
dask-cuda                 0.16.0a200917           py37_92    rapidsai-nightly
dask-cudf                 0.16.0a200917   py37_g858035497_1856    rapidsai-nightly
faiss-proc                1.0.0                      cuda    rapidsai-nightly
libcudf                   0.16.0a200917   cuda10.2_g858035497_1856    rapidsai-nightly
libcuml                   0.16.0a200918   cuda10.2_gfe7fd990a_812    rapidsai-nightly
libcumlprims              0.16.0a200915   cuda10.2_ge147332_25    rapidsai-nightly
librmm                    0.16.0a200917   cuda10.2_gf591436_378    rapidsai-nightly
rmm                       0.16.0a200917   cuda_10.2_py37_gf591436_378    rapidsai-nightly
ucx                       1.8.1+g6b29558       cuda10.2_0    rapidsai-nightly
ucx-proc                  1.0.0                       gpu    rapidsai-nightly
ucx-py                    0.16.0a200917+g6b29558        py37_146    rapidsai-nightly

[Christian8491]

Using this version rapidsai/gpu-bdb#107 the query 02 at sf 10K shows Correctness Assertion True multiple times. Waiting for the merge of that PR to close this issue.

[Christian8491]

As the before mentioned PR was merged. This bug go away.