Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[REVIEW] BlazingSQL: Q02 - Remove ORDER BY tstamp #107

Merged

Conversation

Christian8491
Copy link
Contributor

This PR just remove ORDER BY tstamp_inSec as it is not necessary.

@Christian8491
Copy link
Contributor Author

Tested at sf 1K on the dgx201 (10 repetitions) all the executions show Correctness Assertion True


(bsql-tpcx) christianc@rl-dgx2-d17-u05-rapids-dgx201:~/tpcx-bb/tpcx_bb$ python benchmark_runner.py --config_file=/home/christianc/scripts/benchmark/benchmark_1k_w_valid.yaml 
['02']
Using default arguments
Connected to tcp://10.150.162.27:8786
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/home/christianc/miniconda3/envs/bsql-tpcx/lib/blazingsql-algebra.jar) to field java.nio.Buffer.address
WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
BlazingContext ready
Blazing Queries
02
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/pandas/util/__init__.py:12: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.
  import pandas.util.testing
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True
Standard ETL Query
/home/christianc/miniconda3/envs/bsql-tpcx/lib/python3.7/site-packages/cudf/tests/utils.py:83: FutureWarning: The 'check_less_precise' keyword in testing.assert_*_equal is deprecated and will be removed in a future version. You can stop passing 'check_less_precise' to silence this warning.
  tm.assert_frame_equal(left, right, **kwargs)
Correctness Assertion True

Also was tested at sf 10K using the RAPLAB (16 DGXs, as dgx103 have currently physical issues)

@Christian8491 Christian8491 changed the title [REVIEW] Remove ORDER BY tstamp [REVIEW] BlazingSQL: Q02 - Remove ORDER BY tstamp Sep 19, 2020
Copy link
Member

@beckernick beckernick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Good find

(wcs_click_date_sk * 86400 + wcs_click_time_sk) AS tstamp_inSec
FROM web_clickstreams
WHERE wcs_item_sk IS NOT NULL
AND wcs_user_sk IS NOT NULL
ORDER BY wcs_user_sk, tstamp_inSec
ORDER BY wcs_user_sk
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good change. The UDF logic does this [user_sk, timestamp] sort, but it doesn't need to be global so it's happening in the UDF get_sessions itself.

@beckernick beckernick merged commit bb38ddf into rapidsai:main Sep 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants