Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with load test #267

Open
MohammedElGharawy opened this issue Dec 11, 2022 · 0 comments
Open

Problem with load test #267

MohammedElGharawy opened this issue Dec 11, 2022 · 0 comments

Comments

@MohammedElGharawy
Copy link

MohammedElGharawy commented Dec 11, 2022

After generating test data using PDGF tool and placing the .dat files under $DATA_DIR/SF
what should be the next step to run the benchmark?

running the load test generates .parquet files but when i try to run the benchmark's queries after
i get the following errors

Connected!
Pure Dask Queries
run 1: q01
Encountered Exception while running query
raise ValueError(
ValueError: The following columns were not found in the dataset {'i_category_id', 'i_item_sk'}
The following columns were found Index([], dtype='object') 

Output from load test (SF100)


Connected!
Converting warehouse of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting promotion of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting store of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting customer_demographics of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting product_reviews of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting web_clickstreams of 60 GB to 30 parquet files, chunksize: 128 MiB
Converting web_sales of 44 GB to 22 parquet files, chunksize: 128 MiB
Converting web_page of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting date_dim of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting reason of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting income_band of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting customer of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting item_marketprices of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting web_returns of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting web_site of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting store_returns of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting item of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting inventory of 34 GB to 17 parquet files, chunksize: 128 MiB
Converting household_demographics of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting time_dim of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting customer_address of 0 GB to 1 parquet files, chunksize: 128 MiB
Converting store_sales of 30 GB to 15 parquet files, chunksize: 128 MiB
Converting ship_mode of 0 GB to 1 parquet files, chunksize: 128 MiB
Load test with chunk size of 128 MiB took 537.89s 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant