Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

COST-1201: Enable Trino on a per source basis #2745

Merged
merged 7 commits into from
Mar 23, 2021
Merged

Conversation

dccurtis
Copy link
Contributor

@dccurtis dccurtis commented Mar 22, 2021

This changes introduces a new environment variable list ENABLE_TRINO_SOURCES. This contains a list (comma separated) of source uuids that will be processed with the Trino processing pipeline.

Testing

  1. Start koku-trino with ENABLE_PARQUET_PROCESSING=False. Ingest an AWS source and verify that Trino is not used.
  2. Add a GCP source and add the source-uuid to the ENABLE_TRINO_SOURCES list. Restart koku-worker to load variable and hit /download. Verify source is processed with Trino
  3. Add OCP source and add UUID to trino list. Verify processing is done with Trino.

Test Results
cost_1201_ut.txt

e2e-deploy: https://github.com/RedHatInsights/e2e-deploy/pull/2899

@dccurtis dccurtis self-assigned this Mar 22, 2021
@codecov
Copy link

codecov bot commented Mar 23, 2021

Codecov Report

Merging #2745 (a7e5df7) into master (0c68f06) will decrease coverage by 0.0%.
The diff coverage is 87.5%.

@@           Coverage Diff            @@
##           master   #2745     +/-   ##
========================================
- Coverage    94.7%   94.7%   -0.0%     
========================================
  Files         289     289             
  Lines       22365   22376     +11     
  Branches     2550    2551      +1     
========================================
+ Hits        21177   21181      +4     
- Misses        717     722      +5     
- Partials      471     473      +2     

@dccurtis dccurtis marked this pull request as ready for review March 23, 2021 01:14
@dccurtis dccurtis merged commit ed51b81 into master Mar 23, 2021
@maskarb maskarb deleted the trino_source_enable branch September 2, 2021 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants