Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compiled "analysis" code writing to wrong target directory #2173

Closed
4 tasks
ninashayne opened this issue Mar 1, 2020 · 5 comments · Fixed by #2349
Closed
4 tasks

compiled "analysis" code writing to wrong target directory #2173

ninashayne opened this issue Mar 1, 2020 · 5 comments · Fixed by #2349
Labels
bug Something isn't working good_first_issue Straightforward + self-contained changes, good for new contributors!

Comments

@ninashayne
Copy link

ninashayne commented Mar 1, 2020

Describe the bug

I have a custom path other than "analysis" for some SQL that I want compiled by not run. I ran

  • dbt clean
  • dbt deps
  • dbt compile
    The SQL files that are in my custom analysis path were successfully compiled. However the path where the compiled code lives is: /target/compiled/customer_data_model/analysis. I'd expect it to be in /target/compiled/customer_data_model/custom-analysis-path

Steps To Reproduce

  • I set my analysis path in my dbt_project to "test_paths."
  • I created a model in the directory "test_paths" called "test.sql" with "select 1 as test" as the SQL
  • I ran dbt clean, dbt deps, dbt compile
  • The test.sql model was compiled but the resulting compiled code lives in target/compiled/customer_data_model/analysis

Expected behavior

I'd expect the compiled code to live in target/compiled/customer_data_model/test_paths

Screenshots and log output

If applicable, add screenshots or log output to help explain your problem.

System information

Which database are you using dbt with?

  • [x ] postgres
  • redshift
  • bigquery
  • snowflake
  • other (specify: ____________)

The output of dbt --version:

0.15.2

The operating system you're using:
10.15.3

The output of python --version:
3.7.4

Additional context

Add any other context about the problem here.

@ninashayne ninashayne added bug Something isn't working triage labels Mar 1, 2020
@drewbanin drewbanin removed the triage label Mar 19, 2020
@drewbanin
Copy link
Contributor

Thanks for making this issue @ninashayne! I think this is a good suggestion, but I'm sort of on the fence about what the exact behavior should be. Do you think we should:

  1. never include resource-type subdirectories (analysis/, tests/, etc)?
  2. always include resource-type subdirectories?

I could either imagine throwing away the top-level folder name, or instead adding it to the path.

That would mean that models would live in:

target/compiled/customer_data_model/models/...

And analyses would live in:

target/compiled/customer_data_model/analysis/...

I think that always adding the top-level dir is probably a good idea, but I'm curious to hear what you think! Whatever we choose, we should just make sure to be consistent.

@ninashayne
Copy link
Author

I like the idea of always adding it to the path since instinctively that's what i thought would happen. I just reread the initial issue and realize I left the expected subdirectory off in my initial summary (which might've triggered your question). I like the idea of the compiled code mimicking the resource subdirectory structure.

To give a bit more color to why this even came up -- I want to use the compiled output of some analyses in a downstream process. Keeping them in their own directory helps with organization and standardization (my team can just add additional queries to a given directory without worrying about updating anything else). The downstream process can then just pick up the compiled SQL based on the name of the directory, without needing to know the names of specific analyses and also without picking up everything.

@ninashayne
Copy link
Author

I realize what happened... the markdown compilation hid the part that said 'custom-analysis-path'

@drewbanin
Copy link
Contributor

huh, ok @ninashayne - I read this issue totally differently this morning. I buy what you're saying here and i think we can make it happen :)

@drewbanin drewbanin added the good_first_issue Straightforward + self-contained changes, good for new contributors! label Mar 19, 2020
@ninashayne
Copy link
Author

Thanks @drewbanin! Apologies for the confusion :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working good_first_issue Straightforward + self-contained changes, good for new contributors!
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants