[CT-1035] [Feature] dbt deps should take require-dbt-version into account when resolving dependencies

[joellabes]

Is this your first time submitting a feature request?

• I have read the expectations for open source contributors
• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion

Describe the feature

Because dbt deps doesn't take require-dbt-version into account when resolving dependencies, package vendors have to ship many more versions of packages to thread the needles of compatibility themselves.

👇 This is a lazy example because I couldn't be bothered installing an old version of dbt Core, but the actual use case is an old Core installation trying to get the newest utils version.

# packages.yml
packages:
  - package: dbt-labs/dbt_utils
    version: "<0.8.0"

Running dbt deps with the above code on dbt Core 1.2.0 works fine, even though utils 0.7.6 has a require-dbt-version of require-dbt-version: [">=0.20.0", "<1.1.0"]. At runtime, we get an error:

  Failed to read package: Runtime Error
    This version of dbt is not supported with the 'dbt_utils' package.
      Installed version of dbt: =1.2.0
      Required version of dbt for 'dbt_utils': ['>=0.20.0', '<1.1.0']
    Check for a different version of the 'dbt_utils' package, or run dbt again with --no-version-check

---

(Please assume that dbt utils 0.9.0 is already out and requires dbt Core 1.2 or above)

As the maintainer of a package like dbt-audit-helper, I'd like to be able to change my utils dependency from [">=0.8.0", "<0.9.0"] to [">=0.8.0", "<1.0.0"]. Then, I would like people whose packages.yml looks like this:

# packages.yml
packages:
  - package: dbt-labs/audit_helper
    version: [">=0.5.0", "<0.6.0"]

to get the correct version of dbt utils for their Core version: utils 0.8.latest for Core 1.0 or 1.2, and 0.9.latest for 1.2 and above.

Describe alternatives you've considered

Releasing a new version of audit helper which only allows utils 0.9.x. As an end user this is a problem because it means that every package that I depend on has to update at the same time, or I am trapped on an older version for longer.

Doing a lot more backwards compatibility work in dbt utils so that 0.9 can also work with any Core 1.x version. This would solve the most pressing issue but I imagine that similar issues will arise in the future, especially as we get to utils 1.0 and beyond which might need to start taking advantage of new functionality added in minor versions of Core, but we want to have wide open dependency ranges to minimise thrash in the ecosystem.

Who will this benefit?

Package installers and vendors

Are you interested in contributing this feature?

Anything else?

No response

[jtcohen6]

@joellabes Yes. I think the time has finally come: #5651