[CT-115] Automate Changelog

[emmyoop]

Describe the Feature

Our current process for maintaining an accurate CHANGELOG is very manual and prone to conflicts.

Current Painpoints

• causes merge conflicts when merging/backporting changelog file changes
• hard to know what section to add an entry in the changelog — e.g. what version of the project will this change make it into, coordinating between minor and patch versions, rc’s and betas
• strictly a manual process, it is the responsibility of the PR author to add a changelog entry
• there is no way to enforce formatting and syntax
• you have to create the PR in order to link the PR and issue to the changelog entry
• always asking “does this deserve a changelog entry?”
• we now have a GitHub action to backport PRs, it won’t be useful if there are always merge conflicts due to the changelog file 😥

Automating the process should resolve these issues and make it easier to add a new CHANGELOG entry, reduce all the conflicts we manually resolve and help the new Github action to back port PRs works much more reliably.

Goals

• Automatically generate CHANGELOG entry from a few user inputs
• Keep current CHANGELOG format
• Stop all the back porting merge conflicts (and generally conflicts) caused by the CHANGELOG so that out Githib action for back porting will be more successful

Let's try out changie to see what it's like.

@kwigley's changes is still a wip but we can check it out once it has had more added to it!

Who will this benefit?

Everyone that contributes to dbt-core. We all have CHANGELOG entires!

Tasks to complete this

• add changie to project
• add github action to check if changelog entry exists - possibly run changie new when not
• add github action for release process
• script to add contributors section (or upgrade changie if new release comes out)
• script to handle prerelease collapse on final release
• update onboarding notion doc
• update contributing doc
• update release notion doc

[joellabes]

👀 Following for selfish reasons!

[emmyoop]

changie is missing the functionality to have a Contributors section at the bottom of each release. We can easily collect the data needed for the section but the batch command won't create that aggregated section for Contributors as we have it today. I've started a discussion with the maintainer and it looks like something he's interested in adding but needs to think through the implications. He is definitely interested in building a maintainable product and not breaking (backwards compatibility) which is a good sign.

[leahwicz]

@emmyoop so you are saying that we can collect all the Contributors with Changie but we can't create that aggregated section? Can we create something that isn't an aggregated section? Just like add a person's handle along with their change in the file? Even if we can't, I would be ok with manually entering in contributors as a starting point and then waiting to see if Changie adds the functionality or we switch to Kyle's option

[emmyoop]

@leahwicz that's exactly right. I was going to try it out just like you suggested to make sure everything else works like we need.

To provide a bit more context, changie already has custom fields that you can add via the yaml file

custom:
- key: Author
  label: GitHub Name
  type: string
  minLength: 3
- key: Issue
  label: GitHub Issue Number
  type: int
  minLength: 4

And then you can just refer to them when you define what the CHANGELOG entry will look like:

changeFormat: '- {{.Body}} ([#{{.Custom.Issue}}](https://github.com/dbt-labs/dbt-core/issues/{{.Custom.Issue}})) [@{{.Custom.Author}}](https://github.com/{{.Custom.Author}}) ([#{{.Custom.Issue}}](https://github.com/dbt-labs/dbt-core/issues/{{.Custom.Issue}}))'

resulting in

Added the ability to do something cool! (#1234) @emmyoop (#1234)

So we could essentially automatically add it as part of the line and just copy/paste it into its own section (manually or maybe with some script we write...) after running the batch command until a better solution comes up. Overall seems like less work still.

Edit: fixed changeFormat syntax errors