chore: enforce changelog entries on PR reviews #25459

emilyrohrbough · 2023-01-13T15:07:09Z

With this change, this updates the Semantic PR pre-fixes (semantic types) to align with Cypress's changelog sections:

Changes has user-facing impact:
- breaking - A breaking change that will require a MVB
- dependency - A change to a dependency that impact the user
- deprecation - A API deprecation notice for users
- feat - A new feature
- fix - A bug fix or regression fix.
- misc - a misc user-facing change, like a UI update which is not a fix or enhancement to how Cypress works
- perf - A code change that improves performance
Changes that improves the codebase or system but has no user-facing impact:
- chore - Changes to the build process or auxiliary tools and libraries such as documentation generation
- docs - Documentation only changes
- refactor - A code change that neither fixes a bug nor adds a feature
- revert - Reverts a previous commit
- test - Adding missing or correcting existing tests

The npm release details have been updated to account for these new semantic types.
Change log entries are loosely linted on all pull requests.
The entire change log is loosely linted on develop, release branch and any PR or branch that bumps the Cypress binary version in ./package.json

How has the user experience changed?

PRs must include a changelog entry for all user-facing changes.
Semantic PR title's prefix must match the changelog entry's

Examples:

Runs for example PR (forked): #25460

Missing PR type: https://github.com/cypress-io/cypress/actions/runs/3912637056
Wrong PR type: https://github.com/cypress-io/cypress/actions/runs/3912647093
Failed for single commit non-semantic: https://github.com/cypress-io/cypress/actions/runs/3912656072
Passing check for non-user facing change: https://github.com/cypress-io/cypress/actions/runs/3912670277
Missing changelog entry: https://github.com/cypress-io/cypress/actions/runs/3912676488/jobs/6687620920
Changelog entry does not match pr title:
- https://github.com/cypress-io/cypress/actions/runs/3990406084/jobs/6844005062
Passing check for user-facing changes: https://github.com/cypress-io/cypress/actions/runs/3990470114

Runs for example release-branch PR (forked): #25461

validates entire changelog: fix(TEST PR ONLY): changes in driver #25460

QUESTION

I am wondering if it would be better to add this content to https://github.com/cypress-io/release-automations. We would want to release these changes. There are scripts in release-automations (comment-on-issues) that we will want to pull forward when we write the release workflow.

Thoughts?

PR Tasks

Have tests been added/updated?
[n/a] Has the original issue (or this PR, if no issue exists) been tagged with a release in ZenHub? (user-facing changes only)
[n/a] Has a PR for user-facing changes been opened in cypress-documentation?
[n/a] Have API changes been updated in the type definitions?

…ing out

cypress · 2023-01-13T15:39:31Z

Test summary

26600 • 0 • 1274 • 0 • 45

Run details


Project	cypress
Status	Passed
Commit	`0504525`
Started	Jan 24, 2023 9:08 PM
Ended	Jan 24, 2023 9:26 PM
Duration	17:02 💡
OS	Linux Debian -
Browser	Multiple

View run in Cypress Dashboard ➡️

Flakiness

	create-from-component.cy.ts	1
1	... > runs generated spec
	commands/net_stubbing.cy.ts	4
1	network stubbing > intercepting request > can delay and throttle a StaticResponse
2	... > stops waiting when an fetch request is canceled
3	network stubbing > intercepting request > can delay and throttle a StaticResponse
4	... > with `times` > only uses each handler N times
This comment includes only the first 5 flaky tests. See all 45 flaky tests in the Cypress Dashboard.

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. You can manage this integration in this project's settings in the Cypress Dashboard

emilyrohrbough · 2023-01-13T19:17:22Z

guides/release-process.md

@@ -84,6 +78,16 @@ In the following instructions, "X.Y.Z" is used to denote the [next version of Cy
        - [cypress-realworld-app](https://github.com/cypress-io/cypress-realworld-app) uses yarn and represents a typical consumer implementation.
    - Optionally, do more thorough tests, for example test the new version of Cypress against the Cypress Cloud repo.

+2. Confirm that every issue labeled [stage: pending release](https://github.com/cypress-io/cypress/issues?q=label%3A%22stage%3A+pending+release%22+is%3Aclosed) has a ZenHub release set. **Tip:** there is a command in [`release-automations`](https://github.com/cypress-io/release-automations)'s `issues-in-release` tool to list and check such issues. Without a ZenHub release issues will not be included in the right changelog. Also ensure that every closed issue in any obsolete releases are moved to the appropriate release in ZehHub. For example, if the open releases are 9.5.5 and 9.6.0, the current release is 9.6.0, then all closed issues marked as 9.5.5 should be moved to 9.6.0. Ensure that there are no commits on `develop` since the last release that are user facing and aren't marked with the current release.
+
+3. Create a Release PR Bump, submit, get approvals on, and merge a new PR. This PR Should:


This actually covers ##25162 as well.

BlueWinds · 2023-01-18T15:57:03Z

https://github.com/cypress-io/cypress/actions/runs/3912690128/jobs/6687653259 (SHOULD HAVE FAILED -- TOFIX)
Passing check for user-facing changes: TODO after above fix.

From PR description. Fixed and example?

BlueWinds

Can we also add "write a changelog entry" to the "PR Tasks" part of the pull request template? I think it would be a helpful reminder.

I'm also unclear on the format of changelog entries. When I make a PR, exactly what and where do I add to cli/CHANGELOG.md? In this PR there's the code

## 12.4.0

__Released 01/17/2023 (PENDING)__

**Features:**

When I make a PR, how do I determine what I should be writing in here (what headers, formatting, etc)?

https://github.com/cypress-io/cypress/pull/25460/files looks to be an example of the format expected. What's the process for adding new sections? After 12.4.0 goes out, how does 12.5.0 or 12.4.1 get added? When and where do we switch between 12.4.1 -> 12.5.0 -> 13.0.0 - is that the responsibility of the first PR that adds a patch / feature / breaking change to develop?

guides/release-process.md

mjhenkes · 2023-01-18T16:10:36Z

.releaserc.js

-  ...require('./.releaserc.base'),
+  plugins: [
+    ['@semantic-release/commit-analyzer', {
+      preset: 'angular',


preset angular? That seems odd?

Yeah it does. The @semantic-release/commit-analyzer uses angular by default. Our release rules are built on top of the angular release rules and we use the angular changelog syntax. Under the hood, this tool expects a preset, a custom npm configuration or will fallback to angular: https://github.com/semantic-release/commit-analyzer/blob/master/lib/load-parser-config.js#L20

mjhenkes · 2023-01-18T16:28:14Z

guides/release-process.md

+2. Confirm that every issue labeled [stage: pending release](https://github.com/cypress-io/cypress/issues?q=label%3A%22stage%3A+pending+release%22+is%3Aclosed) has a ZenHub release set. **Tip:** there is a command in [`release-automations`](https://github.com/cypress-io/release-automations)'s `issues-in-release` tool to list and check such issues. Without a ZenHub release issues will not be included in the right changelog. Also ensure that every closed issue in any obsolete releases are moved to the appropriate release in ZehHub. For example, if the open releases are 9.5.5 and 9.6.0, the current release is 9.6.0, then all closed issues marked as 9.5.5 should be moved to 9.6.0. Ensure that there are no commits on `develop` since the last release that are user facing and aren't marked with the current release.
+
+3. Create a Release PR Bump, submit, get approvals on, and merge a new PR. This PR Should:
+ bump the Cypress version, fix an changelog entries and to bump the `cypress-example-kitchensink`](https://github.com/cypress-io/cypress-example-kitchensink/releases) version.


Suggested change

bump the Cypress version, fix an changelog entries and to bump the `cypress-example-kitchensink`](https://github.com/cypress-io/cypress-example-kitchensink/releases) version.

- Bump the Cypress version, fix any changelog entries and to bump the `cypress-example-kitchensink`](https://github.com/cypress-io/cypress-example-kitchensink/releases) version.

mjhenkes · 2023-01-18T17:26:07Z

guides/release-process.md

@@ -84,6 +78,16 @@ In the following instructions, "X.Y.Z" is used to denote the [next version of Cy
        - [cypress-realworld-app](https://github.com/cypress-io/cypress-realworld-app) uses yarn and represents a typical consumer implementation.
    - Optionally, do more thorough tests, for example test the new version of Cypress against the Cypress Cloud repo.

+2. Confirm that every issue labeled [stage: pending release](https://github.com/cypress-io/cypress/issues?q=label%3A%22stage%3A+pending+release%22+is%3Aclosed) has a ZenHub release set. **Tip:** there is a command in [`release-automations`](https://github.com/cypress-io/release-automations)'s `issues-in-release` tool to list and check such issues. Without a ZenHub release issues will not be included in the right changelog. Also ensure that every closed issue in any obsolete releases are moved to the appropriate release in ZehHub. For example, if the open releases are 9.5.5 and 9.6.0, the current release is 9.6.0, then all closed issues marked as 9.5.5 should be moved to 9.6.0. Ensure that there are no commits on `develop` since the last release that are user facing and aren't marked with the current release.
+
+3. Create a Release PR Bump, submit, get approvals on, and merge a new PR. This PR Should:


After this step, is it ok for developers to merge into develop again? I think so, but that assumes nothing goes wrong with the release.

I think most of the time yes, but we have had releases where the binary doesn't work as expected or the docker-image tests have failed and we needed to do quick fixes to move forward with release. However, at that point-in-time, a new NPM release is required with a new version bump so we'd need a new commit anyways and if someone merges content, well lucky them!

SO I do think we could remove the develop hold since real failures with windows/etc should be corrected on this PR if using the release branch naming convention.

mjhenkes · 2023-01-18T18:15:19Z

scripts/get-next-version.js

+      if (!commit.type || !changeCatagories[commit.type]) return
+
+      if (changeCatagories[commit.type].release === 'major') {
+        breakings += 1


breakings could be a bool right?

breakings is just counting the number of breaking changes for the reason below: There is ${breakings} BREAKING CHANGE and ${features} features. Just a nice-to-have insight into why the level is 0 or 1 or 2.

Co-authored-by: Blue F <[email protected]>

mjhenkes

Looks good to me, Maybe add some more comments to your scripts to explain what you're doing.

emilyrohrbough · 2023-01-23T21:04:08Z

@BlueWinds PR summary & examples have been updated.

I pulled forward the Managing the Changelog document and added to this repo. This covers how to format the changelog entries, what happened on release, and to do if an 'unreleased' section has not been added (yes this will be a long-running document, which will give writing style context over time).

Given we are linting for changelog entries and will require it to pass before merging, I don't believe a check on the template is needed because this information is provided to users if they don't include it.

* develop: (27 commits) refactor: remove unused cloud routes (#25584) chore: fix issue template formatting issue (#25587) fix: fixed issue with CT + electron + run mode not exiting properly (#25585) chore(deps): update dependency ua-parser-js to v0.7.33 [security] (#25561) fix: add alternative binary names for edge-beta (#25456) chore: add batch execution to CloudDataSource (#22457) chore: End a/b campaigns for aci smart banners (#25504) chore: release @cypress/schematic-v2.5.0 fix(cypress-schematic): do not disable e2e support file (#25400) chore: adding memory issue template (#25559) feat: Add Angular CT Schematic (#24374) chore: enforce changelog entries on PR reviews (#25459) chore: bump package.json to 12.4.0 [run ci] (#25556) feat: Add 'type' option to `.as` to store aliases by value (#25251) chore: release @cypress/webpack-dev-server-v3.2.3 feat: Display line break in cy.log (#25467) chore: update types (#25538) fix: Revert "fix: adding emergency garbage collection for chromium-based browsers" (#25546) fix: percy - wait to take snapshot until previous tooltips are gone (#25522) feat: support data-qa selector in selector playground (#25475) ...

cypress-bot · 2023-01-27T15:39:53Z

Released in 12.4.1.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v12.4.1, please open a new issue.

emilyrohrbough added 12 commits January 11, 2023 16:16

chore: add semantic-pull request title and changelog entry check

50f61f4

add changelog validation and update details to be more coherent

63c00fa

Merge branch 'develop' into enforce-changelog-entries

8d79094

add changelog validation to circle workflow

f2669ef

run ci

dd617cf

run ci

2b23388

run ci

a9f961f

naming is important.

196131c

run ci

9fcf94f

run ci

83ef4cf

fix a few things and small refactor to collect all errors before bail…

a06c688

…ing out

clean up

8cb317f

mjhenkes self-requested a review January 13, 2023 15:14

emilyrohrbough added 2 commits January 13, 2023 09:21

checkout forked repo

f5c8a88

correct file paths would be helpful.

c33e960

emilyrohrbough added 3 commits January 13, 2023 10:39

real change log entries

6fe9f4c

more sophisticated changelog parsing.

e2a75b1

and update release process

0b1eff2

emilyrohrbough commented Jan 13, 2023

View reviewed changes

fix recommended bump logic

0a94dd4

AtofStryker self-requested a review January 13, 2023 19:46

BlueWinds reviewed Jan 18, 2023

View reviewed changes

guides/release-process.md Outdated Show resolved Hide resolved

mjhenkes reviewed Jan 18, 2023

View reviewed changes

Update guides/release-process.md

b7d9281

Co-authored-by: Blue F <[email protected]>

mjhenkes approved these changes Jan 18, 2023

View reviewed changes

emilyrohrbough added 2 commits January 23, 2023 14:04

add comments

822fda1

Add docs for writing the changelog & tie into contributing guide.

526c9c9

emilyrohrbough requested a review from BlueWinds January 23, 2023 21:04

Merge branch 'develop' into enforce-changelog-entries

109c969

BlueWinds approved these changes Jan 23, 2023

View reviewed changes

emilyrohrbough added 3 commits January 24, 2023 14:39

add 12.4.0 changelog for ref

52c84ba

Merge branch 'develop' into enforce-changelog-entries

23a08e5

fix prasing

0504525

emilyrohrbough merged commit a869d5d into develop Jan 24, 2023

emilyrohrbough deleted the enforce-changelog-entries branch January 24, 2023 21:30

cypress-bot bot locked as resolved and limited conversation to collaborators Jan 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: enforce changelog entries on PR reviews #25459

chore: enforce changelog entries on PR reviews #25459

emilyrohrbough commented Jan 13, 2023 •

edited

Loading

cypress bot commented Jan 13, 2023 •

edited

Loading

emilyrohrbough Jan 13, 2023

BlueWinds commented Jan 18, 2023 •

edited

Loading

BlueWinds left a comment •

edited

Loading

mjhenkes Jan 18, 2023

emilyrohrbough Jan 18, 2023

mjhenkes Jan 18, 2023 •

edited

Loading

mjhenkes Jan 18, 2023

emilyrohrbough Jan 18, 2023

mjhenkes Jan 18, 2023

emilyrohrbough Jan 18, 2023

mjhenkes left a comment

emilyrohrbough commented Jan 23, 2023

cypress-bot bot commented Jan 27, 2023

	bump the Cypress version, fix an changelog entries and to bump the `cypress-example-kitchensink`](https://github.com/cypress-io/cypress-example-kitchensink/releases) version.

	- Bump the Cypress version, fix any changelog entries and to bump the `cypress-example-kitchensink`](https://github.com/cypress-io/cypress-example-kitchensink/releases) version.

chore: enforce changelog entries on PR reviews #25459

chore: enforce changelog entries on PR reviews #25459

Conversation

emilyrohrbough commented Jan 13, 2023 • edited Loading

How has the user experience changed?

Examples:

QUESTION

PR Tasks

cypress bot commented Jan 13, 2023 • edited Loading

Test summary

Run details

Flakiness

emilyrohrbough Jan 13, 2023

Choose a reason for hiding this comment

BlueWinds commented Jan 18, 2023 • edited Loading

BlueWinds left a comment • edited Loading

Choose a reason for hiding this comment

mjhenkes Jan 18, 2023

Choose a reason for hiding this comment

emilyrohrbough Jan 18, 2023

Choose a reason for hiding this comment

mjhenkes Jan 18, 2023 • edited Loading

Choose a reason for hiding this comment

mjhenkes Jan 18, 2023

Choose a reason for hiding this comment

emilyrohrbough Jan 18, 2023

Choose a reason for hiding this comment

mjhenkes Jan 18, 2023

Choose a reason for hiding this comment

emilyrohrbough Jan 18, 2023

Choose a reason for hiding this comment

mjhenkes left a comment

Choose a reason for hiding this comment

emilyrohrbough commented Jan 23, 2023

cypress-bot bot commented Jan 27, 2023

emilyrohrbough commented Jan 13, 2023 •

edited

Loading

cypress bot commented Jan 13, 2023 •

edited

Loading

BlueWinds commented Jan 18, 2023 •

edited

Loading

BlueWinds left a comment •

edited

Loading

mjhenkes Jan 18, 2023 •

edited

Loading