Collect release notes from commits #322

lolski · 2021-11-16T21:49:31Z

What is the goal of this PR?

We've updated the release notes script to collect from commits rather than milestones. It will collect the commits between the last release up to the specified commit being released.

Additionally, it's been rewritten in Kotlin.

What are the changes implemented in this PR?

Update the script for collecting release note from commits, which works as follows:
- given C1, the commit that we want to release, collect L, the list of commits that we want to add to the release note:
  - if this is the first release ever, L would contain all the commits from C1 down to and including the first commit in the repo
  - if this is not the first release:
    - find C2, the commit of the release that immediately precedes this release
    - L would contain all the commits from C1 down to but excluding C2
- write PR description or commit message for each commit in L into the release note
Rewrite in Kotlin

vmax · 2021-11-17T09:39:45Z

tool/release/CreateNotes.kt

+fun main(args: Array<String>) {
+    val buildWorkspaceDirEnv = "BUILD_WORKSPACE_DIRECTORY"
+    if (System.getenv(buildWorkspaceDirEnv) == null) throw RuntimeException("Not running from within Bazel workspace")
+    val workspaceDirectory = System.getenv(buildWorkspaceDirEnv)


@lolski could be collapsed with previous lines:

Suggested change

val workspaceDirectory = System.getenv(buildWorkspaceDirEnv)

val workspaceDirectory = System.getenv("BUILD_WORKSPACE_DIRECTORY")

?: throw RuntimeException("Not running from within Bazel workspace")

I see. Fixed in fa97b42.

vmax · 2021-11-17T09:40:46Z

tool/release/CreateNotes.kt

+
+    val createTokenEnv = "CREATE_NOTES_TOKEN"
+    if (System.getenv(createTokenEnv) == null) throw RuntimeException("$createTokenEnv environment variable is not set")
+    val githubToken = System.getenv(createTokenEnv)


@lolski could be collapsed with previous lines:

Suggested change

val githubToken = System.getenv(createTokenEnv)

val githubToken = System.getenv("CREATE_NOTES_TOKEN")

?: throw RuntimeException("CREATE_NOTES_TOKEN environment variable is not set")

Fixed in fa97b42.

vmax · 2021-11-17T09:46:56Z

tool/release/CreateNotes.kt

+    val repo = args[1]
+    val version = args[2]
+    val toBeReleased = args[3]
+    val releaseTemplateFile = Paths.get(workspaceDirectory, args[4])


@lolski destructuring syntax could probably be used here

Suggested change

val releaseTemplateFile = Paths.get(workspaceDirectory, args[4])

val (org, repo, version, toBeReleased, releaseTemplatePath) = args

Fixed in fa97b42.

vmax · 2021-11-17T09:51:31Z

tool/release/CreateNotes.kt

+        }
+    }
+
+    val releaseNote = """


@lolski could use better-looking indentation if you call .trimIndent() on it:

val releaseNote = """ Install & Run: http://docs.vaticle.com/docs/running-typedb/install-and-run ## New Features ${features.map { feature -> createCommitNote(feature) }.joinToString("\n")} ## Bugs Fixed ${bugs.map { bug -> createCommitNote(bug) }.joinToString("\n")} ## Code Refactor ${refactors.map { refactor -> createCommitNote(refactor) }.joinToString("\n")} ## Other Improvements ${others.map { other -> createCommitNote(other) }.joinToString("\n")} """.trimIndent() Files.write(releaseTemplateFile, releaseNote.toByteArray(UTF_8))

I tried that but it's not working properly.

This format

val markdown = """ Install & Run: $installInstruction ## New Features ${features.map(::createCommitNoteMd).joinToString("\n")} ... """.trimIndent()

resulted in an overindented first few lines:

Install & Run: http://docs.vaticle.com/docs/running-typedb/install-and-run ## New Features - **Implement rule negation validation to prevent contradictions** To properly support negation in rules, we must validate that rules cannot contradict themselves and reach a stable final solution, we require that rules containing negations do not recurse through the negation - this is called stratified negation.This PR implements the commit time cycle detection required to ban cyclical rules through negations. - **Optimise resolution of fully bound concludables** In some cases, a concludable that is being resolved may have all of its retrievable variables (named and anonymous) fully bound. This means that only a single answer is required to prove the concludable holds, rather than all possible ways. We add short circuiting to prevent exploring further rules in a concludable when a single answer has been found.

vmax · 2021-11-17T09:57:27Z

tool/release/CreateNotes.kt

+Install & Run: http://docs.vaticle.com/docs/running-typedb/install-and-run
+
+## New Features
+${features.map { feature -> createCommitNote(feature) }.joinToString("\n")}


@lolski could use method reference here:

Suggested change

${features.map { feature -> createCommitNote(feature) }.joinToString("\n")}

${features.map(::createCommitNote).joinToString("\n")}

Fixed in fa97b42.

vmax · 2021-11-17T09:57:49Z

tool/release/CreateNotes.kt

+${features.map { feature -> createCommitNote(feature) }.joinToString("\n")}
+
+## Bugs Fixed
+${bugs.map { bug -> createCommitNote(bug) }.joinToString("\n")}


@lolski could use method reference here:

Suggested change

${bugs.map { bug -> createCommitNote(bug) }.joinToString("\n")}

${bugs.map(::createCommitNote).joinToString("\n")}

Fixed in fa97b42.

vmax · 2021-11-17T09:58:22Z

tool/release/CreateNotes.kt

+${bugs.map { bug -> createCommitNote(bug) }.joinToString("\n")}
+
+## Code Refactor
+${refactors.map { refactor -> createCommitNote(refactor) }.joinToString("\n")}


@lolski could use method reference here:

Suggested change

${refactors.map { refactor -> createCommitNote(refactor) }.joinToString("\n")}

${refactors.map(::createCommitNote).joinToString("\n")}

Fixed in fa97b42.

vmax · 2021-11-17T09:58:51Z

tool/release/CreateNotes.kt

+${refactors.map { refactor -> createCommitNote(refactor) }.joinToString("\n")}
+
+## Other Improvements
+${others.map { other -> createCommitNote(other) }.joinToString("\n")}


@lolski could use method reference here:

Suggested change

${others.map { other -> createCommitNote(other) }.joinToString("\n")}

${others.map(::createCommitNote).joinToString("\n")}

Fixed in fa97b42.

vmax · 2021-11-17T10:08:00Z

tool/release/CreateNotes.kt

+    val releaseNote = """
+Install & Run: http://docs.vaticle.com/docs/running-typedb/install-and-run
+
+## New Features


@lolski code below could be generified with something like

val compileNotes = { notes: Iterable<CommitNote> -> notes.map(::createCommitNote).joinToString("\n") }

and then in the string e.g.:

## Bugs Fixed ${compileNotes(bugs)}

Hmm I think we should keep the current code, because your suggestion is too terse.

vmax · 2021-11-17T10:09:32Z

tool/release/CreateNotes.kt

+        .setHeaders(HttpHeaders().setAuthorization("Token $githubToken").setAccept("application/vnd.github.v3+json"))
+        .execute()
+    val body = Json.parse(String(response.content.readBytes()))
+    return body.asObject().get("commits").asArray().map { e -> e.asObject().get("sha").asString() }


@lolski Kotlin allows but doesn't require naming lambda arguments:

Suggested change

return body.asObject().get("commits").asArray().map { e -> e.asObject().get("sha").asString() }

return body.asObject().get("commits").asArray().map { it.asObject().get("sha").asString() }

Yes, I know of this, but it felt too magical and I am reluctant to use it.

…sion

… in checking for the validity of the release template file

lolski · 2021-11-18T09:56:58Z

tool/release/createnotes/Main.kt

+
+private fun getEnv(env: String): String {
+    return System.getenv(env) ?: throw RuntimeException("'$env' environment variable must be set.")
+}


lolski · 2021-11-18T11:15:30Z

tool/release/createnotes/Main.kt

+import java.nio.file.Paths
+import kotlin.io.path.notExists
+
+fun main(args: Array<String>) {


This is the entry point of the program. Given a repo, version, commit and a template file:

it will collect all the commits between the commit that is to be released to the last released commit

for each collected commit, write the title and description into the release note

lolski · 2021-11-18T11:18:17Z

tool/release/createnotes/Commit.kt

+import com.vaticle.dependencies.tool.release.createnotes.Constant.github
+import java.nio.file.Path
+
+fun getCommits(org: String, repo: String, current: Version, to: String, baseDir: Path, githubToken: String): List<String> {


This method collects all the commits between the commit that is to be released to the commit that belongs to the immediately preceding release.

If there is no preceding release found, that means this commit is the very first release, and all commits down to the first commit will be collected.

lolski · 2021-11-18T11:25:45Z

tool/release/createnotes/CommitDescription.kt

+    enum class Type { FEATURE, BUG, REFACTOR, OTHER }
+}
+
+fun getCommitDescriptions(org: String, repo: String, commits: List<String>, githubToken: String): List<CommitDescription> {


this function collects the commit description from the commit:

if the commit is created from a PR, the title and description will be used.

if the commit is directly pushed, the first line will be collected and the rest discarded. commit message may be less structured than a PR message and we do not want to accidentally add implementation detail.

now that I think about it, it's better to filter out the "implementation" section the PR description here

lolski · 2021-11-18T11:28:52Z

tool/release/createnotes/Markdown.kt

+import java.nio.file.Path
+import kotlin.text.Charsets.UTF_8
+
+fun createCommitNoteMd(description: CommitDescription): String {


make private

lolski · 2021-11-18T12:14:34Z

tool/release/createnotes/Markdown.kt

+    return "- **${description.title}**\n  $desc"
+}
+
+fun writeReleaseNoteMd(commitDescriptions: List<CommitDescription>, releaseTemplateFile: Path) {


Write the release notes:

Create the markdown string of each CommitDescription into the right section of the markdown string, ie., whether it's a feature, bug, refactor, or "other".

Write the markdown string into the file.

lolski · 2021-11-18T12:16:51Z

tool/release/createnotes/Version.kt

+
+package com.vaticle.dependencies.tool.release.createnotes
+
+data class Version(val major: Int, val minor: Int, val patch: Int, val alpha: Int?): Comparable<Version> {


This class represents our semantic version, such as 2.0.0, 2.0.0-alpha, and 2.0.0-alpha-3.

It implements Comparable because we need it to be sortable.

lolski · 2021-11-18T12:17:58Z

tool/release/createnotes/Version.kt

+                    patch = version2[2].toInt(),
+                    alpha = null
+                )
+            } else if (version1.size == 2) {


This else if branch handles when we get a string such as 2.0.0-alpha. It implies that this is the first alpha and hence set alpha = 1 when instantiating the Version class.

vmax · 2021-11-19T11:21:43Z

tool/release/createnotes/Markdown.kt

+fun createCommitNoteMd(description: CommitDescription): String {
+    val desc = StringBuilder()
+    var header = 0
+    for (line in description.desc.lines()) {


@lolski can make the implementation functional:

val desc = description.desc.lines().reversed().takeWhile { !it.startsWith("## ") }.reversed()

## What is the goal of this PR? We've updated the release notes script to collect from commits rather than milestones. It will collect the commits between the last release up to the specified commit being released. Additionally, it's been rewritten in Kotlin. ## What are the changes implemented in this PR? 1. Update the script for collecting release note from commits, which works as follows: - given C1, the commit that we want to release, collect L, the list of commits that we want to add to the release note: - if this is the first release ever, L would contain all the commits from C1 down to and including the first commit in the repo - if this is not the first release: - find C2, the commit of the release that immediately precedes this release - L would contain all the commits from C1 down to but excluding C2 - write PR description or commit message for each commit in L into the release note 2. Rewrite in Kotlin

## What is the goal of this PR? Allow assembling and deploying Crate packages ## What are the changes implemented in this PR? Implement `assemble_crate` and `deploy_crate` rules Fixes typedb#319

## What is the goal of this PR? Initial implementation of `assemble_crate` and `deploy_crate` done in typedb#322 is not fully compatible with `crates.io` registry and therefore needs to be made compatible. ## What are the changes implemented in this PR? * Make `description`, `license` and `repository` fields mandatory * Correctly handle empty `documentation` * Validate `homepage`, `repository` and `documentation` fields to be valid URLs * Validate `keywords` list to be valid * Account for possibility of validation errors on deployment

lolski added 2 commits November 15, 2021 17:50

wip

096e986

implement release notes script

21b1b86

lolski added priority: high type: feature type: refactor labels Nov 16, 2021

lolski self-assigned this Nov 16, 2021

lolski added this to the Technical Debt milestone Nov 16, 2021

Remove the old implementation

42a4ed9

vmax reviewed Nov 17, 2021

View reviewed changes

lolski added 4 commits November 17, 2021 10:29

Refactor implementation: split into several files

f626e18

Refactor http get calls

7333ea4

Refactor visibilities

b95b250

Refactor CommitDescription, Version. Fixed a bunch of bugs in the Ver…

c9ccbe4

…sion

lolski added priority: blocker and removed priority: high labels Nov 17, 2021

lolski added 4 commits November 17, 2021 13:07

Log PR numbers when collecting the note

49194f0

Remove unused comment

cd74f1f

Find and include first commit if this is the first release. Fix a bug…

92d2144

… in checking for the validity of the release template file

Refactor: use desctructuring and method reference syntax

fa97b42

lolski merged commit 1a10c8d into typedb:master Nov 17, 2021

lolski deleted the release-notes branch November 17, 2021 19:44

lolski commented Nov 18, 2021

View reviewed changes

vmax reviewed Nov 19, 2021

View reviewed changes

This was referenced Nov 19, 2021

Refactor //tool/release:create-notes to compile commit messages #316

Closed

//release:create-notes is hardcoded to the "vaticle" GitHub organisation #294

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Collect release notes from commits #322

Collect release notes from commits #322

lolski commented Nov 16, 2021 •

edited

Loading

vmax Nov 17, 2021 •

edited

Loading

lolski Nov 17, 2021

vmax Nov 17, 2021

lolski Nov 17, 2021

vmax Nov 17, 2021

lolski Nov 17, 2021

vmax Nov 17, 2021

lolski Nov 17, 2021

vmax Nov 17, 2021

lolski Nov 17, 2021

vmax Nov 17, 2021

lolski Nov 17, 2021

vmax Nov 17, 2021

lolski Nov 17, 2021

vmax Nov 17, 2021

lolski Nov 17, 2021

vmax Nov 17, 2021

lolski Nov 17, 2021

vmax Nov 17, 2021

lolski Nov 17, 2021

lolski Nov 18, 2021

lolski Nov 18, 2021

lolski Nov 18, 2021

lolski Nov 18, 2021

lolski Nov 18, 2021

lolski Nov 18, 2021

lolski Nov 18, 2021

lolski Nov 18, 2021

lolski Nov 18, 2021

vmax Nov 19, 2021

	val workspaceDirectory = System.getenv(buildWorkspaceDirEnv)
	val workspaceDirectory = System.getenv("BUILD_WORKSPACE_DIRECTORY")
	?: throw RuntimeException("Not running from within Bazel workspace")

	val githubToken = System.getenv(createTokenEnv)
	val githubToken = System.getenv("CREATE_NOTES_TOKEN")
	?: throw RuntimeException("CREATE_NOTES_TOKEN environment variable is not set")

	val releaseTemplateFile = Paths.get(workspaceDirectory, args[4])
	val (org, repo, version, toBeReleased, releaseTemplatePath) = args

	${features.map { feature -> createCommitNote(feature) }.joinToString("\n")}
	${features.map(::createCommitNote).joinToString("\n")}

	${bugs.map { bug -> createCommitNote(bug) }.joinToString("\n")}
	${bugs.map(::createCommitNote).joinToString("\n")}

	${refactors.map { refactor -> createCommitNote(refactor) }.joinToString("\n")}
	${refactors.map(::createCommitNote).joinToString("\n")}

	${others.map { other -> createCommitNote(other) }.joinToString("\n")}
	${others.map(::createCommitNote).joinToString("\n")}

	return body.asObject().get("commits").asArray().map { e -> e.asObject().get("sha").asString() }
	return body.asObject().get("commits").asArray().map { it.asObject().get("sha").asString() }


		package com.vaticle.dependencies.tool.release.createnotes

		data class Version(val major: Int, val minor: Int, val patch: Int, val alpha: Int?): Comparable<Version> {

Collect release notes from commits #322

Collect release notes from commits #322

Conversation

lolski commented Nov 16, 2021 • edited Loading

What is the goal of this PR?

What are the changes implemented in this PR?

vmax Nov 17, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lolski commented Nov 16, 2021 •

edited

Loading

vmax Nov 17, 2021 •

edited

Loading