Enable everything #1

SaschaMann · 2021-03-05T12:45:54Z

No description provided.

SaschaMann · 2021-03-05T12:47:28Z

synced-files/.github/labels.yml

@@ -0,0 +1,107 @@
+- name: "bug :bug:"


This is just a file I used for testing.

SaschaMann · 2021-03-05T12:48:09Z

.github/workflows/sync.yml

+          //return trackRepos.flatMap(({ full_name }) => full_name === 'exercism/research_experiment_1' ? [] : [full_name])
+
+          // TODO: Before merge, remove this and uncomment the line above
+          return ['exercism/julia']


Suggested change

//return trackRepos.flatMap(({ full_name }) => full_name === 'exercism/research_experiment_1' ? [] : [full_name])

// TODO: Before merge, remove this and uncomment the line above

return ['exercism/julia']

return trackRepos.flatMap(({ full_name }) => full_name === 'exercism/research_experiment_1' ? [] : [full_name])

^ To 'arm' the repo/workflow before merging

SaschaMann · 2021-03-05T12:50:38Z

How it works:

On each push to the main branch, all files in synced-files/ will be copied to all Exercism track repos.
A PR will be opened on each repo, which will (in the future) be automatically merged by a webhook integration.

After pushing, you have 5 minutes to abort the workflow, in case there are any errors in the files that should be synced.

iHiD · 2021-03-05T13:14:59Z

.github/workflows/sync.yml

+          })
+
+          // research_experiment_1 is not a real track, don't treat it as one
+          //return trackRepos.flatMap(({ full_name }) => full_name === 'exercism/research_experiment_1' ? [] : [full_name])


I've just removed the label from this track (cc @ErikSchierboom for your scripts too)

ErikSchierboom

It looks good to me, but it is hard to review these CI actions :)

.github/workflows/sync.yml

ErikSchierboom · 2021-03-09T08:08:53Z

.github/workflows/sync.yml

+        with:
+          github-token: ${{ secrets.BOT_PERSONAL_ACCESS_TOKEN }}
+          script: |
+            const botRepos = await github.paginate(github.repos.listForAuthenticatedUser, {


Weird that you can filter on forks. 🤷

ErikSchierboom · 2021-03-09T08:23:16Z

.github/workflows/sync.yml

+            // Replacing the bot username with exercism is slightly hacky, but necessary
+            // for the comparison below. Unfortunately, GitHub's API response does not seem
+            // to contain any reference to the upstream repo of the fork.
+            const botForks = botRepos.flatMap(({ full_name, fork }) => fork ? [full_name.replace(process.env.BOT_USERNAME, 'exercism')] : [])


The graphql API does allow for referencing the upstream repo, which they refer to as the "parent". E.g., the following graphql will output all the forked repos with their forked name and upstream name:

{ user(login: "ErikSchierboom") { repositories(first: 100, isFork: true) { nodes { nameWithOwner parent { nameWithOwner } } } } }

Which outputs:

{ "data": { "user": { "repositories": { "nodes": [ { "nameWithOwner": "ErikSchierboom/fsharp", "parent": { "nameWithOwner": "exercism/fsharp" } }, { "nameWithOwner": "ErikSchierboom/problem-specifications", "parent": { "nameWithOwner": "exercism/problem-specifications" } }, ... ] } } } }

You can try this yourself at https://docs.github.com/en/graphql/overview/explorer

According to the docs, actions/github-script allows you to run graphql queries: https://github.com/actions/toolkit/tree/master/packages/github

The main disadvantage is that you'd have to do manual pagination, so it might be not that convenient after all.

That would be useful, I'll take a look.

Let's leave this open as a future TODO for now, I'm not sure if the added complexity of manual pagination is worth it, and I don't know graphql at all.

.github/workflows/sync.yml

ErikSchierboom · 2021-03-09T08:24:30Z

.github/workflows/sync.yml

+            // Forking repos happens asynchronously on GitHub, therefore
+            // we need to sleep a bit to ensure it has been created
+            await new Promise(x => setTimeout(x, 60000))


Could we use a different mechanism (e.g. polling) to be certain that the fork was actually created?

I don't think it's worth it. Tracks aren't created that often, this will rarely ever run

.github/workflows/sync.yml

There are three GitHub Actions workflows that are present in many Exercism repositories, but whose contents are identical: 1. Track repos: the workflow to run `configlet lint` 2. Tooling repos: the workflow to deploy the tool as a Docker image 3. All repos: the workflow to sync labels Having duplicated workflows has two main disadvantages: 1. Updating the workflow means submitting a PR to each repository 2. Each repository gets their own dependabot update PRs Both of these disadvantages result in (many) notifications for track maintainers, which can be quite annoying. Solving these problems is done in three steps: 1. We've extracted the shared workflow functionality into reusable workflows that are stored in the exercism/github-actions repository: https://github.com/exercism/github-actions/tree/main/.github/workflows. 2. Modify the workflows currently having duplicated functionality to instead use the reusable workflows. 3. Automatically synchronize the modified workflows from the https://github.com/exercism/org-wide-files repository. Once all these steps have completed, the three above-mentioned cross-repository workflows: - will be identical across the different repositories - will rely on the reusable workflows for their functionality - won't receive dependabot updates as they don't have versioned github action imports (the reusable workflows use `@main`) - can easily be updated in a single location without needing bulk PRs While working on this, we've also made two tiny non-related changes (yeah, this is bad practice, I know :)): 1. We've prepared the `configlet` workflow to allow tracks to opt into also running `configlet fmt` (check the formatting of files in a track's repository). We still need to do a little work in the `org-wide-files` repo to allow track repositories to enable this, but in the end this should be as simple as setting a boolean flag to `true` in a (new) configuration file. 2. The workflows now all have explicit permissions, which improves their security. Note #1: sometimes we merge minor changes like typo fixes in `org-wide-files` without triggering a bulk PR. Therefore, there might be more files that are synced than just the reusable workflows. Note #2: there will be a follow-up PR cleaning up duplicate configlet workflows in case the track's configlet workflow has a different name.

Use reusable workflows for cross-repository workflows There are three GitHub Actions workflows that are present in many Exercism repositories, but whose contents are identical: 1. Track repos: the workflow to run `configlet lint` 2. Tooling repos: the workflow to deploy the tool as a Docker image 3. All repos: the workflow to sync labels Having duplicated workflows has two main disadvantages: 1. Updating the workflow means submitting a PR to each repository 2. Each repository gets their own dependabot update PRs Both of these disadvantages result in (many) notifications for track maintainers, which can be quite annoying. Solving these problems is done in three steps: 1. We've extracted the shared workflow functionality into reusable workflows that are stored in the exercism/github-actions repository: https://github.com/exercism/github-actions/tree/main/.github/workflows. 2. Modify the workflows currently having duplicated functionality to instead use the reusable workflows. 3. Automatically synchronize the modified workflows from the https://github.com/exercism/org-wide-files repository. Once all these steps have completed, the three above-mentioned cross-repository workflows: - will be identical across the different repositories - will rely on the reusable workflows for their functionality - won't receive dependabot updates as they don't have versioned github action imports (the reusable workflows use `@main`) - can easily be updated in a single location without needing bulk PRs While working on this, we've also made two tiny non-related changes (yeah, this is bad practice, I know :)): 1. We've prepared the `configlet` workflow to allow tracks to opt into also running `configlet fmt` (check the formatting of files in a track's repository). We still need to do a little work in the `org-wide-files` repo to allow track repositories to enable this, but in the end this should be as simple as setting a boolean flag to `true` in a (new) configuration file. 2. The workflows now all have explicit permissions, which improves their security. Note #1: sometimes we merge minor changes like typo fixes in `org-wide-files` without triggering a bulk PR. Therefore, there might be more files that are synced than just the reusable workflows. Note #2: there will be a follow-up PR cleaning up duplicate configlet workflows in case the track's configlet workflow has a different name.

SaschaMann requested review from ErikSchierboom and iHiD March 5, 2021 12:45

Add MVP

7f08d95

SaschaMann force-pushed the dev branch from 4e1cdc2 to 7f08d95 Compare March 5, 2021 12:46

SaschaMann commented Mar 5, 2021

View reviewed changes

synced-files/.github/labels.yml Outdated

@@ -0,0 +1,107 @@

- name: "bug :bug:"

Copy link

Contributor Author

SaschaMann Mar 5, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a file I used for testing.

SaschaMann commented Mar 5, 2021

View reviewed changes

Add very basic docs

087dfa2

SaschaMann marked this pull request as ready for review March 5, 2021 12:51

iHiD reviewed Mar 5, 2021

View reviewed changes

SaschaMann added 3 commits March 6, 2021 10:43

Use LICENSE as MVP example

660d14b

Format YAML

0b38c7e

Remove research1 handling

6956ca3

ErikSchierboom approved these changes Mar 9, 2021

View reviewed changes

Apply suggestsions from code review

234cc23

SaschaMann closed this May 28, 2021

SaschaMann deleted the dev branch May 29, 2021 11:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enable everything #1

Enable everything #1

SaschaMann commented Mar 5, 2021

SaschaMann Mar 5, 2021

SaschaMann Mar 5, 2021

SaschaMann commented Mar 5, 2021

iHiD Mar 5, 2021

ErikSchierboom Mar 5, 2021

ErikSchierboom left a comment

ErikSchierboom Mar 9, 2021

ErikSchierboom Mar 9, 2021

SaschaMann Mar 9, 2021

SaschaMann Mar 9, 2021 •

edited

Loading

ErikSchierboom Mar 9, 2021

ErikSchierboom Mar 9, 2021

SaschaMann Mar 9, 2021

Enable everything #1

Enable everything #1

Conversation

SaschaMann commented Mar 5, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SaschaMann commented Mar 5, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ErikSchierboom left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SaschaMann Mar 9, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SaschaMann Mar 9, 2021 •

edited

Loading