-
Notifications
You must be signed in to change notification settings - Fork 17
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
chore(ci): prevent commit including generated files (#379)
* chore(ci): add pre-commit hook to unstage generated codes * chore: test commit * chore: implement pre-commit script * chore: fix lint error * chore: fix lint error * chore: call js file directly * chore: update .cache_version * Update scripts/ci/husky/pre-commit.js Co-authored-by: Pierre Millot <[email protected]> * Update scripts/ci/husky/pre-commit.js Co-authored-by: Pierre Millot <[email protected]> * chore: use postinstall instead of prepare * chore: better error message * chore: fix patterns * chore: unstage files automatically and exit with 0 * chore: change loop * chore: skip deleted files * chore: split config to a separate file * chore: update docs * chore: update cache version Co-authored-by: Pierre Millot <[email protected]>
- Loading branch information
1 parent
04c1122
commit 9fbba0e
Showing
12 changed files
with
204 additions
and
21 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1 @@ | ||
| 8.0.5.1 | ||
| 8.0.5.2 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| #!/bin/sh | ||
| . "$(dirname "$0")/_/husky.sh" | ||
|
|
||
| ./scripts/ci/husky/pre-commit.js |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| // eslint-disable-next-line import/no-commonjs | ||
| module.exports = { | ||
| patterns: [ | ||
| // Ignore the roots and go down the tree by negating hand written files | ||
| 'clients/**', | ||
| '!clients/README.md', | ||
| '!clients/**/.openapi-generator-ignore', | ||
|
|
||
| // Java | ||
| '!clients/algoliasearch-client-java-2/algoliasearch-core/src/com/algolia/exceptions/*', | ||
| '!clients/algoliasearch-client-java-2/algoliasearch-core/src/com/algolia/utils/*', | ||
| 'clients/algoliasearch-client-java-2/algoliasearch-core/com/algolia/utils/echo/EchoResponse*.java', | ||
| '!clients/algoliasearch-client-java-2/algoliasearch-core/com/algolia/utils/echo/EchoResponseInterface.java', | ||
|
|
||
| // JavaScript | ||
| '!clients/algoliasearch-client-javascript/*', | ||
| '!clients/algoliasearch-client-javascript/.github/**', | ||
| '!clients/algoliasearch-client-javascript/.yarn/**', | ||
| '!clients/algoliasearch-client-javascript/scripts/**', | ||
| '!clients/algoliasearch-client-javascript/packages/algoliasearch/**', | ||
| '!clients/algoliasearch-client-javascript/packages/requester-*/**', | ||
| '!clients/algoliasearch-client-javascript/packages/client-common/**', | ||
|
|
||
| // PHP | ||
| '!clients/algoliasearch-client-php/lib/Configuration/*', | ||
| 'clients/algoliasearch-client-php/lib/*.php', | ||
| 'clients/algoliasearch-client-php/lib/Api/*', | ||
| 'clients/algoliasearch-client-php/lib/Configuration/Configuration.php', | ||
| ], | ||
| }; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| /* eslint-disable import/no-commonjs */ | ||
| // eslint-disable-next-line @typescript-eslint/no-var-requires | ||
| const { createMemoizedMicromatchMatcher } = require('../pre-commit'); | ||
|
|
||
| describe('createMemoizedMicromatchMatcher', () => { | ||
| it('matches correctly', () => { | ||
| const matcher = createMemoizedMicromatchMatcher([ | ||
| 'clients/**', | ||
| '!clients/README.md', | ||
| ]); | ||
|
|
||
| expect(matcher('clients/README.md')).toEqual(false); | ||
| expect(matcher('clients/CONTRIBUTING.md')).toEqual(true); | ||
| }); | ||
|
|
||
| it('prioritizes the exact match when two patterns conflict', () => { | ||
| const matcher = createMemoizedMicromatchMatcher([ | ||
| '!lib/Configuration/*', | ||
| 'lib/Configuration/Configuration.php', | ||
| ]); | ||
|
|
||
| expect(matcher('lib/Configuration/Configuration.php')).toEqual(true); | ||
| }); | ||
| }); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| #!/usr/bin/env node | ||
| /* eslint-disable no-console */ | ||
| /* eslint-disable import/no-commonjs */ | ||
| /* eslint-disable @typescript-eslint/no-var-requires */ | ||
| const chalk = require('chalk'); | ||
| const execa = require('execa'); | ||
| const micromatch = require('micromatch'); | ||
|
|
||
| const GENERATED_FILE_PATTERNS = | ||
| require('../../../config/generation.config').patterns; | ||
|
|
||
| const run = async (command, { cwd } = {}) => { | ||
| return ( | ||
| (await execa.command(command, { shell: 'bash', all: true, cwd })).all ?? '' | ||
| ); | ||
| }; | ||
|
|
||
| function createMemoizedMicromatchMatcher(patterns = []) { | ||
| const exactMatchers = []; | ||
| const positiveMatchers = []; | ||
| const negativeMatchers = []; | ||
|
|
||
| patterns.forEach((pattern) => { | ||
| if (pattern.startsWith('!')) { | ||
| // Patterns starting with `!` are negated | ||
| negativeMatchers.push(micromatch.matcher(pattern.slice(1))); | ||
| } else if (!pattern.includes('*')) { | ||
| exactMatchers.push(micromatch.matcher(pattern)); | ||
| } else { | ||
| positiveMatchers.push(micromatch.matcher(pattern)); | ||
| } | ||
| }); | ||
|
|
||
| return function matcher(str) { | ||
| if (exactMatchers.some((match) => match(str))) { | ||
| return true; | ||
| } | ||
|
|
||
| // As `some` returns false on empty array, test will always fail if we only | ||
| // provide `negativeMatchers`. We fallback to `true` is it's the case. | ||
| const hasPositiveMatchers = | ||
| Boolean(positiveMatchers.length === 0 && negativeMatchers.length) || | ||
| positiveMatchers.some((match) => match(str)); | ||
|
|
||
| return hasPositiveMatchers && !negativeMatchers.some((match) => match(str)); | ||
| }; | ||
| } | ||
|
|
||
| async function preCommit() { | ||
| const stagedFiles = (await run(`git diff --name-only --cached`)).split('\n'); | ||
| const deletedFiles = new Set( | ||
| (await run(`git ls-files --deleted`)).split('\n') | ||
| ); | ||
| const matcher = createMemoizedMicromatchMatcher(GENERATED_FILE_PATTERNS); | ||
|
|
||
| for (const stagedFile of stagedFiles) { | ||
| // keep the deleted files staged even if they were generated before. | ||
| if (deletedFiles.has(stagedFile)) { | ||
| continue; | ||
| } | ||
|
|
||
| if (!matcher(stagedFile)) { | ||
| continue; | ||
| } | ||
|
|
||
| console.log( | ||
| chalk.bgYellow('[INFO]'), | ||
| `Generated file found, unstaging: ${stagedFile}` | ||
| ); | ||
|
|
||
| await run(`git restore --staged ${stagedFile}`); | ||
| } | ||
| } | ||
|
|
||
| if (require.main === module && !process.env.CI) { | ||
| preCommit(); | ||
| } | ||
|
|
||
| module.exports = { | ||
| createMemoizedMicromatchMatcher, | ||
| }; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,17 @@ | ||
| --- | ||
| title: Commit and Pull-request | ||
| --- | ||
|
|
||
| # Commit and Pull-request | ||
|
|
||
| ## Commit | ||
|
|
||
| If you accidentally include generated files in your commit, the `pre-commit` hook will automatically unstage them. | ||
|
|
||
| We create commits on the CI as well, and in that case, we skip this unstaging behavior with the environment variable `CI=true` given. | ||
|
|
||
| If you want to change the patterns of generated file paths, see [config/generation.config.js](https://github.com/algolia/api-clients-automation/blob/main/config/generation.config.js). | ||
|
|
||
| ## Pull-request | ||
|
|
||
| Semantic title is required. It's validated by [GitHub Action](https://github.com/deepakputhraya/action-pr-title). See [pr-title.yml](https://github.com/algolia/api-clients-automation/blob/main/.github/workflows/pr-title.yml) for the complete regular expressions. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters