Speed up migrations linting in CI

[0x962]

We're running django-migrations-linter as part of our CI, it's a great tool, thank you!

Do you have any recommendations on speeding up this step when using Github actions?

I did notice the --git-commit-id parameter, but was unable to make it work using the github.event.pull_request.sha variable. (Raises a diff error)

[David-Wobrock]

Hi @0x962

Thanks for submitting an issue.

A good way to improve speed is indeed to only lint newly added migrations. This can be done in multiple ways:

• --git-commit-id, if it doesn't work, maybe you can run the lintmigrations command with some verbosity options to debug a bit and submit another issue to see if the issue comes from the linter itself
• --include-migrations-from FILE_PATH you can provider a file, that contains one migration per line. And the linter will only consider those migrations.

The linter also writes a little cache, which is mostly leveraged on your local environment. I don't think GitHub Actions has some persistence to keep the cache from one run to the next 🤔

[blueyed]

@David-Wobrock

The linter also writes a little cache, which is mostly leveraged on your local environment. I don't think GitHub Actions has some persistence to keep the cache from one run to the next thinking

GitHub actions supports caching in general.

@0x962
The cache should be at ~/.cache/django-migration-linter.
But then it is also unclear how good cache invalidation works / if it might cause any problems, of course.

[David-Wobrock]

Hi @blueyed

Good to know for GH Actions, thanks!

The linter cache uses a hash of the file's content as key (see https://github.com/3YOURMIND/django-migration-linter/blob/main/django_migration_linter/migration_linter.py#L213) and stores the linting result as value for each file. Then, the cache is expectedly depend of the selected Django project, database and the linter's version.
The goal is to avoid generating the SQL statements and running the linting rules.

[0x962]

Thank you for your comments, @blueyed and @David-Wobrock.

For other people who stumble upon this thread and use this amazing tool, this is my Github actions implementation. ( The relevant steps )

      - name: Caches django migrations linter
        uses: actions/cache@v2
        with:
          path: |
            $HOME/.cache/django-migration-linter
          key: ${{ runner.os }}-migrations-linter-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-migrations-linter-${{ github.sha }}
            ${{ runner.os }}-migrations-linter-
            ${{ runner.os }}-

      - name: Run Migrations Linter
        run: MIGRATE=True python manage.py lintmigrations

[David-Wobrock]

[...] and use this amazing tool [...]

🤗