Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Add check to ensure locked dependencies have source distributions available. #14742

Merged
merged 10 commits into from
Feb 13, 2023
27 changes: 27 additions & 0 deletions .github/workflows/poetry_lockfile.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
on:
push:
branches: ["develop", "release-*"]
paths:
- poetry.lock
pull_request:
paths:
- poetry.lock
Comment on lines +1 to +8
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice that we only trigger this if needed.

The downside is that it can't be fed into the tests-done job to tell github it's safe to merge this PR. But I think we can live with that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's frustrating how the default behaviour isn't just to make sure all the tests pass, but ah well


concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

permissions:
contents: write
reivilibre marked this conversation as resolved.
Show resolved Hide resolved

jobs:
check-sdists:
name: "Check locked dependencies have sdists"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: '3.x'
- run: pip install tomli
- run: ./scripts-dev/check_locked_deps_have_sdists.py
1 change: 1 addition & 0 deletions changelog.d/14742.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add check to ensure locked dependencies have source distributions available.
30 changes: 27 additions & 3 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,9 @@ twine = "*"
# Towncrier min version comes from #3425. Rationale unclear.
towncrier = ">=18.6.0rc1"

# Used for checking the Poetry lockfile
tomli = ">=1.2.3"

reivilibre marked this conversation as resolved.
Show resolved Hide resolved
[build-system]
# The upper bounds here are defensive, intended to prevent situations like
# #13849 and #14079 where we see buildtime or runtime errors caused by build
Expand Down
63 changes: 63 additions & 0 deletions scripts-dev/check_locked_deps_have_sdists.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
#! /usr/bin/env python
# Copyright 2022 The Matrix.org Foundation C.I.C.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import sys
from pathlib import Path
from typing import Dict, List

import tomli


def main() -> None:
lockfile_path = Path(__file__).parent.parent.joinpath("poetry.lock")
with open(lockfile_path, "rb") as lockfile:
lockfile_content = tomli.load(lockfile)

packages_to_assets: Dict[str, List[Dict[str, str]]]
# There seem to be two different formats for storing the list of files per package.
if lockfile_content.get("metadata", {}).get("files") is not None:
# either [metadata.files]
packages_to_assets = lockfile_content["metadata"]["files"]
else:
# or a `files` inline table in each [[package]]
packages_to_assets = {
package["name"]: package["files"] for package in lockfile_content["package"]
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to be part of the poetry 1.3 lockfile format, see python-poetry/poetry#6393

Copy link
Contributor

@DMRobertson DMRobertson Jan 17, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that I'm about to require the new lockfile format in #14860


success = True

for package_name, assets in packages_to_assets.items():
has_sdist = any(asset["file"].endswith(".tar.gz") for asset in assets)
if not has_sdist:
success = False
print(
f"Locked package {package_name!r} does not have a source distribution!",
file=sys.stderr,
)

if not success:
print(
"\nThere were some problems with the Poetry lockfile (poetry.lock).",
file=sys.stderr,
)
sys.exit(1)

print(
f"Poetry lockfile OK. {len(packages_to_assets)} locked packages checked.",
file=sys.stderr,
)


if __name__ == "__main__":
main()