Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kde-neon extension fixes #5261

Merged
merged 7 commits into from
Feb 20, 2025

Conversation

ScarlettGatelyMoore
Copy link
Contributor

Don't add X11 to layouts twice. Add mesa libs to core24 ld library path.

  • [x ] Have you followed the guidelines for contributing?
  • [x ] Have you signed the CLA?
  • [x ] Have you successfully run make lint?
  • [x ] Have you successfully run make test?

Copy link
Collaborator

@mr-cal mr-cal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! You will need to do a make format and make lint to fix the CI failure. (and make setup too, we had some major changes to the dev environment for Snapcraft)

A few other spread tests are broken this week. We're planning to fix them soon, so you can ignore them.

@mr-cal mr-cal requested a review from lengau February 14, 2025 20:28
Copy link
Contributor

@lengau lengau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@ScarlettGatelyMoore
Copy link
Contributor Author

Thanks! You will need to do a make format and make lint to fix the CI failure. (and make setup too, we had some major changes to the dev environment for Snapcraft)

A few other spread tests are broken this week. We're planning to fix them soon, so you can ignore them.

Ok, I am on plucky and make setup didn't like libgit2-1.8 so I built 1.7 from source and was able to move forward. make format was a success and I committed. make lint failed with

Pyproject file parse attempt 1 error: {"name":"TomlError","fromTOML":true,"wrapped":null,"line":102,"col":31,"pos":2614}
Pyproject file parse attempt 2 error: {"name":"TomlError","fromTOML":true,"wrapped":null,"line":102,"col":31,"pos":2614}
Pyproject file parse attempt 3 error: {"name":"TomlError","fromTOML":true,"wrapped":null,"line":102,"col":31,"pos":2614}
Pyproject file parse attempt 4 error: {"name":"TomlError","fromTOML":true,"wrapped":null,"line":102,"col":31,"pos":2614}
Pyproject file parse attempt 5 error: {"name":"TomlError","fromTOML":true,"wrapped":null,"line":102,"col":31,"pos":2614}
Pyproject file parse attempt 6 error: {"name":"TomlError","fromTOML":true,"wrapped":null,"line":102,"col":31,"pos":2614}
Config file "/home/scarlett/ubuntu-core/snapcraft/pyproject.toml" could not be parsed. Verify that format is correct.
0 errors, 0 warnings, 0 informations
make: *** [common.mk:133: lint-pyright] Error 3

@bepri
Copy link
Contributor

bepri commented Feb 18, 2025

The line number that error is originating from makes me think there might be local modifications to your pyproject.toml for the dependency of pygit2, and that introduced a syntax error. If running uv build fails then that's probably the case. If not, could you show this diff? git diff origin/main -- pyproject.toml

Also, are you able to share the error message you got from installing libgit2? I think the "cleanest" fix here would be this change to the makefile:

ifeq ($(wildcard /usr/share/doc/libgit2-dev/copyright),)
APT_PACKAGES += libgit2-dev
endif

becomes

ifeq ($(wildcard /usr/share/doc/libgit2-1.7/copyright),)
APT_PACKAGES += libgit2-1.7
endif

But I don't have a plucky machine to test on.

@ScarlettGatelyMoore
Copy link
Contributor Author

ScarlettGatelyMoore commented Feb 18, 2025

`> git diff origin/main -- pyproject.toml

scarlett@scarlett-work2:~/ubuntu-core/snapcraft$ git diff origin/main -- pyproject.toml
diff --git a/pyproject.toml b/pyproject.toml
index 62b20ec2..f26ab211 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,14 +1,197 @@
-[tool.black]
-extend-exclude = '''
-/(

  • | snapcraft_legacy
  • | tools
  • | docs/sphinx-resources
    -)/
    -'''
    -# Targeting future versions as well so we don't have black reformatting code
    -# en masse later.
    -target-version = ["py312"]
    +[project]
    +name = "snapcraft"
    +description = "Create snaps"
    +readme = "README.md"
    +dynamic = ["version"]
    +dependencies = [
  • "attrs",
  • "catkin-pkg==1.0.0 ; sys_platform == 'linux'",
  • "click==8.1.7",
  • "craft-application[remote]~=4.9",
  • "craft-archives~=2.0",
  • "craft-cli~=2.15.0",
  • "craft-grammar>=2.0.1,<3.0.0",
  • "craft-parts==2.4.1",
  • "craft-platforms~=0.6",
  • "craft-providers~=2.1",
  • "craft-store>=3.0.2,<4.0.0",
  • "cryptography==43.0.3", # Pinned due to Snapcraft conflates standard PyPI packages with local plugins #5217
  • "gnupg",
  • "jsonschema==2.5.1",
  • "launchpadlib",
  • "lazr.restfulclient",
  • "lxml",
  • "macaroonbakery",
  • "mypy-extensions",
  • "overrides",
  • "packaging",
  • "progressbar",
  • "pydantic~=2.8",
  • "pyelftools",
  • Pygit2 and libgit2 need to match versions.

  • Further info: https://www.pygit2.org/install.html#version-numbers

  • "pygit2~=1.13.0",
  • "pylxd ; sys_platform == 'linux'",
  • "pymacaroons",
  • "python-debian ; sys_platform == 'linux'",
  • "pyxdg",
  • "pyyaml",
  • "raven",
  • "requests-toolbelt",
  • "requests-unixsocket2",
  • "requests",
  • "simplejson",
  • "snap-helpers",
  • "tabulate",
  • "toml",
  • "tinydb",
  • "typing-extensions",
  • "validators>=0.28.3",
    +]
    +classifiers = [
  • "Development Status :: 5 - Production/Stable",
  • "Environment :: Console",
  • "Operating System :: POSIX :: Linux",
  • "Intended Audience :: Developers",
  • "Intended Audience :: System Administrators",
  • "Natural Language :: English",
  • "Topic :: Software Development :: Build Tools",
  • "Topic :: System :: Software Distribution",
  • "License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
  • "Programming Language :: Python :: 3",
  • "Programming Language :: Python :: 3.12",
    +]
    +requires-python = ">=3.12"

+[dependency-groups]
+dev = [

  • "coverage[toml]",
  • "pyflakes",
  • "fixtures",
  • For Tiobe TiCS

  • "flake8",
  • "mccabe",
  • "testscenarios",
  • "pexpect",
  • "pip",
  • "pycodestyle",
  • "pyftpdlib",
  • "pyinstaller ; sys_platform == 'win32'",
  • For Tiobe TiCS

  • "pylint",
  • "pyramid",
  • "pytest",
  • "pytest-cov",
  • "pytest-check",
  • "pytest-mock",
  • "pytest-subprocess",
    +]
    +docs = [
  • "canonical-sphinx[full]>=0.2.0",
  • "docutils<0.20", # Frozen until we can update sphinx dependencies.
  • "sphinx-autobuild",
  • "sphinx-autodoc-typehints",
  • "sphinxcontrib-details-directive",
  • "sphinx-lint",
  • "sphinx-toolbox",
  • "sphinxext-rediraffe==0.2.7",
  • "pyspelling",
    +]
    +lint = [
  • "yamllint~=1.34",
  • "codespell[toml]",
  • { include-group = "types" },
    +]
    +types = [
  • "mypy",
  • "types-PyYAML",
  • Pinned because any higher requires urllib3>=2

  • "types-requests==2.31.0.6",
  • "types-setuptools",
  • "types-simplejson",
  • "types-tabulate",
  • "types-toml",
    +]
    +dev-jammy = [
  • Stay on 2.4 for Jammy

  • "python-apt~=2.4.0;sys_platform=='linux'",
    +]
    +dev-noble = [
  • 2.7 for Noble

  • "python-apt~=2.7.0;sys_platform=='linux'",
    +]
    +dev-oracular = [
  • 2.9 for Oracular+

  • "python-apt>=2.9.0;sys_platform=='linux'",
    +]
    +dev-plucky = [
  • 2.9 for Oracular+

  • "python-apt>=2.9.0;sys_platform=='linux'",
    +]

+[tool.uv]
+conflicts = [

  • [
  •    { group = "dev-jammy" },
    
  •    { group = "dev-noble" },
    
  •    { group = "dev-oracular" },
    
  •    { group = "dev-plucky" },
    
  • ]
    +]

+[[tool.uv.index]]
+name = "python-apt-wheels"
+url = "https://people.canonical.com/~lengau/python-apt-ubuntu-wheels/" # workaround to get python-apt to install across multiple platforms
+
+[project.scripts]
+snapcraft = "snapcraft.application:main"
+snapcraft_legacy = "snapcraft_legacy.cli.main:run"
+
+[build-system]
+requires = [

  • "setuptools>=69.0",
  • "setuptools_scm[toml]>=7.1",
    +]
    +build-backend = "setuptools.build_meta"

+[tool.setuptools]
+script-files = ["bin/snapcraftctl", "bin/snapcraftctl-compat"]
+
+[tool.setuptools_scm]
+write_to = "snapcraft/_version.py"
+# the version comes from the latest annotated git tag formatted as 'X.Y.Z'
+# version scheme:
+# - X.Y.Z.post+g.d<%Y%m%d>
+# parts of scheme:
+# - X.Y.Z - most recent git tag
+# - post+g - present when current commit is not tagged
+# - +dirty - present when working dir is dirty
+# version scheme when no tags exist:
+# - 0.0.post+g
+version_scheme = "post-release"
+# deviations from the default 'git describe' command:
+# - only match annotated tags
+# - only match tags formatted as 'X.Y.Z'
+git_describe_command = [

  • "git",
  • "describe",
  • "--dirty",
  • "--long",
  • "--match",
  • "[0-9].[0-9].[0-9]*",
  • "--exclude",
  • "[^0-9.]",
    +]
    +# Appends "+dirty" to the version when working dir is dirty
    +# Necessary to avoid version numbers being too long
    +local_scheme = "dirty-tag"

+[tool.setuptools.packages.find]
+include = ["snapcraft", "extensions", "keyrings", "schema", "snapcraft_legacy"]
+namespaces = true
+
+[tool.setuptools.package-data]
+snapcraft = ["templates/*"]

[tool.mypy]
python_version = "3.12"
@@ -31,11 +214,6 @@ include = ["snapcraft", "tests"]
exclude = ["build", "tests/legacy", "tests/spread"]
pythonVersion = "3.12"

-[tool.pytest.ini_options]
-minversion = "7.0"
-required_plugins = ["pytest-cov>=4.0", "pytest-mock>=3.12", "pytest-subprocess>=1.4"]
-addopts = ["--cov=snapcraft"]

Most of this ruff configuration comes from craft-parts

[tool.ruff]
target-version = "py312"
@@ -45,6 +223,7 @@ extend-exclude = [
"pycache",
"legacy",
"tests/legacy",

  • "snapcraft/_version.py", # setuptools_scm generates old-style type annotations and single quotes
    ]
    lint.select = [
    "E", "F", # The rules built into Flake8
    @@ -129,5 +308,47 @@ max-branches = 16
    ]
    "init.py" = ["I001"] # Imports in init filesare allowed to be out of order

-[tool.ruff.lint.flake8-annotations]
-suppress-none-returning = true # We don't need to explicitly point out that a function doesn't return anything
+[tool.ruff.lint.pydocstyle]
+ignore-decorators = [ # Functions with these decorators don't have to have docstrings.

  • "typing.overload", # Default configuration
  • The next four are all variations on override, so child classes don't have to repeat parent classes' docstrings.

  • "overrides.override",
  • "overrides.overrides",
  • "typing.override",
  • "typing_extensions.override",
    +]

+[tool.ruff.lint.pep8-naming]
+# Allow Pydantic's @validator decorator to trigger class method treatment.
+classmethod-decorators = ["pydantic.validator", "pydantic.root_validator"]
+
+[tool.pytest.ini_options]
+minversion = "7.0"
+testpaths = "tests"
+xfail_strict = true
+markers = ["slow: slow tests"]
+
+[tool.coverage.run]
+branch = true
+omit = ["test/**"]
+
+[tool.coverage.report]
+skip_empty = true
+exclude_also = [

  • "if (typing\.)?TYPE_CHECKING:",
    +]

+[tool.codespell]
+ignore-words-list = [

  • "buildd",
  • "crate",
  • "assertIn",
  • "socio-economic",
    +]
    +skip = [
  • Keyrings aren't code

  • "*.asc",
  • These tests include some non-English text

  • "*/test_appstream.py",
  • "*/test_update_metadata.py",
    +]
    (END)

uv build succeeds`

@bepri
Copy link
Contributor

bepri commented Feb 18, 2025

Sorry, I should've thought of this - you'll want to update that main branch first. I'm basically just wanting to know any differences between your local pyproject.toml and the one here, if any.

While we're at it, could you also show the output of pyright --version and uv version?

Ultimately, this is all passing in CI so it should be okay, but it would be awesome to know why make setup && make lint isn't working for you so we can fix it for others.

@ScarlettGatelyMoore
Copy link
Contributor Author

pyright --version

There is no longer a diff

pyright 1.1.393
uv 0.6.1 (c91ee82a8 2025-02-17)

make setup works but now make lint fails with

Makefile:26: warning: overriding recipe for target 'lint-docs'
common.mk:166: warning: ignoring old recipe for target 'lint-docs'
ruff check snapcraft tests
ruff format --diff snapcraft tests
ruff failed
Cause: TOML parse error at line 228, column 1
|
228 | lint.select = [
| ^^^^
unknown field lint, expected one of allowed-confusables, builtins, cache-dir, dummy-variable-rgx, exclude, extend, extend-exclude, extend-include, extend-ignore, extend-select, extend-fixable, extend-unfixable, external, fix, fix-only, fixable, output-format, force-exclude, ignore, ignore-init-module-imports, include, line-length, tab-size, logger-objects, required-version, respect-gitignore, select, show-source, show-fixes, src, namespace-packages, target-version, preview, task-tags, typing-modules, unfixable, flake8-annotations, flake8-bandit, flake8-bugbear, flake8-builtins, flake8-comprehensions, flake8-copyright, flake8-errmsg, flake8-quotes, flake8-self, flake8-tidy-imports, flake8-type-checking, flake8-gettext, flake8-implicit-str-concat, flake8-import-conventions, flake8-pytest-style, flake8-unused-arguments, isort, mccabe, pep8-naming, pycodestyle, pydocstyle, pyflakes, pylint, pyupgrade, format, per-file-ignores, extend-per-file-ignores

make: *** [common.mk:101: lint-ruff] Error 2

@ScarlettGatelyMoore
Copy link
Contributor Author

pyright --version

There is no longer a diff

pyright 1.1.393 uv 0.6.1 (c91ee82a8 2025-02-17)

make setup works but now make lint fails with

Makefile:26: warning: overriding recipe for target 'lint-docs' common.mk:166: warning: ignoring old recipe for target 'lint-docs' ruff check snapcraft tests ruff format --diff snapcraft tests ruff failed Cause: TOML parse error at line 228, column 1 | 228 | lint.select = [ | ^^^^ unknown field lint, expected one of allowed-confusables, builtins, cache-dir, dummy-variable-rgx, exclude, extend, extend-exclude, extend-include, extend-ignore, extend-select, extend-fixable, extend-unfixable, external, fix, fix-only, fixable, output-format, force-exclude, ignore, ignore-init-module-imports, include, line-length, tab-size, logger-objects, required-version, respect-gitignore, select, show-source, show-fixes, src, namespace-packages, target-version, preview, task-tags, typing-modules, unfixable, flake8-annotations, flake8-bandit, flake8-bugbear, flake8-builtins, flake8-comprehensions, flake8-copyright, flake8-errmsg, flake8-quotes, flake8-self, flake8-tidy-imports, flake8-type-checking, flake8-gettext, flake8-implicit-str-concat, flake8-import-conventions, flake8-pytest-style, flake8-unused-arguments, isort, mccabe, pep8-naming, pycodestyle, pydocstyle, pyflakes, pylint, pyupgrade, format, per-file-ignores, extend-per-file-ignores

make: *** [common.mk:101: lint-ruff] Error 2

Nevermind, I am a dummy. I was not in venv. Still the same original error

@mr-cal
Copy link
Collaborator

mr-cal commented Feb 20, 2025

FYI @medubelko, with this PR, we will need to split https://canonical-snapcraft.readthedocs-hosted.com/en/latest/reference/extensions/kde-neon-extensions.html#included-build-environment-variables into a core22 and core24 sections.

I can make an issue in Github, we'll have similar work for #5271

@mr-cal mr-cal merged commit 346aa2e into canonical:main Feb 20, 2025
14 of 17 checks passed
@medubelko
Copy link
Contributor

I can make an issue in Github

Yes, please. It's something we can also add to CODA.

@mr-cal
Copy link
Collaborator

mr-cal commented Feb 20, 2025

I can make an issue in Github

Yes, please. It's something we can also add to CODA.

#5275

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants