Introduce Job approach and JobDependency #183

fepitre · 2025-02-18T14:39:25Z

No description provided.

codecov · 2025-02-19T09:03:17Z

Codecov Report

Attention: Patch coverage is 91.92399% with 34 lines in your changes missing coverage. Please review.

Project coverage is 78.74%. Comparing base (7437576) to head (7c12b4d).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
qubesbuilder/plugins/__init__.py	86.50%	17 Missing ⚠️
qubesbuilder/plugins/installer/__init__.py	68.75%	10 Missing ⚠️
qubesbuilder/plugins/template/__init__.py	89.65%	3 Missing ⚠️
qubesbuilder/plugins/fetch/__init__.py	81.81%	2 Missing ⚠️
qubesbuilder/cli/cli_repository.py	92.30%	1 Missing ⚠️
qubesbuilder/config.py	97.67%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #183      +/-   ##
==========================================
+ Coverage   78.21%   78.74%   +0.52%     
==========================================
  Files          47       47              
  Lines        5413     5388      -25     
==========================================
+ Hits         4234     4243       +9     
+ Misses       1179     1145      -34

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Also apply few reformat

Inside the CI we apply filter for pytest and some were ignored.

marmarek · 2025-02-25T03:30:35Z

qubesbuilder/config.py

+        for distribution in distributions:
+            for component in components:
+                for plugin in plugins:
+                    if "DistributionComponentPlugin" in [


Using isinstance() here would lead to cyclic imports, right?

I think that was the issue, I can double check.

I just remind that the problem is because plugins inherit from multiple classes and it looks like isinstance does not find parent class type.

marmarek · 2025-02-25T03:32:44Z

qubesbuilder/pluginmanager.py

@@ -75,7 +75,7 @@ def _get_plugin_entities(self):

        return entities

-    def _get_instances_with_attr(self, module_attr, **kwargs):
+    def _get_instances_with_attr(self, module_attr):


This no longer return instances, but plugin classes.

marmarek · 2025-02-25T03:39:40Z

qubesbuilder/plugins/__init__.py

-            if dependency.builder_object == "component":
+            if dependency.builder_object == "plugin":
+                if (
+                    dependency.builder_object == "plugin"


yes copy paste..

marmarek · 2025-02-25T03:41:46Z

qubesbuilder/plugins/__init__.py

                )
                if not component:
                    raise PluginError(
                        f"Cannot find component '{dependency}' in configuration file."
                    )
-                if not self.config.sources_dir / dependency.name:
+                if not self.config.sources_dir / dependency.reference:


missing .exists()?

marmarek · 2025-02-25T03:56:00Z

qubesbuilder/plugins/fetch/__init__.py

        """
        Run plugin for given stage.
        """

-        if stage != "fetch":
+        if self.stage != "fetch":


Is this still necessary with stages defined at the class level?

marmarek · 2025-02-25T04:04:31Z

qubesbuilder/plugins/sign_rpm/__init__.py


-        if stage != "sign" or not self.has_component_packages("sign"):
+        if self.stage != "sign" or not self.has_component_packages("sign"):


marmarek · 2025-02-25T04:04:59Z

qubesbuilder/plugins/source_archlinux/__init__.py


-        if stage != "prep" or not self.has_component_packages("prep"):
+        if self.stage != "prep" or not self.has_component_packages("prep"):


marmarek · 2025-02-25T04:05:09Z

qubesbuilder/plugins/source_deb/__init__.py


-        if stage != "prep" or not self.has_component_packages("prep"):
+        if self.stage != "prep" or not self.has_component_packages("prep"):


marmarek · 2025-02-25T04:05:20Z

qubesbuilder/plugins/source_rpm/__init__.py


-        if stage != "prep" or not self.has_component_packages("prep"):
+        if self.stage != "prep" or not self.has_component_packages("prep"):


marmarek · 2025-02-25T04:11:45Z

qubesbuilder/plugins/upload/__init__.py

-    def run(self, stage: str, repository_publish: str = None):
-        if stage != "upload":
+    def run(self, repository_publish: Optional[str] = None):
+        if self.stage != "upload":


marmarek · 2025-02-25T13:36:24Z

Looks good now I think. Can you schedule a pipeline with QubesOS/qubes-builderv2-github#23 on this PR?

fepitre force-pushed the devel20250213 branch 4 times, most recently from 0f08475 to def4697 Compare February 18, 2025 15:05

template: don't fail on publish only notify user

110ef68

Closes QubesOS/qubes-issues#8198

fepitre force-pushed the devel20250213 branch 4 times, most recently from 9a876e0 to eb4032a Compare February 19, 2025 08:25

Refactor code to introduce Job approach and JobDependency

f8656e5

fepitre force-pushed the devel20250213 branch from eb4032a to f8656e5 Compare February 19, 2025 10:58

fepitre mentioned this pull request Feb 19, 2025

Adapt code with JobDependency refactor QubesOS/qubes-builderv2-github#23

Open

fepitre marked this pull request as ready for review February 20, 2025 16:48

fepitre added 3 commits February 21, 2025 14:49

Add test for validating JobDependency

f63bd41

Also apply few reformat

test_cli: rename and move tests to the top

9971af3

Inside the CI we apply filter for pytest and some were ignored.

tests: don't use session scope artifacts dir for JobDependency

67b1c22

marmarek requested changes Feb 25, 2025

View reviewed changes

Remove redundant single stage checks and fix dependency path check

7c12b4d

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce Job approach and JobDependency #183

Introduce Job approach and JobDependency #183

fepitre commented Feb 18, 2025

codecov bot commented Feb 19, 2025 •

edited

Loading

marmarek Feb 25, 2025

fepitre Feb 25, 2025

fepitre Feb 25, 2025

marmarek Feb 25, 2025

marmarek Feb 25, 2025

fepitre Feb 25, 2025

marmarek Feb 25, 2025

marmarek Feb 25, 2025

marmarek Feb 25, 2025

marmarek Feb 25, 2025

marmarek Feb 25, 2025

marmarek Feb 25, 2025

marmarek Feb 25, 2025

marmarek commented Feb 25, 2025


		if stage != "sign" or not self.has_component_packages("sign"):
		if self.stage != "sign" or not self.has_component_packages("sign"):


		if stage != "prep" or not self.has_component_packages("prep"):
		if self.stage != "prep" or not self.has_component_packages("prep"):

Introduce Job approach and JobDependency #183

Are you sure you want to change the base?

Introduce Job approach and JobDependency #183

Conversation

fepitre commented Feb 18, 2025

codecov bot commented Feb 19, 2025 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marmarek commented Feb 25, 2025

codecov bot commented Feb 19, 2025 •

edited

Loading