[WIT-355] Create the CI inside the dbt workload template repository

# New features and improvements

- Add CI to create .whl artifact.
- Push the artifact to the Gitlab Package Registry.
- Add a new section `Artifacts` similar to the one present inside README of specific provisioners to explain about recent changes.

# Related issue

Closes WIT-355

# Definition of Done

## All developments
- [x] Feature was implemented as per the requirements
- [x] If some code parts are complex they must be commented with code documentation
- [x] If feature affects the corresponding Governance Policy, then the policy has been updated and a new version of the policy set up (before the template is QA tested)
- [x] QA tests are successful (please comment with a link to the created component & tested DP)
    - Template must be tested by:
        - Creating a component with it in QA env
        - Committing the DP
        - Launching a test on the DP in the Control Panel
        - Deploying the DP
            - Mocked components are acceptable: eg for a template of a Spark workload we do not need to actually implement business logic. We don't need to actually develop on the component for every single feature.
- [x] Documentations have been updated
    - Both the documentation of the template, and the documentation of the component inside the skeleton, must be updated (if one, the other or both of course depends on the kind of work we're doing)
    - Documentation has been updated with explanation of the new feature if it is user-facing (eg component now has additional setting) or it impacts him in some other way (eg optional field that becomes mandatory)
    - If it is a breaking change we have documented them as such in the MR description in a "Breaking Changes" section

## Wizard development
- [x] The wizard design has been implemented according to the provided wireframes, mockups, or design specifications
- [x] The UI elements are visually consistent, aesthetically pleasing, and aligned with the brand guidelines
- [x] Dynamic elements, such as dropdowns, sliders, and modals, function as expected and enhance the user experience. Also edge conditions have been tested
- [x]  Labels everywhere are meaningful to the user. All labels have been reviewed
- [x] Embedded images use sensible hosting strategy
- [x] The wizard steps are clear to the users

## Skeleton/component template development
- [x] If feature affects the skeleton of a component that contains a codebase (eg introducing new feature in the codebase), unit tests for that feature are added to the codebase

README.md

@@ -52,6 +52,16 @@ To verify the component before deploying it along with the Data Product, the com
 For more information, please refer to the [official documentation](https://docs.witboost.agilelab.it/docs/p1_user/p5_managing_policies/p5_1_overview).
 
 
+## Artifacts
+
+This project uses Python `setuptools` and `build` for packaging. Build artifacts with:
+
+```python
+python -m build --wheel
+```
+
+If you wish to change the default version, you can do so by going inside the `version.py` file and change it to suit your needs.
+
 ## License
 
 This project is available under the [Apache License, Version 2.0](https://opensource.org/licenses/Apache-2.0); see [LICENSE](LICENSE) for full details.

skeleton/.gitlab-ci.yml

@@ -0,0 +1,15 @@
+image: python:3.11-slim
+
+stages:
+  - publish
+
+publish:
+  stage: publish
+  before_script:
+    - apt-get update -qy
+    - apt-get install -y git
+    - ls -la
+  script:
+    - pip install setuptools build twine
+    - python -m build --wheel
+    - TWINE_PASSWORD=${CI_JOB_TOKEN} TWINE_USERNAME=gitlab-ci-token python -m twine upload --verbose --repository-url ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/pypi dist/*.whl

skeleton/catalog-info.yaml

@@ -7,9 +7,8 @@ metadata:
     gitlab.com/project-slug: 'AgileDMBSandbox/${{orgname}}/mesh.repository/${{ values.domain | replace(r/domain:| |-/, "") }}/${{ values.dataproduct.split(".")[1] | replace(r/ |-/g, "") }}/${{ values.name.split(" ") | join("") | lower }}'
     backstage.io/techdocs-ref: dir:.
   tags:
-    - aws
     - sql
-    - snowflake
+    - dbt
     - workload
 spec:
   type: workload
@@ -37,4 +36,4 @@ spec:
     specific:
       dbtProjectName: ${{ values.dbtProjectName }}
       gitUrl: |
-        https://gitlab.com/AgileDMBSandbox/${{orgname}}/mesh.repository/${{ values.domain | replace(r/domain:| |-/, "") }}/${{ values.dataproduct.split(".")[1] | replace(r/ |-/g, "") }}/${{ values.name.split(" ") | join("") | lower }}.git
+        https://gitlab.com%2FAgileDMBSandbox%2F${{orgname}}%2Fmesh.repository%2F${{ values.domain | replace(r/domain:| |-/, "") }}%2F${{ values.dataproduct.split(".")[1] | replace(r/ |-/g, "") }}%2F${{ values.name.split(" ") | join("") | lower }}.git

skeleton/pyproject.toml

@@ -0,0 +1,11 @@
+[build-system]
+requires = ["setuptools", "build", "twine", "setuptools-scm"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "dbt_workload"
+dynamic = ["version"]
+
+[tool.setuptools_scm]
+write_to = "version.py"
+fallback_version = "0.1.0"

skeleton/setup.cfg

@@ -0,0 +1,33 @@
+[metadata]
+name = dbt_workload
+author = Agile Lab
+author_email = [email protected]
+description = This package is used as an input to the Specific Provisioner
+maintainer = Agile Lab S.r.l
+license = Apache v2
+classifiers =
+    Programming Language :: Python :: 3
+    Operating System :: OS Independent
+    Topic :: Software Development :: Build Tools
+    License :: OSI Approved :: Apache v2 License
+
+[options]
+python_requires = >=3.7
+packages=find:
+setup_requires =
+    setuptools_scm
+
+[options.packages.find]
+exclude =
+    docs*
+    mkdocs.yml
+    README.md
+
+[options.package_data]
+dbt_workload = version.py
+
+[tool.setuptools_scm]
+write_to = version.py
+fallback_version = 0.1.0
+version_scheme = semver
+local_scheme = no-local-version

skeleton/version.py

@@ -0,0 +1 @@
+__version__ = '0.1.0'

template.yaml

@@ -15,7 +15,7 @@ spec:
   type: workload
 
   parameters:
-    - title: Component basic information
+    - title: Component Basic Information
       required:
         - name
         - domain
@@ -110,6 +110,8 @@ spec:
       input:
         url: ./skeleton
         targetPath: '.'
+        copyWithoutTemplating:
+          - .gitlab-ci.yml
         values:
           name: '${{ parameters.name }}'
           fullyQualifiedName: '${{ parameters.fullyQualifiedName }}'