Searchspace improvements and project meta modernization (#214)

* Implemented PySMT in Kernel Tuner

* Added mapping of parameter names to integers for better perfomance

* Improved tests and compatibility for parameter mapping

* Switch from setup.py/cfg to pyproject.toml

* Added linting information, VS Code settings and recommendations

* Always return compiled functions as a list, application of formatter

* Improvements to string to restriction parsing

* Major speedup due to restriction splitting with parameter usage detection

* Refactored tests for parsing restrictions due to changes

* Minor improvements to an assert and a testcase

* Searchspace list to NumPy conversion only happens if needed

* Searchspace objects can now be initialized from ATF logfiles (caches), unified searchspace builder returns

* Added safeguard to prevent changing ATF block size names after compilation stage

* Fixed Constraint conversions

* Changed to non-forwardchecking by default, moved MaxProdConstraint to constraint

* Parsing of common operations to python-constraint built-in Constraints

* Fixed an issue with converting restrictions to Constraints

* Enabled converting to Constraint with '>' and '<' as well, added use of MinProdConstraint

* Added mapping of '>' to '>=' and '<' to '<=' for single-variable restriction

* Return None on trying to convert to numeric Constraint

* Automatic transformation of restrictions with multiple comparators to multiple restrictions with a single comparator, increasing the chance of converting to a built-in restriction

* Added support for PySMT on generated searchspaces

* Several improvements and fixes for PySMT builder, especially for preserving order

* Fixed an issue where restrictions would not be parsed to specific constraints, added tests for conversion to constraints, added requirement for Python version

* Removed redundant parameter mapping

* Minor fixes for testing

* Made scikit-opt (skopt) optional in strategy bayes_opt

* Changed to Poetry and pyproject.toml

* Added tests for pyproject.toml

* Updated changelog to reflect changes, updated VS Code settings

* Updated documentation to use pyproject file metadata, other minor updates

* Minor bugfixes based on SonarCloud

* Re-added support for Python 3.8, updated dependencies accordingly

* Updated GitHub Action workflows to use Poetry and Nox

* Removed unnecessary pull_request trigger for howfairis, added manual trigger for build & test

* Updated workflows with new Setup Nox 2 and new checkout

* Removed Windows from test OSes

* Updated publishing workflow, expanded TODOs in changelog

* Finished PyPI publication workflow with Poetry

* Fixed errors and warnings with the docs, as well as compatiblity issues

* Updated dependencies

* Updated documentation for installation, development setup etc.

* Minor updates: include notebook files in package, version bump

* Removed usage of OrderedDict as per issue #209, minor changes to tests for this, reogranized imports and made sure some appearantly unused imports are not automatically removed

* Updated test workflow to include upload to CodeCov for each OS, improved Noxfile so only the last test generates coverage report, added PyPI classifiers

* Disable automatic upload to CodeCov for now

* Solved issue #139: Reimplemented Latin Hypercube Sampling with SciPy

* Improved Nox with optional dependencies for CUDA, HIP, OpenCL and arguments to disable these

* Updated Contributing and Installation guidelines, specifically added development environment setup instructions, also added code syntax highlighting for Sphinx

* Minor changes to dependencies and environment setup

* Minor changes to dependencies and environment setup

* Added bruteforce solver to Searchspace object, improved tests for Searchspace

* Solved a bug in the constraints parsing that caused wrong searchspace outcomes, added and expanded searchspace tests, resolved warnings

* Added missing nox-poetry to test dependencies

* Added nox-poetry to GitHub test Action

* Added Poetry setup to Github Test Action

* Added extensive development environment setup instructions, updated installation documentation

* Add an exception for the nvml_ parameter check if in simulation mode

* Added optional additional (non-dependency) installation to Noxfile, with CUDA version differentiation. Added a tolerance to the energy power frequency model test.

* Fixed additional test input argument

* Improved detection of CUDA version using NVCC

* Improved automatic selection of cupy prebuilt version, from most exact to most general, and improved warnings

* Set the default environment used by Nox using a file

* Updated the documentation for the Noxenv file and other minor improvements

* Updated VS code settings for testing from VS code, updated documentation

* Fixed HIP import error, made backend import error messages point to documentation

* Restored accidentally removed import

* Added an option for Nox to remove the other environment caches before each session is ran, and to clean up temporary files

* Temporarily skip broken HIP tests as per issue #217, avoided error in Noxfile when no temporary files are present

* Improved Noxfile functionality for removing environments after use, and documentation on this feature

.github/workflows/cffconvert.yml

@@ -1,19 +1,20 @@
-name: cffconvert
+# This workflow validates the citation file in the repository
+
+name: Citation file validation
 
 on:
-  push:
-    paths:
-      - CITATION.cff
+    push:
+        paths:
+            - CITATION.cff
 
 jobs:
-  validate:
-    name: "validate"
-    runs-on: ubuntu-latest
-    steps:
-      - name: Check out a copy of the repository
-        uses: actions/checkout@v2
+    validate:
+        runs-on: ubuntu-latest
+        steps:
+            - name: Check out a copy of the repository
+              uses: actions/checkout@v4
 
-      - name: Check whether the citation metadata from CITATION.cff is valid
-        uses: citation-file-format/[email protected]
-        with:
-          args: "--validate"
+            - name: Check whether the citation metadata from CITATION.cff is valid
+              uses: citation-file-format/[email protected]
+              with:
+                  args: "--validate"

.github/workflows/docs-on-release.yml

@@ -1,42 +1,44 @@
-name: Create versioned documentation on release
+name: Build versioned documentation on release
 
 on:
-  release:
-    types: [published]
+    release:
+        types: [published]
 
-  workflow_dispatch:
+    # Allows you to run this workflow manually from the Actions tab
+    workflow_dispatch:
 
 jobs:
-  build:
-    environment: dev_environment
-    runs-on: ubuntu-latest
-    steps:
-    - name: Checkout
-      uses: actions/checkout@master
-      with:
-        fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
-    - name: Set env
-      run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
-    - name: Install dependencies
-      run: |
-        sudo apt-get update;
-        sudo apt-get install pandoc
-        python -m pip install --upgrade pip
-        pip install .[doc]
-    - name: Build and Commit
-      uses: sphinx-notes/pages@v2
-      with:
-        documentation_path: doc/source
-        target_path: ${{ env.RELEASE_VERSION }}
-    - name: Redirect stable to new release
-      run: |
-          echo "Redirecting stable to newly released version " $RELEASE_VERSION
-          rm -rf stable
-          ln -s $RELEASE_VERSION stable
-          git add stable
-          git commit -m "redirect stable to new version $RELEASE_VERSION"
-    - name: Push changes
-      uses: ad-m/github-push-action@master
-      with:
-        github_token: ${{ secrets.GITHUB_TOKEN }}
-        branch: gh-pages
+    build:
+        environment: dev_environment
+        runs-on: ubuntu-latest
+        steps:
+            - name: Checkout
+              uses: actions/checkout@v4
+              with:
+                  fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
+            - name: Set env
+              run: echo "RELEASE_VERSION=${GITHUB_REF#refs/*/}" >> $GITHUB_ENV
+            - name: Install dependencies
+              run: |
+                  sudo apt-get update;
+                  sudo apt-get install pandoc
+                  python -m pip install --upgrade pip
+                  pip install poetry
+                  poetry install --with docs
+            - name: Build and Commit
+              uses: sphinx-notes/pages@v2 # NOTE when switching to v3, export dependencies to requirements.txt in pyproject.toml: `poetry export --with docs --without-hashes --format=requirements.txt > docs/requirements.txt`
+              with:
+                  documentation_path: doc/source
+                  target_path: ${{ env.RELEASE_VERSION }}
+            - name: Redirect stable to new release
+              run: |
+                  echo "Redirecting stable to newly released version " $RELEASE_VERSION
+                  rm -rf stable
+                  ln -s $RELEASE_VERSION stable
+                  git add stable
+                  git commit -m "redirect stable to new version $RELEASE_VERSION"
+            - name: Push changes
+              uses: ad-m/github-push-action@master
+              with:
+                  github_token: ${{ secrets.GITHUB_TOKEN }}
+                  branch: gh-pages

.github/workflows/docs.yml

@@ -1,40 +1,36 @@
-# This is a basic workflow to help you get started with Actions
 name: Build documentation
 
-# Controls when the workflow will run
 on:
-  # Triggers the workflow on push or pull request events but only for the master branch
-  push:
-    branches: [ master ]
+    push:
+        branches: [master]
 
-  # Allows you to run this workflow manually from the Actions tab
-  workflow_dispatch:
+    # Allows you to run this workflow manually from the Actions tab
+    workflow_dispatch:
 
-# A workflow run is made up of one or more jobs that can run sequentially or in parallel
 jobs:
-  # This workflow contains a single job called "build"
-  build:
-
-    # The type of runner that the job will run on
-    runs-on: ubuntu-latest
-    steps:
-    - name: Checkout
-      uses: actions/checkout@master
-      with:
-        fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
-    - name: Install dependencies
-      run: |
-        sudo apt-get update;
-        sudo apt-get install pandoc
-        python -m pip install --upgrade pip
-        pip install .[doc]
-    - name: Build and Commit
-      uses: sphinx-notes/pages@v2
-      with:
-        documentation_path: doc/source
-        target_path: latest
-    - name: Push changes
-      uses: ad-m/github-push-action@master
-      with:
-        github_token: ${{ secrets.GITHUB_TOKEN }}
-        branch: gh-pages
+    # This workflow contains a single job called "build"
+    build:
+        # The type of runner that the job will run on
+        runs-on: ubuntu-latest
+        steps:
+            - name: Checkout
+              uses: actions/checkout@v4
+              with:
+                  fetch-depth: 0 # otherwise, you will failed to push refs to dest repo
+            - name: Install dependencies
+              run: |
+                  sudo apt-get update;
+                  sudo apt-get install pandoc
+                  python -m pip install --upgrade pip
+                  pip install poetry
+                  poetry install --with docs
+            - name: Build and Commit
+              uses: sphinx-notes/pages@v2 # NOTE when switching to v3, export dependencies to requirements.txt in pyproject.toml: `poetry export --with docs --without-hashes --format=requirements.txt > docs/requirements.txt`
+              with:
+                  documentation_path: doc/source
+                  target_path: latest
+            - name: Push changes
+              uses: ad-m/github-push-action@master
+              with:
+                  github_token: ${{ secrets.GITHUB_TOKEN }}
+                  branch: gh-pages

.github/workflows/publish-python-package.yml

@@ -0,0 +1,56 @@
+# This workflow checks out a new release, builds it as a package (source and wheel) and publishes it to PyPI.
+
+name: Publish Package
+
+# Controls when the workflow will run
+on:
+    # Workflow will run when a release has been published for the package
+    release:
+        types:
+            - published
+
+    # Allows you to run this workflow manually from the Actions tab
+    workflow_dispatch:
+
+jobs:
+    build_and_publish_as_package:
+        name: Package and upload release to PyPI
+        runs-on: ubuntu-latest
+        environment:
+            name: pypi
+            url: https://pypi.org/p/kernel_tuner
+        permissions:
+            id-token: write # IMPORTANT: this permission is mandatory for trusted publishing
+        steps:
+            - uses: actions/checkout@v4
+            - name: Setup Python
+              uses: actions/setup-python@v4
+              with:
+                  python-version: "3.11"
+            - name: Setup Poetry
+              uses: Gr1N/setup-poetry@v8
+            - name: Build the source distribution and pure-Python wheel
+              run: |
+                  poetry install
+                  poetry build
+                  ls ./dist
+            - name: Check that the number of wheels is as expected and there is one source distribution
+              run: |
+                  SOURCES_COUNT=$(ls -lR ./dist/*.tar.gz | wc -l)
+                  echo "Number of source distributions: $SOURCES_COUNT"
+                  if [ "$SOURCES_COUNT" -ne 1 ]; then
+                    echo "::error::Number of source distributions $SOURCES_COUNT not equal to 1"
+                    exit 1;
+                  fi
+
+                  EXPECTED_WHEELS_COUNT=1
+                  WHEELS_COUNT=$(ls -lR ./dist/*.whl | wc -l)
+                  echo "Number of wheel distributions: $WHEELS_COUNT"
+                  if [ "$WHEELS_COUNT" -ne "$EXPECTED_WHEELS_COUNT" ]; then
+                    echo "::error::Number of wheel distributions $WHEELS_COUNT not equal to $EXPECTED_WHEELS_COUNT"
+                    exit 1;
+                  fi
+            - name: Publish package distributions to PyPI
+              uses: pypa/gh-action-pypi-publish@release/v1
+              with:
+                  skip-existing: true

.github/workflows/test-python-package.yml

@@ -0,0 +1,42 @@
+# This workflow will use Nox to run tests and lint for the supported Python versions, and upload the test coverage data.
+
+name: Test
+
+on:
+    push:
+        branches:
+            - master
+            - release/*
+    pull_request:
+        branches:
+            - master
+
+    # Allows you to run this workflow manually from the Actions tab
+    workflow_dispatch:
+
+jobs:
+    build:
+        name: Test on ${{ matrix.os }} with all supported Python versions
+        runs-on: ${{ format('{0}-latest', matrix.os)  }} # "-latest" is added here so we can use OS in the format expected by CodeCov
+
+        strategy:
+            matrix:
+                os: [ubuntu, macos]
+
+        steps:
+            - uses: actions/checkout@v4
+            - name: Setup Nox
+              uses: fjwillemsen/[email protected]
+            - name: Setup Poetry
+              uses: Gr1N/setup-poetry@v8
+            - name: Run tests with Nox
+              run: |
+                  pip install nox-poetry
+                  nox -- skip-gpu
+            # - name: Upload Coverage report to CodeCov
+            #   uses: codecov/codecov-action@v3
+            #   with:
+            #       token: ${{ secrets.CODECOV_TOKEN }}
+            #       files: ./coverage
+            #       os: ${{ matrix.os }}
+            #       fail_ci_if_error: false # option to Specify if CI pipeline should fail when Codecov runs into errors during upload