Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: binary execution mode compatibility #406

Merged
merged 40 commits into from
Feb 8, 2024
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
89f23c8
fix: fixing vanity address and init command issues with pyinstaller
negar-abbasi Feb 2, 2024
df11673
test: adding tests
negar-abbasi Feb 2, 2024
c5e48dc
chore: refinements
aorumbayev Feb 2, 2024
59d6502
test: some test refinements
negar-abbasi Feb 5, 2024
bb9560f
chore: minor tweaks
negar-abbasi Feb 5, 2024
dbc8bb1
fix: fixing variable names
negar-abbasi Feb 6, 2024
f62db7a
fix: fixing based on PR comments
negar-abbasi Feb 6, 2024
18813d1
test: fixing test
negar-abbasi Feb 6, 2024
d12b00f
test: add email env
negar-abbasi Feb 6, 2024
9865467
test: add --no-git to the tests
negar-abbasi Feb 6, 2024
8d0731c
chore: adding git identity prior to pytest in ci
aorumbayev Feb 6, 2024
c7fa6bf
chore: testing ci
aorumbayev Feb 6, 2024
8e55080
chore: adding portability tests
aorumbayev Feb 6, 2024
932bac7
chore: testing ci
aorumbayev Feb 6, 2024
733a406
chore: testing ci
aorumbayev Feb 6, 2024
a7677d2
chore: testing ci
aorumbayev Feb 6, 2024
060528c
chore: test ci
aorumbayev Feb 6, 2024
eb7d156
chore: test
aorumbayev Feb 6, 2024
0658976
chore: testing ci
aorumbayev Feb 6, 2024
79a2212
chore: test ci
aorumbayev Feb 6, 2024
33cf287
test: fixing test names-PR reviews
negar-abbasi Feb 7, 2024
eaf21ef
docs: add docs about python_path usage in templates
negar-abbasi Feb 7, 2024
d94bb6d
docs: change wordings
negar-abbasi Feb 7, 2024
3d1e6a0
chore: replacing init test to run against dummy template
aorumbayev Feb 7, 2024
fabb87a
chore: addressing pr comments
aorumbayev Feb 7, 2024
fa13da8
chore: testing ci
aorumbayev Feb 7, 2024
368f44b
chore: test
aorumbayev Feb 7, 2024
1897edb
chore: test
aorumbayev Feb 7, 2024
b5d6671
chore: test
aorumbayev Feb 7, 2024
fc44437
chore: testing ci
aorumbayev Feb 7, 2024
97dba7d
chore: testing ci
aorumbayev Feb 7, 2024
b1c83d5
chore: test
aorumbayev Feb 7, 2024
189145d
chore: testing ci
aorumbayev Feb 7, 2024
062f98c
chore: minor refinements
aorumbayev Feb 7, 2024
4d89eb5
chore: minor tweaks
aorumbayev Feb 7, 2024
eae58b5
fix: addressing pr comments
aorumbayev Feb 7, 2024
f0c3c9c
test: fix a typo in test
negar-abbasi Feb 7, 2024
747655c
chore: fixing generator test for windows
aorumbayev Feb 7, 2024
342d51d
fix: PR comments
negar-abbasi Feb 8, 2024
6be3146
docs: updating tutorial docs
aorumbayev Feb 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 34 additions & 14 deletions .github/actions/build-binaries/action.yaml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
name: "Setup, Build, and Test"
description: "Set up Python with Poetry, build and test binaries"
name: "Setup, Build, and Test Pyinstaller Binaries"
description: "Build, test and upload binaries to GitHub releases and/or artifacts (ensure poetry dependencies are installed before running this action)"
inputs:
package_name:
description: "The name of the package to build and test"
required: true
upload_binaries:
production_release:
description: "Flag to determine if this is a production release"
required: true
operating_system:
Expand All @@ -24,41 +24,61 @@ runs:
run: ${{ inputs.build_command }}
shell: bash

- name: Test Executable
- name: Set TEST_CLI_PATH
run: |
ls -l dist
./dist/${{ inputs.package_name }}/${{ inputs.package_name }}${{ inputs.operating_system == 'windows-latest' && '.exe' || '' }} --help
ls -la dist/algokit/
aorumbayev marked this conversation as resolved.
Show resolved Hide resolved
TEST_CLI_PATH="${{ github.workspace }}/dist/${{ inputs.package_name }}/${{ inputs.package_name }}"
if [ "${{ inputs.operating_system }}" == "Windows" ]; then
TEST_CLI_PATH="${{ github.workspace }}\dist\${{ inputs.package_name }}\${{ inputs.package_name }}.exe"
fi
echo "TEST_CLI_PATH=$TEST_CLI_PATH" >> $GITHUB_ENV
shell: bash

- name: Run portability tests
run: |
echo "Executing: ${{ env.TEST_CLI_PATH }}"
git config --global user.email "[email protected]" && git config --global user.name "github-actions"
poetry run pytest tests/ -m pyinstaller_binary_tests --cli_path "${{ env.TEST_CLI_PATH }}" --log-cli-level=INFO
shell: ${{ inputs.operating_system == 'Windows' && 'cmd' || 'bash' }}

- name: Set release version
shell: bash
continue-on-error: true
if: ${{ inputs.upload_binaries == 'true' }}
if: ${{ inputs.production_release == 'true' }}
run: |
echo "RELEASE_VERSION=$(git describe --tags $(git rev-list --tags --max-count=1))" >> $GITHUB_ENV
git describe --tags $(git rev-list --tags --max-count=1)

- name: Zip binaries
shell: bash
continue-on-error: true
if: ${{ inputs.upload_binaries == 'true' }}
run: |
ls -l dist
cd dist/algokit/
tar -zcvf ../../algokit-${{ env.RELEASE_VERSION }}-${{ inputs.operating_system }}-py${{ inputs.python_version }}.tar.gz *
if [ "${{ inputs.production_release }}" == "true" ]; then
tar -zcvf ../../algokit-${{ env.RELEASE_VERSION }}-${{ inputs.operating_system }}-py${{ inputs.python_version }}.tar.gz *
else
tar -zcvf ../../algokit-${{ inputs.operating_system }}-py${{ inputs.python_version }}.tar.gz *
fi
cd ../..
ls -l

- name: Upload binary as artifact
if: ${{ inputs.upload_binaries == 'true' }}
- name: Upload binary as artifact (release)
if: ${{ inputs.production_release == 'true' }}
uses: actions/upload-artifact@v4
with:
name: algokit-cli-${{ inputs.operating_system }}-py${{ inputs.python_version }}
name: algokit-${{ inputs.operating_system }}-py${{ inputs.python_version }}
path: algokit-${{ env.RELEASE_VERSION }}-${{ inputs.operating_system }}-py${{ inputs.python_version }}.tar.gz

- name: Upload binary as artifact (dev)
if: ${{ inputs.production_release != 'true' }}
uses: actions/upload-artifact@v4
with:
name: algokit-${{ inputs.operating_system }}-py${{ inputs.python_version }}
path: algokit-${{ inputs.operating_system }}-py${{ inputs.python_version }}.tar.gz

- name: Append binary to release
continue-on-error: true
if: ${{ inputs.upload_binaries == 'true' }}
if: ${{ inputs.production_release == 'true' }}
uses: softprops/action-gh-release@v1
with:
files: |
Expand Down
121 changes: 21 additions & 100 deletions .github/workflows/build-binaries.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,133 +3,54 @@ name: Build, Test and Publish Pyinstaller Binaries
on:
workflow_call:
inputs:
upload_binaries:
production_release:
required: true
type: string
python_version:
required: true
type: string

jobs:
build-binaries-ubuntu:
runs-on: ubuntu-latest
build-binaries:
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
include:
- os: ubuntu-latest
build_command: "poetry run poe package_unix"
- os: windows-latest
build_command: "poetry run poe package_windows"
- os: macos-latest
build_command: "poetry run poe package_unix"
steps:
- name: Checkout source code (for release)
- name: Checkout source code
uses: actions/checkout@v4
if: ${{ inputs.upload_binaries == 'true' }}
with:
fetch-depth: 0

- name: Checkout source code (for build)
uses: actions/checkout@v4
if: ${{ inputs.upload_binaries != 'true' }}
with:
fetch-depth: 1

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ inputs.python_version }}

- name: Set up Poetry
uses: ./.github/actions/setup-poetry

- uses: actions/cache@v4
name: Setup poetry cache
with:
path: ./.venv
key: venv-${{ hashFiles('poetry.lock') }}-${{ runner.os }}-${{ inputs.python_version }}

- name: Install dependencies
run: poetry install --no-interaction

- name: Build linux binary
uses: ./.github/actions/build-binaries
with:
python_version: ${{ inputs.python_version }}
package_name: "algokit"
upload_binaries: ${{ inputs.upload_binaries }}
operating_system: ${{ runner.os }}
build_command: "poetry run poe package_unix"

build-binaries-windows:
runs-on: windows-latest
steps:
- name: Checkout source code (for release)
uses: actions/checkout@v4
if: ${{ inputs.upload_binaries == 'true' }}
with:
fetch-depth: 0

- name: Checkout source code (for build)
uses: actions/checkout@v4
if: ${{ inputs.upload_binaries != 'true' }}
with:
fetch-depth: 1

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: ${{ inputs.python_version }}

- name: Set up Poetry
uses: ./.github/actions/setup-poetry

- uses: actions/cache@v4
name: Setup poetry cache
with:
path: ./.venv
key: venv-${{ hashFiles('poetry.lock') }}-${{ runner.os }}-${{ inputs.python_version }}

- name: Install dependencies
run: poetry install --no-interaction

- name: Build windows binary
uses: ./.github/actions/build-binaries
with:
python_version: ${{ inputs.python_version }}
package_name: "algokit"
upload_binaries: ${{ inputs.upload_binaries }}
operating_system: ${{ runner.os }}
build_command: "poetry run poe package_windows"

build-binaries-macos:
runs-on: macos-latest
steps:
- name: Checkout source code (for release)
uses: actions/checkout@v4
if: ${{ inputs.upload_binaries == 'true' }}
with:
fetch-depth: 0

- name: Checkout source code (for build)
uses: actions/checkout@v4
if: ${{ inputs.upload_binaries != 'true' }}
with:
fetch-depth: 1
fetch-depth: ${{ inputs.production_release == 'true' && '0' || '1' }}

- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ inputs.python_version }}

- name: Set up Poetry
uses: ./.github/actions/setup-poetry

- uses: actions/cache@v4
name: Setup poetry cache
- name: Setup poetry cache
uses: actions/cache@v4
with:
path: ./.venv
key: venv-${{ hashFiles('poetry.lock') }}-${{ runner.os }}-${{ inputs.python_version }}

- name: Install dependencies
run: poetry install --no-interaction

- name: Build macos binary
- name: Build & test binary
uses: ./.github/actions/build-binaries
with:
python_version: ${{ inputs.python_version }}
package_name: "algokit"
upload_binaries: ${{ inputs.upload_binaries }}
production_release: ${{ inputs.production_release }}
operating_system: ${{ runner.os }}
build_command: "poetry run poe package_unix"
build_command: ${{ matrix.build_command }}
4 changes: 3 additions & 1 deletion .github/workflows/build-python.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}

Expand All @@ -35,8 +35,10 @@ jobs:

- name: pytest + coverage
shell: bash
# git config is needed due to several tests relying on e2e copier invocation and copier relies on git during `copy` command
run: |
set -o pipefail
git config --global user.email "[email protected]" && git config --global user.name "github-actions"
poetry run pytest -n auto --junitxml=pytest-junit.xml --cov-report=term-missing:skip-covered --cov=src | tee pytest-coverage.txt
id: pytest

Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/cd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ jobs:
fetch-depth: 0

- name: Set up Python
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: "3.10"

Expand Down Expand Up @@ -123,7 +123,7 @@ jobs:
uses: ./.github/workflows/build-binaries.yaml
needs: release
with:
upload_binaries: "true"
production_release: "true"
python_version: "3.12"

cd-publish-release-packages:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/check-python.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
uses: actions/checkout@v4

- name: Set up Python 3.12
uses: actions/setup-python@v4
uses: actions/setup-python@v5
with:
python-version: "3.12"

Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/pr.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,14 +12,14 @@ jobs:
uses: ./.github/workflows/check-python.yaml

pr-build:
name: Build and Test Python
name: Build & Test Python
needs: pr-check
uses: ./.github/workflows/build-python.yaml

pr-binaries-build:
name: Build Binaries
name: Build & Test Binaries
needs: pr-check
uses: ./.github/workflows/build-binaries.yaml
with:
upload_binaries: "false"
production_release: "false"
python_version: "3.12"
17 changes: 17 additions & 0 deletions docs/tutorials/algokit-template.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,23 @@ When creating an AlgoKit template, there are a few default behaviors that you ca

By combining predefined Copier answers with these default behaviors, you can create a smooth, efficient, and intuitive initialization experience for the users of your template.

### Executing Python Tasks in Templates
If you need to use Python scripts as tasks within your Copier templates, ensure that you have Python installed on the host machine.
By convention, AlgoKit automatically detects the Python installation on your machine and fills in the `python_path` variable accordingly.
This process ensures that any Python scripts included as tasks within your Copier templates will execute using the system's Python interpreter.
It's important to note that the use of `_copier_python` is not recommended. Here's an example of specifying a Python script execution in your `copier.yaml` without needing to explicitly use `_copier_python`:

```yaml
- '{{ python_path }} inject.py'
```
And to define `python_path` in your Copier questions:

```yaml
python_path:
type: str
help: Path to the Python executable.
```

### Working with Generators

After mastering the use of `copier` and building your templates based on the official AlgoKit template repositories, you can enhance your proficiency by learning to define `custom generators`. Essentially, generators are smaller-scope `copier` templates designed to provide additional functionality after a project has been initialized from the template.
Expand Down
Loading