Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update starter tempalte #12

Merged
merged 48 commits into from
Dec 20, 2023
Merged
Show file tree
Hide file tree
Changes from 47 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
f4b9955
Simplified starter
htahir1 Nov 29, 2023
2dd1ac4
Template
htahir1 Dec 1, 2023
ff02dbf
Latest
htahir1 Dec 1, 2023
2d566dc
Steps
htahir1 Dec 1, 2023
8ea0c85
Works until inference pipeline
htahir1 Dec 5, 2023
16d0faa
Works until inference pipeline
htahir1 Dec 5, 2023
5f781bf
Run py cleaned
htahir1 Dec 5, 2023
2f7ef97
Fixing
htahir1 Dec 5, 2023
fd1aad6
Updated
htahir1 Dec 6, 2023
c8bf3e3
Updated
htahir1 Dec 7, 2023
7d3e4bc
new notebook
htahir1 Dec 8, 2023
f3ef39b
new notebook
htahir1 Dec 8, 2023
639ad9e
new notebook
htahir1 Dec 8, 2023
cf08f35
Images
htahir1 Dec 12, 2023
548293d
Cleaned up and finalized for alexej
htahir1 Dec 13, 2023
8557a3d
Initial changes
AlexejPenner Dec 13, 2023
072d3ef
Removed some copier options
AlexejPenner Dec 13, 2023
4259fc8
Compared with e2e template
AlexejPenner Dec 13, 2023
7b218b5
Removed templateized ipynb
AlexejPenner Dec 13, 2023
f99caf2
Many things in this commit - Formatted, darglinted, added complex pro…
htahir1 Dec 14, 2023
d5bc5e6
Latest changes
htahir1 Dec 14, 2023
88036a2
README
htahir1 Dec 14, 2023
fdca688
New readme
htahir1 Dec 14, 2023
8ce022b
LatesT
htahir1 Dec 14, 2023
dfcbe51
LatesT
htahir1 Dec 14, 2023
2af724f
Woot
htahir1 Dec 14, 2023
0374dcd
took out final comment
htahir1 Dec 14, 2023
c04dfbc
Notebook cleaned
htahir1 Dec 14, 2023
4743775
Further cleanup, fix caching and artifact version fetching, move rand…
AlexejPenner Dec 14, 2023
bfd271a
readme update
htahir1 Dec 14, 2023
cddbed5
Fixed non-rendering visualizations
AlexejPenner Dec 14, 2023
9fe56dd
Updated
htahir1 Dec 18, 2023
a2ca4f8
Updated
htahir1 Dec 18, 2023
0238510
new CTA
htahir1 Dec 18, 2023
4f44541
new CTA
htahir1 Dec 18, 2023
066517f
Updated
htahir1 Dec 18, 2023
2806440
add image optimizer
avishniakov Dec 18, 2023
dd35b68
Optimised images with calibre/image-actions
github-actions[bot] Dec 18, 2023
c91cff2
imporve ci
avishniakov Dec 18, 2023
05b7a2e
fix ci
avishniakov Dec 18, 2023
621e38c
fix ci/tests
avishniakov Dec 18, 2023
b00197c
vscode
avishniakov Dec 18, 2023
6f7b6cc
remove mlflow reqs
avishniakov Dec 18, 2023
012096e
fix tests
avishniakov Dec 18, 2023
be772f9
add `--no-cache`
avishniakov Dec 18, 2023
b257658
use SafeLoader
avishniakov Dec 18, 2023
831997c
fix img paths
avishniakov Dec 19, 2023
652a614
YAML requirements fixed
htahir1 Dec 19, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/actions/starter_template_test/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ runs:
- name: Concatenate requirements
shell: bash
run: |
zenml integration export-requirements -o ./local_checkout/integration-requirements.txt sklearn mlflow s3 kubernetes kubeflow slack evidently
zenml integration export-requirements -o ./local_checkout/integration-requirements.txt sklearn
cat ./local_checkout/requirements.txt ./local_checkout/test-requirements.txt ./local_checkout/integration-requirements.txt >> ./local_checkout/all-requirements.txt

- name: Install requirements
Expand Down
21 changes: 21 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,25 @@ name: CI

on:
workflow_dispatch:
inputs:
ref-template:
description: 'Branch or tag ref to check out for template'
type: string
required: false
ref-zenml:
description: 'Branch or tag ref to check out for ZenML'
type: string
required: false
workflow_call:
inputs:
ref-template:
description: 'Branch or tag ref to check out for template'
type: string
required: false
ref-zenml:
description: 'Branch or tag ref to check out for ZenML'
type: string
required: false
push:
branches: ["main", "develop"]
paths-ignore: ["README.md"]
Expand All @@ -19,6 +37,7 @@ jobs:
run-tests:
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
stack-name: [local]
os: [windows-latest, ubuntu-latest, macos-latest]
Expand All @@ -36,3 +55,5 @@ jobs:
with:
stack-name: ${{ matrix.stack-name }}
python-version: ${{ matrix.python-version }}
ref-zenml: ${{ inputs.ref-zenml || 'feature/update-quickstart-from-template' }}
ref-template: ${{ inputs.ref-template || github.ref }}
26 changes: 26 additions & 0 deletions .github/workflows/image-optimizer.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Compress Images
on:
pull_request:
# Run Image Actions when JPG, JPEG, PNG or WebP files are added or changed.
# See https://help.github.com/en/actions/automating-your-workflow-with-github-actions/workflow-syntax-for-github-actions#onpushpull_requestpaths for reference.
paths:
- '**.jpg'
- '**.jpeg'
- '**.png'
- '**.webp'
jobs:
build:
# Only run on non-draft PRs within the same repository.
if: github.event.pull_request.head.repo.full_name == github.repository && github.event.pull_request.draft == false
name: calibreapp/image-actions
runs-on: ubuntu-latest
steps:
- name: Checkout Repo
uses: actions/checkout@v3

- name: Compress Images
uses: calibreapp/image-actions@main
with:
# The `GITHUB_TOKEN` is automatically generated by GitHub and scoped only to the repository that is currently running the action. By default, the action can’t update Pull Requests initiated from forked repositories.
# See https://docs.github.com/en/actions/reference/authentication-in-a-workflow and https://help.github.com/en/articles/virtual-environments-for-github-actions#token-permissions
githubToken: ${{ secrets.GITHUB_TOKEN }}
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,9 @@ __pypackages__/
celerybeat-schedule
celerybeat.pid

# PyCharm Stuff
.idea

# SageMath parsed files
*.sage.py

Expand Down Expand Up @@ -127,3 +130,6 @@ dmypy.json

# Pyre type checker
.pyre/

*.zen
.vscode
25 changes: 10 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,16 @@
# 📜 ZenML Project Templates

This repository contains a collection of templates from which a ZenML project
can be generated: a collection of steps, pipelines, stack configurations and
This repository contains a starter template from which a simple ZenML project
can be generated easily. It contains a collection of steps, pipelines, stack configurations and
other artifacts and useful resources that can get you started with ZenML.

🔥 **Do you have a personal project powered by ZenML that you would like to see here?** At
ZenML, we are looking for design partnerships and collaboration to help us
🔥 **Do you have a personal project powered by ZenML that you would like to see here?**

At ZenML, we are looking for design partnerships and collaboration to help us
better understand the real-world scenarios in which MLOps is being used and to
build the best possible experience for our users. If you are interested in
sharing all or parts of your project with us in the form of a ZenML project
template, please [join our Slack](https://zenml.io/slack-invite/) and leave us a
template, please [join our Slack](https://zenml.io/slack/) and leave us a
message!

## 📦 Prerequisites
Expand All @@ -18,7 +19,7 @@ To use the templates, you need to have Zenml and its `templates` extras
installed:

```bash
pip install zenml[templates]
pip install "zenml[templates]"
```

## 🚀 Generate a ZenML Project
Expand All @@ -35,7 +36,7 @@ library and a set of Jinja2 templates to generate the project. So you may also
interact with Copier directly to generate a project, e.g.:

```bash
copier gh:zenml-io/zenml-project-templates <directory>
copier gh:zenml-io/template-starter <directory>
```

You will be prompted to select the project template and enter various values for
Expand All @@ -47,11 +48,5 @@ the same command again. If you want to skip the prompts to use the values you
already entered and overwrite all files in the existing project, you can run:

```bash
copier -wf gh:zenml-io/zenml-project-templates <directory>
```

## 📃 List of Project Templates

| Project Template | Tags | Description |
|------------------|----------|-----------------------------------------------------------------------------------|
| [ZenML Starter](https://github.com/zenml-io/zenml-project-templates/tree/main/starter) | basic scikit-learn | All the basic ML ingredients you need to get you started with ZenML: parameterized steps, a model training pipeline, a flexible configuration and a simple CLI. All created around a representative and versatile model training use-case implemented with the scikit-learn library. |
copier -wf gh:zenml-io/template-starter <directory>
```
70 changes: 1 addition & 69 deletions copier.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,4 @@
--- # GLOBAL PROMPT --------------------------------

template:
type: str
help: >-
The project template you would like to use
choices:
ZenML Starter: starter
default: starter
project_name:
type: str
help: Short name for your project
Expand Down Expand Up @@ -40,70 +32,10 @@ email:
The email of the person/entity holding the copyright
default: [email protected]
when: "{{ open_source_license }}"
auto_format:
type: bool
help: "Auto-format the generated code with black, ruff and autoflake? \n (NOTE: you need to have these tools installed in your environment)"
default: false

# SUB-TEMPLATE PROMPT [starter] ---------------------
use_step_params:
type: bool
help: "Would you like to see step parameters being used in the generated \n ZenML steps?"
default: true
when: "{{ template == 'starter' }}"
use_custom_artifacts:
type: bool
help: "Would you like to see custom artifact data types and materializers \n being used in the generated ZenML steps and pipelines?"
default: true
when: "{{ template == 'starter' }}"
configurable_dataset:
type: bool
help: "Would you like to be able to select the dataset used in model training\n at runtime through the CLI and as a data loader step parameter?"
default: true
when: "{{ template == 'starter' and use_step_params }}"
sklearn_dataset_name:
type: str
help: "The generated ZenML project will be populated with some example code
\n featuring one of the UCI ML datasets from scikit-learn. \n Which
dataset would you like to see being used in the generated code?"
choices:
UCI Wine Data Set (classification): wine
UCI Iris Data Set (classification): iris
UCI Breast Cancer Wisconsin (Diagnostic) Data Set (classification): breast_cancer
default: wine
when: "{{ template == 'starter' }}"
configurable_model:
type: bool
help: "Would you like to be able to select the type of model used in model \n
training at runtime through the CLI and as a model trainer step \n
parameter?"
default: true
help: >-
Would you like to be able to select the type of model used in model
training at runtime through the CLI and as a model trainer step
parameter ?
default: true
when: "{{ template == 'starter' and use_step_params }}"
sklearn_model_name:
type: str
help: "The generated ZenML project will be populated with some example code
\n featuring one of the scikit-learn classifier models. \n Which model class would you like to see being used in the generated code?"
choices:
Logistic Regression: LogisticRegression
C-Support Vector Classification: SVC
Linear Support Vector Classification: LinearSVC
Random Forest Classifier: RandomForestClassifier
K-Nearest Neighbors Classifier: KNeighborsClassifier
Gaussian Naive Bayes: GaussianNB
Linear Perceptron Classifier: Perceptron
Stochastic Gradient Descent Linear Classifier: SGDClassifier
Decision Tree Classifier: DecisionTreeClassifier
default: LogisticRegression
when: "{{ template == 'starter' }}"

# CONFIGURATION -------------------------
_templates_suffix: ""
_subdirectory: "{{ template }}/template"
_subdirectory: "./template"
_exclude:
- /README.md
_tasks:
Expand Down
4 changes: 3 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
scikit-learn
scikit-learn<1.3
copier
jinja2-time
zenml[server]>=0.52.0
notebook
53 changes: 0 additions & 53 deletions starter/README.md

This file was deleted.

Loading
Loading