Skip to content
This repository has been archived by the owner on Oct 16, 2024. It is now read-only.

requirement, venv, conda builder docs #191

Merged
merged 17 commits into from
Oct 27, 2022
Merged
15 changes: 15 additions & 0 deletions content/docs/object-reference/build/requirements.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# requirements

## `class RequirementsBuilder`

**MlemABC parent type**: `builder`

**MlemABC type**: `requirements`

MlemBuilder implementation for building requirements

**Fields**:

- `target: str` - Target path for requirements

- `req_type: str = "installable"` - Type of requirements, example: unix
36 changes: 36 additions & 0 deletions content/docs/object-reference/build/venv.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# venv

## `class CondaBuilder`

**MlemABC parent type**: `builder`

**MlemABC type**: `conda`

MlemBuilder implementation for building conda environments

**Fields**:

- `target: str = "venv"` - Name of the virtual environment

- `python_version: str = "3.9"` - The python version to use

- `current_env: bool = False` - Whether to install in the current conda env

---

## `class VenvBuilder`

**MlemABC parent type**: `builder`

**MlemABC type**: `venv`

MlemBuilder implementation for building virtual environments

**Fields**:

- `target: str = "venv"` - Name of the virtual environment

- `no_cache: bool = False` - Disable cache

- `current_env: bool = False` - Whether to install in the current virtual env,
must be active
39 changes: 32 additions & 7 deletions content/docs/sidebar.json
Original file line number Diff line number Diff line change
Expand Up @@ -96,18 +96,33 @@
},
{
"slug": "building",
"label": "Exporting models",
"label": "Building models",
"source": "building/index.md",
"children": [
{
"slug": "pip",
"label": "Python Packages",
"source": "pip.md"
"slug": "conda",
"label": "Conda Environment",
"source": "conda.md"
},
{
"slug": "docker",
"label": "Docker",
"source": "docker.md"
},
{
"slug": "pip",
"label": "Python Packages",
"source": "pip.md"
},
{
"slug": "requirements",
"label": "Requirements",
"source": "requirements.md"
},
{
"slug": "venv",
"label": "Virtual Environment",
"source": "venv.md"
}
]
},
Expand Down Expand Up @@ -452,15 +467,25 @@
"label": "Builders",
"source": "build/index.md",
"children": [
{
"slug": "docker",
"label": "Docker",
"source": "docker.md"
},
{
"slug": "pip",
"label": "Python package",
"source": "pip.md"
},
{
"slug": "docker",
"label": "Docker",
"source": "docker.md"
"slug": "requirements",
"label": "Requirements",
"source": "requirements.md"
},
{
"slug": "venv",
"label": "Virtual Environment",
"source": "venv.md"
}
]
},
Expand Down
136 changes: 136 additions & 0 deletions content/docs/user-guide/building/conda.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
# Conda Environments

Like [virtual environments](/doc/user-guide/building/venv),
[conda environments](https://docs.conda.io/projects/conda/en/latest/user-guide/concepts/environments.html)
follow the same paradigm of isolating dependencies for a package or a model.
But, they exist globally and are saved in a single location. Further, they don't
need to be limited to installation of python packages.

## Description

Currently, Conda based requirements cannot be determined automatically. But, one
can pass them manually.

In addition to installing conda packages, `pip` based packages (gathered from
the model) will also be installed in the `conda` environment.

## Preparation

Make sure that `conda` command line utility is installed and is accessible.

### Generating a new conda environment

```cli
$ mlem build conda --model model --target newenv \
--conda_reqs.0.package_name xtensor \
--conda_reqs.1.package_name openssl
⏳️ Loading model from model.mlem
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

environment location: /path/to/envs/newenv

added / updated specs:
- python=3.9


The following NEW packages will be INSTALLED:

ca-certificates pkgs/main/osx-arm64::ca-certificates-2022.07.19-hca03da5_0 None
certifi pkgs/main/osx-arm64::certifi-2022.9.24-py39hca03da5_0 None
libcxx pkgs/main/osx-arm64::libcxx-14.0.6-h848a8c0_0 None
libffi pkgs/main/osx-arm64::libffi-3.4.2-hc377ac9_4 None
ncurses pkgs/main/osx-arm64::ncurses-6.3-h1a28f6b_3 None
openssl pkgs/main/osx-arm64::openssl-1.1.1q-h1a28f6b_0 None
pip pkgs/main/osx-arm64::pip-22.2.2-py39hca03da5_0 None
python pkgs/main/osx-arm64::python-3.9.13-hbdb9e5c_1 None
readline pkgs/main/osx-arm64::readline-8.1.2-h1a28f6b_1 None
setuptools pkgs/main/osx-arm64::setuptools-63.4.1-py39hca03da5_0 None
sqlite pkgs/main/osx-arm64::sqlite-3.39.3-h1058600_0 None
tk pkgs/main/osx-arm64::tk-8.6.12-hb8d0fd4_0 None
tzdata pkgs/main/noarch::tzdata-2022c-h04d1e81_0 None
wheel pkgs/main/noarch::wheel-0.37.1-pyhd3eb1b0_0 None
xz pkgs/main/osx-arm64::xz-5.2.6-h1a28f6b_0 None
zlib pkgs/main/osx-arm64::zlib-1.2.12-h5a0b063_3 None


Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate /path/to/envs/newenv
#
# To deactivate an active environment, use
#
# $ conda deactivate

Retrieving notices: ...working... done
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

environment location: /path/to/envs/newenv

added / updated specs:
- conda-forge::openssl
- conda-forge::xtensor


The following NEW packages will be INSTALLED:

xtensor conda-forge/osx-arm64::xtensor-0.24.3-hf86a087_0 None
xtl conda-forge/osx-arm64::xtl-0.7.4-hc021e02_0 None

The following packages will be UPDATED:

openssl pkgs/main::openssl-1.1.1q-h1a28f6b_0 --> conda-forge::openssl-1.1.1q-h03a7124_1 None


Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Retrieving notices: ...working... done
Collecting scikit-learn==1.0.2
Using cached scikit_learn-1.0.2-cp39-cp39-macosx_12_0_arm64.whl (6.9 MB)
Collecting pandas==1.4.2
Using cached pandas-1.4.2-cp39-cp39-macosx_11_0_arm64.whl (10.1 MB)
Collecting numpy==1.22.3
Using cached numpy-1.22.3-cp39-cp39-macosx_11_0_arm64.whl (12.8 MB)
Collecting threadpoolctl>=2.0.0
Using cached threadpoolctl-3.1.0-py3-none-any.whl (14 kB)
Collecting scipy>=1.1.0
Using cached scipy-1.9.2-cp39-cp39-macosx_12_0_arm64.whl (28.6 MB)
Collecting joblib>=0.11
Using cached joblib-1.2.0-py3-none-any.whl (297 kB)
Collecting pytz>=2020.1
Using cached pytz-2022.4-py2.py3-none-any.whl (500 kB)
Collecting python-dateutil>=2.8.1
Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting six>=1.5
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Installing collected packages: pytz, threadpoolctl, six, numpy, joblib, scipy, python-dateutil, scikit-learn, pandas
Successfully installed joblib-1.2.0 numpy-1.22.3 pandas-1.4.2 python-dateutil-2.8.2 pytz-2022.4 scikit-learn-1.0.2 scipy-1.9.2 six-1.16.0 threadpoolctl-3.1.0
```

If the `target` is not passed, the default name for the new environment is
`venv`.

Other options include using:

- `--python_version 3.7` -- to use a custom python version, by default it is
inferred automatically.
- `--current_env True` -- whether to install the requirements in a currently
activated conda environment.

While options for passing a list of conda requirements include:

- `--conda_reqs.0.package_name` -- name of the conda package
- `--conda_reqs.0.spec` -- denotes selectors for a package such as '>=1.8,<2'
(optional)
- `--conda_reqs.0.channel_name` -- denotes the channel from which a package is
to be installed (default is `conda-forge`)
8 changes: 5 additions & 3 deletions content/docs/user-guide/building/index.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Exporting models (building)
# Building models

Building is a way to “bake” your model into something usable in production like
a Docker image, or export your model into another format. You can see the full
list of available builders [here](/doc/object-reference/build).
a Docker image, or export your model into another format or even export the
underlying requirements and dependencies of the model, allowing one to create
virtual environments out of it. You can see the full list of available builders
[here](/doc/object-reference/build).

To build your MLEM model you need to use either
[CLI](/doc/command-reference/build) or [API](/doc/api-reference/build) `build`
Expand Down
Loading