Skip to content
This repository has been archived by the owner on Oct 16, 2024. It is now read-only.

Content for /docs #43

Merged
merged 129 commits into from
May 24, 2022
Merged
Show file tree
Hide file tree
Changes from 104 commits
Commits
Show all changes
129 commits
Select commit Hold shift + click to select a range
6585bc2
documentation stubs
mike0sv Apr 14, 2022
c879e62
docs structure
mike0sv Apr 21, 2022
45d27ec
Add gatsby-theme-iterative
rogermparent Apr 22, 2022
f7f9e02
Remove preact in favor of React 18
rogermparent Apr 23, 2022
be5a6a4
Apply list-style: none where necessary
rogermparent Apr 25, 2022
e8f6d1f
Fix header by adding a container and bg color
rogermparent Apr 25, 2022
e007489
Move Header and Footer into theme global with containers
rogermparent Apr 26, 2022
14bdc98
Merge branch 'main' of github.com:iterative/mlem.ai into docs-engine
rogermparent Apr 26, 2022
436c3a0
Use old site's custom media config
rogermparent Apr 26, 2022
b6d0c20
Add docs link
rogermparent Apr 26, 2022
6881860
Add header, font, and GH link fixes
rogermparent Apr 26, 2022
13c4708
Upgrade theme package
rogermparent Apr 27, 2022
9dc8af1
Add sidebar stub
rogermparent Apr 27, 2022
fa11158
footer css fix
yathomasi Apr 28, 2022
7588deb
svg fix
yathomasi Apr 28, 2022
9d849e8
tutorial
mike0sv Apr 29, 2022
3ba3813
better sidebar
mike0sv Apr 29, 2022
37bb027
Remove docs content
rogermparent Apr 29, 2022
3c12118
Re-add docs content
rogermparent Apr 29, 2022
b917932
Add spaces inbetween details tags and content
rogermparent Apr 29, 2022
de04871
improving tutorial
mike0sv Apr 29, 2022
d4649e0
some user guidance
mike0sv Apr 30, 2022
5ad0166
more docs
mike0sv Apr 30, 2022
98b1d1f
MLemObject API list
mike0sv Apr 30, 2022
e131bdc
sticky navbar
yathomasi May 2, 2022
4a2ad81
fix some links
mike0sv May 2, 2022
3fe8683
mlem objects and abcs
mike0sv May 3, 2022
d93db74
Merge branch 'docs-engine' of github.com:iterative/mlem.ai into docs-…
rogermparent May 4, 2022
40f197d
Override base.css
rogermparent May 5, 2022
d208f42
Merge branch 'main' of github.com:iterative/mlem.ai into docs-engine
rogermparent May 5, 2022
122b46f
Merge branch 'docs-engine' of github.com:iterative/mlem.ai into docs-…
rogermparent May 5, 2022
cb66733
add analytics
aguschin May 5, 2022
c7a6d99
Use dvc.org link for discord
rogermparent May 5, 2022
3080ecd
Fix padding even more
rogermparent May 5, 2022
08d2c5e
fixing typos
aguschin May 6, 2022
ee1f253
more Use Cases
aguschin May 8, 2022
70119f2
format markdown files
aguschin May 8, 2022
9869cdc
commit wrongly formatted markdown
aguschin May 9, 2022
21979f7
refactor tutorial
mike0sv May 9, 2022
1443e3d
refactor tutorial
mike0sv May 9, 2022
f8daae4
Override RightPanel with new discord link
rogermparent May 9, 2022
9bc1e2c
MLEM def - extract MR stuff. Rename 'start' -> 'get-started' for nami…
omesser May 12, 2022
f69abc2
Fix broken link
omesser May 12, 2022
7d398b7
Some get-started/index rephrases
omesser May 12, 2022
8fdee21
CR fix - reword in header/index.tsx
omesser May 12, 2022
b1308c1
Override docs layout with a version that doesn't have search
rogermparent May 12, 2022
fd51131
Upgrade packages
rogermparent May 12, 2022
831c310
Merge branch 'docs-engine' into docs-content
rogermparent May 12, 2022
d08e59e
Rework home-slides, add what-is-mlem, make index intro more concise
omesser May 13, 2022
46fc659
install mlem in get-started
omesser May 13, 2022
9299573
typo fix
omesser May 13, 2022
962a470
Merge branch 'main' into docs-engine
rogermparent May 13, 2022
a7fdd10
Run prettier
rogermparent May 13, 2022
110cbaf
Fix ts lint issues in project code
rogermparent May 13, 2022
3ed3a86
no datasets for GS
mike0sv May 17, 2022
1811edc
mark WIP
mike0sv May 17, 2022
02866ec
install: copy edits
jorgeorpinel May 17, 2022
4c4023b
add index.md for CLI and API reference
madhur-tandon May 17, 2022
bb1b45b
start: shorter index intro
jorgeorpinel May 17, 2022
43e7506
add cli docs for init
madhur-tandon May 17, 2022
819600b
add cli docs for mlem list
madhur-tandon May 17, 2022
55859a6
add cli docs for pprint
madhur-tandon May 17, 2022
ba26f71
add cli docs for serve
madhur-tandon May 17, 2022
23367b7
suggested formatting changes
madhur-tandon May 17, 2022
f0b509b
add cli docs for types
madhur-tandon May 18, 2022
551246a
subsume arguments inside synopsis
madhur-tandon May 18, 2022
5ae6d8e
fix arguments for cli types
madhur-tandon May 18, 2022
450cde7
add cli docs for link
madhur-tandon May 18, 2022
e33b054
add cli docs for clone
madhur-tandon May 18, 2022
1346c77
cli docs for create, update serve and types
madhur-tandon May 18, 2022
7ac71e3
cli import docs
madhur-tandon May 18, 2022
83b434b
cli apply docs
madhur-tandon May 18, 2022
bfa9d9e
fix formatting for cli import
madhur-tandon May 18, 2022
a94b446
fix typos and add link for deploy
madhur-tandon May 18, 2022
f010c3f
Make further changes required to make linting pass
rogermparent May 18, 2022
d13ed3f
Fix top margin on footer
rogermparent May 18, 2022
5fe10aa
Merge branch 'main' of github.com:iterative/mlem.ai into docs-engine
rogermparent May 18, 2022
2eae931
Merge branch 'docs-engine' of github.com:iterative/mlem.ai into docs-…
rogermparent May 18, 2022
d78bd59
cli docs for deploy and its subcommands
madhur-tandon May 19, 2022
18bd383
use correct labels for subcommands of deploy in sidebar
madhur-tandon May 19, 2022
810ab3c
cli docs for pack
madhur-tandon May 19, 2022
ad3d579
add cli for apply-remote
madhur-tandon May 19, 2022
7258aad
api docs for init
madhur-tandon May 19, 2022
db7e0ab
api docs for save and load
madhur-tandon May 19, 2022
95a23ef
api docs for load_meta
madhur-tandon May 19, 2022
774d646
Upgrade packages
rogermparent May 19, 2022
444935a
Upgrade packages
rogermparent May 19, 2022
17bd0c8
Add xs-scr media query
rogermparent May 19, 2022
f1331d2
Merge branch 'docs-engine' into docs-content
rogermparent May 19, 2022
1ffb064
Merge branch 'main' of github.com:iterative/mlem.ai into docs-engine
rogermparent May 19, 2022
5b71896
add api docs for ls
madhur-tandon May 19, 2022
09b4335
add model registry use case from dvc.org
aguschin May 20, 2022
ed6164d
fixing folders/path names; fixing api-reference headers
aguschin May 20, 2022
3cca714
run linter
aguschin May 20, 2022
e7fe56c
Add docs page stub
rogermparent May 20, 2022
ea3a92a
Merge branch 'docs-engine' of github.com:iterative/mlem.ai into docs-…
rogermparent May 20, 2022
3125e2c
Fix broken cli-reference links. Remove self referential link from mle…
omesser May 20, 2022
b45684c
running yarn format
omesser May 20, 2022
c5de607
Merge branch 'main' of github.com:iterative/mlem.ai into docs-content
rogermparent May 20, 2022
001a46b
Update src/components/NavBar/OtherToolsPopup/index.tsx
aguschin May 21, 2022
4ec72a2
Command reference pass. Also using mlem for shell blocks where mlem c…
omesser May 22, 2022
91fa7ba
Format & lint
omesser May 22, 2022
f39dde1
copy contributing from dvc.org
aguschin May 22, 2022
2a14a27
yarn run fixes
aguschin May 22, 2022
13bcd12
incorporate feedback; edit mlem+dvc page in UC
aguschin May 22, 2022
dcab49b
add api docs for import_object
madhur-tandon May 23, 2022
620f9e9
add api docs for link
madhur-tandon May 23, 2022
f5abbb2
add api docs for clone
madhur-tandon May 23, 2022
7bd5ed5
api docs for apply
madhur-tandon May 23, 2022
6e123a0
api docs for pack
madhur-tandon May 23, 2022
815e7fb
api docs for serve
madhur-tandon May 23, 2022
1394792
add api docs for apply remote
madhur-tandon May 23, 2022
c6474ad
add description for apply-remote
madhur-tandon May 23, 2022
e6644e7
Replace old separate highlighters with new unified cli highlighter
rogermparent May 23, 2022
4ab62c6
Run prettier
rogermparent May 23, 2022
e3b2ed8
api docs for deploy - TODO for usage and example
madhur-tandon May 23, 2022
5e6dc73
remove empty index files
jorgeorpinel May 24, 2022
4e9c956
remove useless file
jorgeorpinel May 24, 2022
5da57d1
cleanup sidebar.json
jorgeorpinel May 24, 2022
1b70da0
remove some empty content not in nav
jorgeorpinel May 24, 2022
c8c602b
use HTML comments for TODOs
jorgeorpinel May 24, 2022
24742d8
Fix a bunch of typos
jorgeorpinel May 24, 2022
c8776ff
python -> py (md fenced blocks)
jorgeorpinel May 24, 2022
8f4d29c
md fenced block lang. review (`cli` everywhere)
jorgeorpinel May 24, 2022
8b4229e
run Prettier
jorgeorpinel May 24, 2022
0d212a3
ref: remove most notes (md block quotes)
jorgeorpinel May 24, 2022
1aee3c7
updating MLEM+DVC doc
aguschin May 24, 2022
997ea66
Merge branch 'docs-content' of github.com:iterative/mlem.ai into docs…
aguschin May 24, 2022
91a15c3
remove WIP items from sidebar
aguschin May 24, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions content/docs/api-reference/apply.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# apply

Apply provided model against provided data

Args:
model (ModelMeta): MLEM model.
data (Any): Input to the model.
method (str, optional): Which model method to use.
If None, use the only method model has.
If more than one is available, will fail.
output (str, optional): If value is provided,
assume it's path and save output there.
link (bool): Whether to create a link to saved output in MLEM root folder.
external (bool): Whether to save result outside mlem dir

Returns:
If `output=None`, returns results for given data.
Otherwise returns None.
21 changes: 21 additions & 0 deletions content/docs/api-reference/clone.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# clone

Clones MLEM object from `path` to `out` and returns Python representation for
the created object

Args:
path (str): Path to the object. Could be local path or path inside a git repo.
target (str): Path to save the copy of initial object to.
repo (Optional[str], optional): URL to repo if object is located there.
rev (Optional[str], optional): revision, could be git commit SHA, branch name or tag.
fs (Optional[AbstractFileSystem], optional): filesystem to load object from
target_repo (Optional[str], optional): path to repo to save cloned object to
target_fs (Optional[AbstractFileSystem], optional): target filesystem
follow_links (bool, optional): If object we read is a MLEM link, whether to load
the actual object link points to. Defaults to True.
load_value (bool, optional): Load actual python object incorporated in MlemMeta object. Defaults to False.
link: whether to create link in target repo
external: wheter to put object inside mlem dir in target repo

Returns:
MlemMeta: Copy of initial object saved to `out`
1 change: 1 addition & 0 deletions content/docs/api-reference/deploy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# deploy
4 changes: 4 additions & 0 deletions content/docs/api-reference/import_object.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# import_object

Try to load an object as MLEM model (or dataset) and return it, optionally
saving to the specified target location
15 changes: 15 additions & 0 deletions content/docs/api-reference/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Python API

MLEM can be used as a python library, simply [install](/doc/install) with `pip`
or `conda`. This reference provides the details about the functions in the API
module `mlem.api`, which can be imported in any regular way, for example:

```py
import mlem.api
```

The purpose of this API is to provide programmatic access to operate on models
and datasets from Python code.

Please choose a function from the navigation sidebar to the left, or click the
`Next` button below to jump into the first one ↘
39 changes: 39 additions & 0 deletions content/docs/api-reference/init.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# mlem.api.init()

Creates `.mlem` directory in `path`

```py
def init(path: str = ".") -> None
```

### Usage:

```py
from mlem.api import init

init(path)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add multiple examples with explanations like

init()  # init in current directory
init("s3://bucket")  # init in cloud
etc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, this looks like examples actually. Let me check dvc docs to see what is the difference

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jorgeorpinel What is the purpose of usage section? We can see signature above and some examples in the end. For DVC python API usage has the same content as one of the examples in most cases

```

## Description

Initializes a MLEM repository by creating a `.mlem` directory inside the given
path. A new and empty `config.yaml` is also created inside it.

## Parameters

- **`path`** (required) - location of the target where an MLEM repository has to
be initialized i.e. a `.mlem` folder has to be created.

## Exceptions

None

## Examples

```py
from mlem.api import init

init()
```

> By default, the path is `.` i.e. the current directory
18 changes: 18 additions & 0 deletions content/docs/api-reference/link.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# link

Creates MlemLink for an `source` object and dumps it if `target` is provided

Args:
source (Union[str, MlemMeta]): The object to create link from.
source_repo (str, optional): Path to mlem repo where to load obj from
rev (str, optional): Revision if object is stored in git repo.
target (str, optional): Where to store the link object.
target_repo (str, optional): If provided,
treat `target` as link name and dump link in MLEM DIR
follow_links (bool): Whether to make link to the underlying object
if `source` is itself a link. Defaults to True.
external (bool): Whether to save link outside mlem dir
absolute (bool): Whether to make link absolute or relative to mlem repo

Returns:
MlemLink: Link object to the `source`.
58 changes: 58 additions & 0 deletions content/docs/api-reference/load.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# mlem.api.load()

Load python object saved by MLEM

```py
def load(
path: str,
repo: Optional[str] = None,
rev: Optional[str] = None,
batch_size: Optional[int] = None,
follow_links: bool = True,
) -> Any
```

### Usage:

```py
import os
from mlem.api import load

out_path = os.path.join(os.getcwd(), "saved-model")
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not needed?

loaded = load(out_path)
```

## Description

Loads a python object from a given path. The path can belong to different file
systems (eg: `S3`). The function returns the underlying python object saved by
MLEM.

## Parameters

- **`path`** (required) - Path to the object. Could be local path or path inside
a git repo.
- `repo` (optional) - URL to repo if object is located there.
- `rev` (optional) - revision, could be git commit SHA, branch name or tag.
- `follow_links` (optional) - If object we read is a MLEM link, whether to load
the actual object link points to. Defaults to True.

## Exceptions

None
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably here we can actually not find the object :) or project

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same for load_meta


## Example: Load a trained model saved with MLEM

```py
import os
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
from mlem.api import load

path = os.path.join(os.getcwd(), "saved-model")

model = load(path)
assert isinstance(model, DecisionTreeClassifier)
train, _ = load_iris(return_X_y=True)
model.predict(train)
```
73 changes: 73 additions & 0 deletions content/docs/api-reference/load_meta.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# mlem.api.load_meta()

Loads MlemObject from a given path

```py
def load_meta(
path: str,
repo: Optional[str] = None,
rev: Optional[str] = None,
follow_links: bool = True,
load_value: bool = False,
fs: Optional[AbstractFileSystem] = None,
*,
force_type: Optional[Type[T]] = None,
) -> MlemObject
```

### Usage:

```py
import os
from mlem.api import load_meta

out_path = os.path.join(os.getcwd(), "saved-model")
loaded = load_meta(out_path)
```

## Description

Loads a [MlemObject](/doc/user-guide/basic-concepts#mlem-objects) from a given
path. This differs from [load](/doc/api-reference/load) since the latter loads
the actual python object incorporated within MlemObject. In fact, `load` uses
`load_meta` beneath and uses its `get_value()` method to get the underlying
python object.

## Parameters

- **`path`** (required) - Path to the object. Could be local path or path inside
a git repo.
- `repo` (optional) - URL to repo if object is located there.
- `rev` (optional) - revision, could be git commit SHA, branch name or tag.
- `follow_links` (optional) - If object we read is a MLEM link, whether to load
the actual object link points to. Defaults to True.
- `load_value` (optional) - Load actual python object incorporated in
MlemObject. Defaults to False.
- `fs` (optional) - filesystem to load from. If not provided, will be inferred
from path
- `force_type` (optional) - type of meta to be loaded. Defaults to MlemObject
(any mlem meta)

## Exceptions

- `WrongMetaType` - Thrown if the loaded meta object has a different type than
what is expected (force_type or MlemObject)

## Examples

```py
import os
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier

from mlem.core.objects import MlemModel
from mlem.api import load_meta

train, _ = load_iris(return_X_y=True)
out_path = os.path.join(os.getcwd(), "saved-model")
meta = load_meta(out_path, load_value=True, force_type=MlemModel)

model = meta.get_value()
assert isinstance(model, DecisionTreeClassifier)
model.predict(train)
```
53 changes: 53 additions & 0 deletions content/docs/api-reference/ls.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# mlem.api.ls()

Get a view of the MLEM repository by listing all of its MLEM Objects

```py
def ls(
repo: str = ".",
rev: Optional[str] = None,
fs: Optional[AbstractFileSystem] = None,
type_filter: Union[
Type[MlemObject], Iterable[Type[MlemObject]], None
] = None,
include_links: bool = True,
) -> Dict[Type[MlemObject], List[MlemObject]]
```

### Usage:

```py
from mlem.api import ls

objects = ls(".", rev=None, type_filter=None, include_links=True)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's strange to set default values. Better to have more examples where those values are not default (with explanation what it will do and where)

```

## Description

Populates a dictionary where keys are different `types` of
[MlemObjects](/doc/user-guide/basic-concepts#mlem-objects) and values are a
collection of MlemObjects of that type. This API is internally used by the CLI
command [list](/doc/command-reference/list).

## Parameters

- **`repo`** (required) - Path or URL to repo
- `rev` (optional) - revision, could be git commit SHA, branch name or tag.
- `fs` (optional) - filesystem to load from. If not provided, will be inferred
from repo
- `type_filter` (optional) - type of objects to be listed (eg: models / dataset
/ etc.)
- `include_links` (optional) - whether to include links while fetching the list
of MlemObjects. Defaults to True

## Exceptions

None

## Examples

```py
from mlem.api import ls

objects = ls(".")
```
24 changes: 24 additions & 0 deletions content/docs/api-reference/mlem-object.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# MlemObject API

[comment]: <> (list notable functions of `MlemMeta` subclasses )

- MlemMeta.read
- MLemMeta.load_value
- MlemMeta.get_value
- MlemMeta.dump
- MlemMeta.make_link
- MlemMeta.clone
- MlemMeta.update

- MlemLink.load_link
- MlemLink.parse_link
- MlemLink.from_location

- ModelMeta.from_obj
- DatasetMeta.from_data

- TargetEnvMeta.deploy
- TargetEnvMeta.destroy
- TargetEnvMeta.get_status

Same for DeployMeta
9 changes: 9 additions & 0 deletions content/docs/api-reference/pack.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# pack

Package a [MLEM model](/doc/user-guide/mlem-abcs#modeltype) in
docker-build-ready folder or directly build a docker image.

Args:
packager (Union[str, Packager]): Packager to use.
Out-of-the-box supported string values are "docker_dir" and "docker".
model (Union[str, ModelMeta]): The model to pack.
Loading