Skip to content
This repository has been archived by the owner on Oct 16, 2024. It is now read-only.

Content for /docs #43

Merged
merged 129 commits into from
May 24, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
129 commits
Select commit Hold shift + click to select a range
6585bc2
documentation stubs
mike0sv Apr 14, 2022
c879e62
docs structure
mike0sv Apr 21, 2022
45d27ec
Add gatsby-theme-iterative
rogermparent Apr 22, 2022
f7f9e02
Remove preact in favor of React 18
rogermparent Apr 23, 2022
be5a6a4
Apply list-style: none where necessary
rogermparent Apr 25, 2022
e8f6d1f
Fix header by adding a container and bg color
rogermparent Apr 25, 2022
e007489
Move Header and Footer into theme global with containers
rogermparent Apr 26, 2022
14bdc98
Merge branch 'main' of github.com:iterative/mlem.ai into docs-engine
rogermparent Apr 26, 2022
436c3a0
Use old site's custom media config
rogermparent Apr 26, 2022
b6d0c20
Add docs link
rogermparent Apr 26, 2022
6881860
Add header, font, and GH link fixes
rogermparent Apr 26, 2022
13c4708
Upgrade theme package
rogermparent Apr 27, 2022
9dc8af1
Add sidebar stub
rogermparent Apr 27, 2022
fa11158
footer css fix
yathomasi Apr 28, 2022
7588deb
svg fix
yathomasi Apr 28, 2022
9d849e8
tutorial
mike0sv Apr 29, 2022
3ba3813
better sidebar
mike0sv Apr 29, 2022
37bb027
Remove docs content
rogermparent Apr 29, 2022
3c12118
Re-add docs content
rogermparent Apr 29, 2022
b917932
Add spaces inbetween details tags and content
rogermparent Apr 29, 2022
de04871
improving tutorial
mike0sv Apr 29, 2022
d4649e0
some user guidance
mike0sv Apr 30, 2022
5ad0166
more docs
mike0sv Apr 30, 2022
98b1d1f
MLemObject API list
mike0sv Apr 30, 2022
e131bdc
sticky navbar
yathomasi May 2, 2022
4a2ad81
fix some links
mike0sv May 2, 2022
3fe8683
mlem objects and abcs
mike0sv May 3, 2022
d93db74
Merge branch 'docs-engine' of github.com:iterative/mlem.ai into docs-…
rogermparent May 4, 2022
40f197d
Override base.css
rogermparent May 5, 2022
d208f42
Merge branch 'main' of github.com:iterative/mlem.ai into docs-engine
rogermparent May 5, 2022
122b46f
Merge branch 'docs-engine' of github.com:iterative/mlem.ai into docs-…
rogermparent May 5, 2022
cb66733
add analytics
aguschin May 5, 2022
c7a6d99
Use dvc.org link for discord
rogermparent May 5, 2022
3080ecd
Fix padding even more
rogermparent May 5, 2022
08d2c5e
fixing typos
aguschin May 6, 2022
ee1f253
more Use Cases
aguschin May 8, 2022
70119f2
format markdown files
aguschin May 8, 2022
9869cdc
commit wrongly formatted markdown
aguschin May 9, 2022
21979f7
refactor tutorial
mike0sv May 9, 2022
1443e3d
refactor tutorial
mike0sv May 9, 2022
f8daae4
Override RightPanel with new discord link
rogermparent May 9, 2022
9bc1e2c
MLEM def - extract MR stuff. Rename 'start' -> 'get-started' for nami…
omesser May 12, 2022
f69abc2
Fix broken link
omesser May 12, 2022
7d398b7
Some get-started/index rephrases
omesser May 12, 2022
8fdee21
CR fix - reword in header/index.tsx
omesser May 12, 2022
b1308c1
Override docs layout with a version that doesn't have search
rogermparent May 12, 2022
fd51131
Upgrade packages
rogermparent May 12, 2022
831c310
Merge branch 'docs-engine' into docs-content
rogermparent May 12, 2022
d08e59e
Rework home-slides, add what-is-mlem, make index intro more concise
omesser May 13, 2022
46fc659
install mlem in get-started
omesser May 13, 2022
9299573
typo fix
omesser May 13, 2022
962a470
Merge branch 'main' into docs-engine
rogermparent May 13, 2022
a7fdd10
Run prettier
rogermparent May 13, 2022
110cbaf
Fix ts lint issues in project code
rogermparent May 13, 2022
3ed3a86
no datasets for GS
mike0sv May 17, 2022
1811edc
mark WIP
mike0sv May 17, 2022
02866ec
install: copy edits
jorgeorpinel May 17, 2022
4c4023b
add index.md for CLI and API reference
madhur-tandon May 17, 2022
bb1b45b
start: shorter index intro
jorgeorpinel May 17, 2022
43e7506
add cli docs for init
madhur-tandon May 17, 2022
819600b
add cli docs for mlem list
madhur-tandon May 17, 2022
55859a6
add cli docs for pprint
madhur-tandon May 17, 2022
ba26f71
add cli docs for serve
madhur-tandon May 17, 2022
23367b7
suggested formatting changes
madhur-tandon May 17, 2022
f0b509b
add cli docs for types
madhur-tandon May 18, 2022
551246a
subsume arguments inside synopsis
madhur-tandon May 18, 2022
5ae6d8e
fix arguments for cli types
madhur-tandon May 18, 2022
450cde7
add cli docs for link
madhur-tandon May 18, 2022
e33b054
add cli docs for clone
madhur-tandon May 18, 2022
1346c77
cli docs for create, update serve and types
madhur-tandon May 18, 2022
7ac71e3
cli import docs
madhur-tandon May 18, 2022
83b434b
cli apply docs
madhur-tandon May 18, 2022
bfa9d9e
fix formatting for cli import
madhur-tandon May 18, 2022
a94b446
fix typos and add link for deploy
madhur-tandon May 18, 2022
f010c3f
Make further changes required to make linting pass
rogermparent May 18, 2022
d13ed3f
Fix top margin on footer
rogermparent May 18, 2022
5fe10aa
Merge branch 'main' of github.com:iterative/mlem.ai into docs-engine
rogermparent May 18, 2022
2eae931
Merge branch 'docs-engine' of github.com:iterative/mlem.ai into docs-…
rogermparent May 18, 2022
d78bd59
cli docs for deploy and its subcommands
madhur-tandon May 19, 2022
18bd383
use correct labels for subcommands of deploy in sidebar
madhur-tandon May 19, 2022
810ab3c
cli docs for pack
madhur-tandon May 19, 2022
ad3d579
add cli for apply-remote
madhur-tandon May 19, 2022
7258aad
api docs for init
madhur-tandon May 19, 2022
db7e0ab
api docs for save and load
madhur-tandon May 19, 2022
95a23ef
api docs for load_meta
madhur-tandon May 19, 2022
774d646
Upgrade packages
rogermparent May 19, 2022
444935a
Upgrade packages
rogermparent May 19, 2022
17bd0c8
Add xs-scr media query
rogermparent May 19, 2022
f1331d2
Merge branch 'docs-engine' into docs-content
rogermparent May 19, 2022
1ffb064
Merge branch 'main' of github.com:iterative/mlem.ai into docs-engine
rogermparent May 19, 2022
5b71896
add api docs for ls
madhur-tandon May 19, 2022
09b4335
add model registry use case from dvc.org
aguschin May 20, 2022
ed6164d
fixing folders/path names; fixing api-reference headers
aguschin May 20, 2022
3cca714
run linter
aguschin May 20, 2022
e7fe56c
Add docs page stub
rogermparent May 20, 2022
ea3a92a
Merge branch 'docs-engine' of github.com:iterative/mlem.ai into docs-…
rogermparent May 20, 2022
3125e2c
Fix broken cli-reference links. Remove self referential link from mle…
omesser May 20, 2022
b45684c
running yarn format
omesser May 20, 2022
c5de607
Merge branch 'main' of github.com:iterative/mlem.ai into docs-content
rogermparent May 20, 2022
001a46b
Update src/components/NavBar/OtherToolsPopup/index.tsx
aguschin May 21, 2022
4ec72a2
Command reference pass. Also using mlem for shell blocks where mlem c…
omesser May 22, 2022
91fa7ba
Format & lint
omesser May 22, 2022
f39dde1
copy contributing from dvc.org
aguschin May 22, 2022
2a14a27
yarn run fixes
aguschin May 22, 2022
13bcd12
incorporate feedback; edit mlem+dvc page in UC
aguschin May 22, 2022
dcab49b
add api docs for import_object
madhur-tandon May 23, 2022
620f9e9
add api docs for link
madhur-tandon May 23, 2022
f5abbb2
add api docs for clone
madhur-tandon May 23, 2022
7bd5ed5
api docs for apply
madhur-tandon May 23, 2022
6e123a0
api docs for pack
madhur-tandon May 23, 2022
815e7fb
api docs for serve
madhur-tandon May 23, 2022
1394792
add api docs for apply remote
madhur-tandon May 23, 2022
c6474ad
add description for apply-remote
madhur-tandon May 23, 2022
e6644e7
Replace old separate highlighters with new unified cli highlighter
rogermparent May 23, 2022
4ab62c6
Run prettier
rogermparent May 23, 2022
e3b2ed8
api docs for deploy - TODO for usage and example
madhur-tandon May 23, 2022
5e6dc73
remove empty index files
jorgeorpinel May 24, 2022
4e9c956
remove useless file
jorgeorpinel May 24, 2022
5da57d1
cleanup sidebar.json
jorgeorpinel May 24, 2022
1b70da0
remove some empty content not in nav
jorgeorpinel May 24, 2022
c8c602b
use HTML comments for TODOs
jorgeorpinel May 24, 2022
24742d8
Fix a bunch of typos
jorgeorpinel May 24, 2022
c8776ff
python -> py (md fenced blocks)
jorgeorpinel May 24, 2022
8f4d29c
md fenced block lang. review (`cli` everywhere)
jorgeorpinel May 24, 2022
8b4229e
run Prettier
jorgeorpinel May 24, 2022
0d212a3
ref: remove most notes (md block quotes)
jorgeorpinel May 24, 2022
1aee3c7
updating MLEM+DVC doc
aguschin May 24, 2022
997ea66
Merge branch 'docs-content' of github.com:iterative/mlem.ai into docs…
aguschin May 24, 2022
91a15c3
remove WIP items from sidebar
aguschin May 24, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions content/docs/api-reference/apply.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# apply
Apply provided model against provided data

Args:
model (ModelMeta): MLEM model.
data (Any): Input to the model.
method (str, optional): Which model method to use.
If None, use the only method model has.
If more than one is available, will fail.
output (str, optional): If value is provided,
assume it's path and save output there.
link (bool): Whether to create a link to saved output in MLEM root folder.
external (bool): Whether to save result outside mlem dir

Returns:
If `output=None`, returns results for given data.
Otherwise returns None.


21 changes: 21 additions & 0 deletions content/docs/api-reference/clone.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# clone
Clones MLEM object from `path` to `out`
and returns Python representation for the created object

Args:
path (str): Path to the object. Could be local path or path inside a git repo.
target (str): Path to save the copy of initial object to.
repo (Optional[str], optional): URL to repo if object is located there.
rev (Optional[str], optional): revision, could be git commit SHA, branch name or tag.
fs (Optional[AbstractFileSystem], optional): filesystem to load object from
target_repo (Optional[str], optional): path to repo to save cloned object to
target_fs (Optional[AbstractFileSystem], optional): target filesystem
follow_links (bool, optional): If object we read is a MLEM link, whether to load
the actual object link points to. Defaults to True.
load_value (bool, optional): Load actual python object incorporated in MlemMeta object. Defaults to False.
link: whether to create link in target repo
external: wheter to put object inside mlem dir in target repo

Returns:
MlemMeta: Copy of initial object saved to `out`

1 change: 1 addition & 0 deletions content/docs/api-reference/deploy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# deploy
4 changes: 4 additions & 0 deletions content/docs/api-reference/import_object.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# import_object
Try to load an object as MLEM model (or dataset) and return it,
optionally saving to the specified target location

1 change: 1 addition & 0 deletions content/docs/api-reference/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
do we want each command in separate file?
2 changes: 2 additions & 0 deletions content/docs/api-reference/init.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# init
Creates .mlem directory in `path`
18 changes: 18 additions & 0 deletions content/docs/api-reference/link.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# link
Creates MlemLink for an `source` object and dumps it if `target` is provided

Args:
source (Union[str, MlemMeta]): The object to create link from.
source_repo (str, optional): Path to mlem repo where to load obj from
rev (str, optional): Revision if object is stored in git repo.
target (str, optional): Where to store the link object.
target_repo (str, optional): If provided,
treat `target` as link name and dump link in MLEM DIR
follow_links (bool): Whether to make link to the underlying object
if `source` is itself a link. Defaults to True.
external (bool): Whether to save link outside mlem dir
absolute (bool): Whether to make link absolute or relative to mlem repo

Returns:
MlemLink: Link object to the `source`.

13 changes: 13 additions & 0 deletions content/docs/api-reference/load.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# load
Load python object saved by MLEM

Args:
path (str): Path to the object. Could be local path or path inside a git repo.
repo (Optional[str], optional): URL to repo if object is located there.
rev (Optional[str], optional): revision, could be git commit SHA, branch name or tag.
follow_links (bool, optional): If object we read is a MLEM link, whether to load the
actual object link points to. Defaults to True.

Returns:
Any: Python object saved by MLEM

15 changes: 15 additions & 0 deletions content/docs/api-reference/load_meta.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# load_meta
Load MlemMeta object

Args:
path (str): Path to the object. Could be local path or path inside a git repo.
repo (Optional[str], optional): URL to repo if object is located there.
rev (Optional[str], optional): revision, could be git commit SHA, branch name or tag.
follow_links (bool, optional): If object we read is a MLEM link, whether to load the
actual object link points to. Defaults to True.
load_value (bool, optional): Load actual python object incorporated in MlemMeta object. Defaults to False.
fs: filesystem to load from. If not provided, will be inferred from path
force_type: type of meta to be loaded. Defaults to MlemMeta (any mlem meta)
Returns:
MlemMeta: Saved MlemMeta object

1 change: 1 addition & 0 deletions content/docs/api-reference/ls.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# ls
8 changes: 8 additions & 0 deletions content/docs/api-reference/pack.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# pack
Pack model in docker-build-ready folder or directly build a docker image.

Args:
packager (Union[str, Packager]): Packager to use.
Out-of-the-box supported string values are "docker_dir" and "docker".
model (Union[str, ModelMeta]): The model to pack.

23 changes: 23 additions & 0 deletions content/docs/api-reference/save.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# save
Saves given object to a given path

Args:
obj: Object to dump
path: If not located on LocalFileSystem, then should be uri
or `fs` argument should be provided
repo: path to mlem repo (optional)
dvc: Store the object's artifacts with dvc
tmp_sample_data: If the object is a model or function, you can
provide input data sample, so MLEM will include it's schema
in the model's metadata
fs: FileSystem for the `path` argument
link: Whether to create a link in .mlem folder found for `path`
external: if obj is saved to repo, whether to put it outside of .mlem dir
description: description for object
params: arbitrary params for object
tags: tags for object
update: whether to keep old description/tags/params if new values were not provided

Returns:
None

7 changes: 7 additions & 0 deletions content/docs/api-reference/serve.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# serve
Serve model via HTTP/HTTPS.

Args:
model (ModelMeta): The model to serve.
server (Union[Server, str]): Out-of-the-box supported one is "fastapi".

39 changes: 39 additions & 0 deletions content/docs/cli-reference/apply.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# apply
Usage: [options] model data

Apply a model to a dataset. Resulting dataset will be saved as MLEM object
to `output` if it is provided, otherwise will be printed

Arguments:

MODEL Path to model object [required]
DATA Path to dataset object [required]

Options:

-r, --repo TEXT Path to MLEM repo [default: (none)]
--rev TEXT Repo revision to use [default: (none)]
-o, --output TEXT Where to store the outputs.
-m, --method TEXT Which model method is to apply [default: predict]
--data-repo, --dr TEXT Repo with dataset
--data-rev TEXT Revision of dataset
-i, --import Try to import data on-the-fly
--import-type, --it TEXT Specify how to read data file for import
--link / --no-link Whether to create link for output in .mlem
directory
-e, --external Save result not in .mlem, but directly in repo
--json Output as json
--help Show this message and exit.

Examples:

Apply local mlem model to local mlem dataset
$ mlem apply mymodel mydatset --method predict --output myprediction

Apply local mlem model to local data file
$ mlem apply mymodel data.csv --method predict --import --import-type pandas[csv] --output myprediction

Apply a version of remote model to a version of remote dataset
$ mlem apply models/logreg --repo https://github.com/iterative/example-mlem --rev main
data/test_x --data-repo https://github.com/iterative/example-mlem --data-rev main
--method predict --output myprediction
27 changes: 27 additions & 0 deletions content/docs/cli-reference/clone.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# clone
Usage: [options] uri target

Download MLEM object from `uri` and save it to `target`

Arguments:

URI URI to object you want to clone [required]
TARGET Path to store the downloaded object. [required]

Options:

-r, --repo TEXT Path to MLEM repo [default: (none)]
--rev TEXT Repo revision to use [default: (none)]
--target-repo, --tr TEXT Repo to save target to [default: (none)]
-e, --external Save result not in .mlem, but directly in repo
--link / --no-link Whether to create link for output in .mlem
directory
--help Show this message and exit.

Examples:

Copy remote model to local directory
$ mlem clone models/logreg --repo https://github.com/iterative/example-mlem --rev main mymodel

Copy remote model to remote MLEM repo
$ mlem clone models/logreg --repo https://github.com/iterative/example-mlem --rev main mymodel --tr s3://mybucket/mymodel
24 changes: 24 additions & 0 deletions content/docs/cli-reference/create.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# create
Usage: [options] object_type [subtype] path

Creates new mlem object metafile from conf args and config files

Arguments:

OBJECT_TYPE Type of metafile to create [required]
[SUBTYPE] Subtype of MLEM object [default: ]
PATH Where to save object [required]

Options:

-c, --conf TEXT Values for object fields in format
`field.nested.name=value`
-r, --repo TEXT Path to MLEM repo [default: (none)]
-e, --external Save result not in .mlem, but directly in repo
--link / --no-link Whether to create link for output in .mlem directory
--help Show this message and exit.

Examples:

Create heroku deployment
$ mlem create env heroku production -c api_key=<...>
34 changes: 34 additions & 0 deletions content/docs/cli-reference/import.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# import
Usage: [options] uri target

Create MLEM model or dataset metadata from file/dir
aguschin marked this conversation as resolved.
Show resolved Hide resolved

Arguments:

URI File to import [required]
TARGET Path whare to save MLEM object [required]

Options:

-r, --repo TEXT Path to MLEM repo [default: (none)]
--rev TEXT Repo revision to use [default: (none)]
--target-repo, --tr TEXT Repo to save target to [default: (none)]
--copy / --no-copy Whether to create a copy of file in target
location or just link existing file [default:
copy]
--type TEXT Specify how to read file [default: (auto infer)]
--link / --no-link Whether to create link for output in .mlem
directory
-e, --external Save result not in .mlem, but directly in repo
--help Show this message and exit.

Examples:

Create MLEM dataset from local csv
$ mlem import data/data.csv data/imported_data --type pandas[csv]

Create MLEM model from local pickle file
$ mlem import data/model.pkl data/imported_model

Create MLEM model from remote pickle file
$ mlem import models/logreg --repo https://github.com/iterative/example-mlem --rev no-dvc data/imported_model --type pickle
Empty file.
18 changes: 18 additions & 0 deletions content/docs/cli-reference/init.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# init
Usage: [options] [path]

Initialize MLEM repo

Arguments:

[PATH] Where to init repo

Options:

--help Show this message and exit.

Examples:

$ mlem init
$ mlem init some/local/path
$ mlem init s3://bucket/path/in/cloud
33 changes: 33 additions & 0 deletions content/docs/cli-reference/link.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# link
Usage: [options] source target

Create link for MLEM object

Arguments:

SOURCE URI to object you are crating link to [required]
TARGET Path to save link object [required]

Options:

--source-repo, --sr TEXT Repo for source object
--rev TEXT Repo revision to use [default: (none)]
--target-repo, --tr TEXT Repo to save target to [default: (none)]
-e, --external Save result not in .mlem, but directly in
repo
--follow-links, --f / --no-follow-links, --nf
If True, first follow links while reading
{source} before creating this link.
[default: follow-links]
--absolute, --abs / --relative, --rel
Which path to linked object to specify:
absolute or relative. [default: relative]
--help Show this message and exit.

Examples:

Add alias to local object
$ mlem link my_model latest

Add remote object to your repo without copy
$ mlem link models/logreg --source-repo https://github.com/iteartive/example-mlem remote_model
22 changes: 22 additions & 0 deletions content/docs/cli-reference/list.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# list
Usage: [options] [repo]

List MLEM objects of in repo

Arguments:

[REPO] Repo to list from [default: (current directory)]

Options:

-t, --type [all|link|model|dataset|env|deployment|packager]
Type of objects to list [default: all]
--rev TEXT Repo revision to use [default: (none)]
+l, --links / -l, --no-links Include links [default: l]
--json Output as json
--help Show this message and exit.

Examples:

$ mlem list https://github.com/iterative/example-mlem
$ mlem list -t models
28 changes: 28 additions & 0 deletions content/docs/cli-reference/pack.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# pack
Usage: [options] model [subtype]

Pack model

Arguments:

MODEL Path to model [required]
[SUBTYPE] Type of packing. Choices: ['docker', 'docker_dir']

Options:

-r, --repo TEXT Path to MLEM repo [default: (none)]
--rev TEXT Repo revision to use [default: (none)]
-l, --load TEXT File to load packing config from
-c, --conf TEXT Options for packing in format `field.name=value`
-f, --file_conf TEXT File with options for packing in format
`field.name=path_to_config`
--help Show this message and exit.

Examples:

Build docker image from model
$ mlem pack mymodel docker -c server.type=fastapi -c image.name=myimage

Create pack docker_dir declaration and build it
$ mlem create packager docker_dir -c server=fastapi -c target=build pack_dock
$ mlem pack mymodel --load pack_dock
Loading