This repository has been archived by the owner on Oct 16, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 12
Content for /docs #43
Merged
Merged
Changes from 104 commits
Commits
Show all changes
129 commits
Select commit
Hold shift + click to select a range
6585bc2
documentation stubs
mike0sv c879e62
docs structure
mike0sv 45d27ec
Add gatsby-theme-iterative
rogermparent f7f9e02
Remove preact in favor of React 18
rogermparent be5a6a4
Apply list-style: none where necessary
rogermparent e8f6d1f
Fix header by adding a container and bg color
rogermparent e007489
Move Header and Footer into theme global with containers
rogermparent 14bdc98
Merge branch 'main' of github.com:iterative/mlem.ai into docs-engine
rogermparent 436c3a0
Use old site's custom media config
rogermparent b6d0c20
Add docs link
rogermparent 6881860
Add header, font, and GH link fixes
rogermparent 13c4708
Upgrade theme package
rogermparent 9dc8af1
Add sidebar stub
rogermparent fa11158
footer css fix
yathomasi 7588deb
svg fix
yathomasi 9d849e8
tutorial
mike0sv 3ba3813
better sidebar
mike0sv 37bb027
Remove docs content
rogermparent 3c12118
Re-add docs content
rogermparent b917932
Add spaces inbetween details tags and content
rogermparent de04871
improving tutorial
mike0sv d4649e0
some user guidance
mike0sv 5ad0166
more docs
mike0sv 98b1d1f
MLemObject API list
mike0sv e131bdc
sticky navbar
yathomasi 4a2ad81
fix some links
mike0sv 3fe8683
mlem objects and abcs
mike0sv d93db74
Merge branch 'docs-engine' of github.com:iterative/mlem.ai into docs-…
rogermparent 40f197d
Override base.css
rogermparent d208f42
Merge branch 'main' of github.com:iterative/mlem.ai into docs-engine
rogermparent 122b46f
Merge branch 'docs-engine' of github.com:iterative/mlem.ai into docs-…
rogermparent cb66733
add analytics
aguschin c7a6d99
Use dvc.org link for discord
rogermparent 3080ecd
Fix padding even more
rogermparent 08d2c5e
fixing typos
aguschin ee1f253
more Use Cases
aguschin 70119f2
format markdown files
aguschin 9869cdc
commit wrongly formatted markdown
aguschin 21979f7
refactor tutorial
mike0sv 1443e3d
refactor tutorial
mike0sv f8daae4
Override RightPanel with new discord link
rogermparent 9bc1e2c
MLEM def - extract MR stuff. Rename 'start' -> 'get-started' for nami…
omesser f69abc2
Fix broken link
omesser 7d398b7
Some get-started/index rephrases
omesser 8fdee21
CR fix - reword in header/index.tsx
omesser b1308c1
Override docs layout with a version that doesn't have search
rogermparent fd51131
Upgrade packages
rogermparent 831c310
Merge branch 'docs-engine' into docs-content
rogermparent d08e59e
Rework home-slides, add what-is-mlem, make index intro more concise
omesser 46fc659
install mlem in get-started
omesser 9299573
typo fix
omesser 962a470
Merge branch 'main' into docs-engine
rogermparent a7fdd10
Run prettier
rogermparent 110cbaf
Fix ts lint issues in project code
rogermparent 3ed3a86
no datasets for GS
mike0sv 1811edc
mark WIP
mike0sv 02866ec
install: copy edits
jorgeorpinel 4c4023b
add index.md for CLI and API reference
madhur-tandon bb1b45b
start: shorter index intro
jorgeorpinel 43e7506
add cli docs for init
madhur-tandon 819600b
add cli docs for mlem list
madhur-tandon 55859a6
add cli docs for pprint
madhur-tandon ba26f71
add cli docs for serve
madhur-tandon 23367b7
suggested formatting changes
madhur-tandon f0b509b
add cli docs for types
madhur-tandon 551246a
subsume arguments inside synopsis
madhur-tandon 5ae6d8e
fix arguments for cli types
madhur-tandon 450cde7
add cli docs for link
madhur-tandon e33b054
add cli docs for clone
madhur-tandon 1346c77
cli docs for create, update serve and types
madhur-tandon 7ac71e3
cli import docs
madhur-tandon 83b434b
cli apply docs
madhur-tandon bfa9d9e
fix formatting for cli import
madhur-tandon a94b446
fix typos and add link for deploy
madhur-tandon f010c3f
Make further changes required to make linting pass
rogermparent d13ed3f
Fix top margin on footer
rogermparent 5fe10aa
Merge branch 'main' of github.com:iterative/mlem.ai into docs-engine
rogermparent 2eae931
Merge branch 'docs-engine' of github.com:iterative/mlem.ai into docs-…
rogermparent d78bd59
cli docs for deploy and its subcommands
madhur-tandon 18bd383
use correct labels for subcommands of deploy in sidebar
madhur-tandon 810ab3c
cli docs for pack
madhur-tandon ad3d579
add cli for apply-remote
madhur-tandon 7258aad
api docs for init
madhur-tandon db7e0ab
api docs for save and load
madhur-tandon 95a23ef
api docs for load_meta
madhur-tandon 774d646
Upgrade packages
rogermparent 444935a
Upgrade packages
rogermparent 17bd0c8
Add xs-scr media query
rogermparent f1331d2
Merge branch 'docs-engine' into docs-content
rogermparent 1ffb064
Merge branch 'main' of github.com:iterative/mlem.ai into docs-engine
rogermparent 5b71896
add api docs for ls
madhur-tandon 09b4335
add model registry use case from dvc.org
aguschin ed6164d
fixing folders/path names; fixing api-reference headers
aguschin 3cca714
run linter
aguschin e7fe56c
Add docs page stub
rogermparent ea3a92a
Merge branch 'docs-engine' of github.com:iterative/mlem.ai into docs-…
rogermparent 3125e2c
Fix broken cli-reference links. Remove self referential link from mle…
omesser b45684c
running yarn format
omesser c5de607
Merge branch 'main' of github.com:iterative/mlem.ai into docs-content
rogermparent 001a46b
Update src/components/NavBar/OtherToolsPopup/index.tsx
aguschin 4ec72a2
Command reference pass. Also using mlem for shell blocks where mlem c…
omesser 91fa7ba
Format & lint
omesser f39dde1
copy contributing from dvc.org
aguschin 2a14a27
yarn run fixes
aguschin 13bcd12
incorporate feedback; edit mlem+dvc page in UC
aguschin dcab49b
add api docs for import_object
madhur-tandon 620f9e9
add api docs for link
madhur-tandon f5abbb2
add api docs for clone
madhur-tandon 7bd5ed5
api docs for apply
madhur-tandon 6e123a0
api docs for pack
madhur-tandon 815e7fb
api docs for serve
madhur-tandon 1394792
add api docs for apply remote
madhur-tandon c6474ad
add description for apply-remote
madhur-tandon e6644e7
Replace old separate highlighters with new unified cli highlighter
rogermparent 4ab62c6
Run prettier
rogermparent e3b2ed8
api docs for deploy - TODO for usage and example
madhur-tandon 5e6dc73
remove empty index files
jorgeorpinel 4e9c956
remove useless file
jorgeorpinel 5da57d1
cleanup sidebar.json
jorgeorpinel 1b70da0
remove some empty content not in nav
jorgeorpinel c8c602b
use HTML comments for TODOs
jorgeorpinel 24742d8
Fix a bunch of typos
jorgeorpinel c8776ff
python -> py (md fenced blocks)
jorgeorpinel 8f4d29c
md fenced block lang. review (`cli` everywhere)
jorgeorpinel 8b4229e
run Prettier
jorgeorpinel 0d212a3
ref: remove most notes (md block quotes)
jorgeorpinel 1aee3c7
updating MLEM+DVC doc
aguschin 997ea66
Merge branch 'docs-content' of github.com:iterative/mlem.ai into docs…
aguschin 91a15c3
remove WIP items from sidebar
aguschin File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
# apply | ||
|
||
Apply provided model against provided data | ||
|
||
Args: | ||
model (ModelMeta): MLEM model. | ||
data (Any): Input to the model. | ||
method (str, optional): Which model method to use. | ||
If None, use the only method model has. | ||
If more than one is available, will fail. | ||
output (str, optional): If value is provided, | ||
assume it's path and save output there. | ||
link (bool): Whether to create a link to saved output in MLEM root folder. | ||
external (bool): Whether to save result outside mlem dir | ||
|
||
Returns: | ||
If `output=None`, returns results for given data. | ||
Otherwise returns None. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
# clone | ||
|
||
Clones MLEM object from `path` to `out` and returns Python representation for | ||
the created object | ||
|
||
Args: | ||
path (str): Path to the object. Could be local path or path inside a git repo. | ||
target (str): Path to save the copy of initial object to. | ||
repo (Optional[str], optional): URL to repo if object is located there. | ||
rev (Optional[str], optional): revision, could be git commit SHA, branch name or tag. | ||
fs (Optional[AbstractFileSystem], optional): filesystem to load object from | ||
target_repo (Optional[str], optional): path to repo to save cloned object to | ||
target_fs (Optional[AbstractFileSystem], optional): target filesystem | ||
follow_links (bool, optional): If object we read is a MLEM link, whether to load | ||
the actual object link points to. Defaults to True. | ||
load_value (bool, optional): Load actual python object incorporated in MlemMeta object. Defaults to False. | ||
link: whether to create link in target repo | ||
external: wheter to put object inside mlem dir in target repo | ||
|
||
Returns: | ||
MlemMeta: Copy of initial object saved to `out` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# deploy |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
# import_object | ||
|
||
Try to load an object as MLEM model (or dataset) and return it, optionally | ||
saving to the specified target location |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
# Python API | ||
|
||
MLEM can be used as a python library, simply [install](/doc/install) with `pip` | ||
or `conda`. This reference provides the details about the functions in the API | ||
module `mlem.api`, which can be imported in any regular way, for example: | ||
|
||
```py | ||
import mlem.api | ||
``` | ||
|
||
The purpose of this API is to provide programmatic access to operate on models | ||
and datasets from Python code. | ||
|
||
Please choose a function from the navigation sidebar to the left, or click the | ||
`Next` button below to jump into the first one ↘ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
# mlem.api.init() | ||
|
||
Creates `.mlem` directory in `path` | ||
|
||
```py | ||
def init(path: str = ".") -> None | ||
``` | ||
|
||
### Usage: | ||
|
||
```py | ||
from mlem.api import init | ||
|
||
init(path) | ||
``` | ||
|
||
## Description | ||
|
||
Initializes a MLEM repository by creating a `.mlem` directory inside the given | ||
path. A new and empty `config.yaml` is also created inside it. | ||
|
||
## Parameters | ||
|
||
- **`path`** (required) - location of the target where an MLEM repository has to | ||
be initialized i.e. a `.mlem` folder has to be created. | ||
|
||
## Exceptions | ||
|
||
None | ||
|
||
## Examples | ||
|
||
```py | ||
from mlem.api import init | ||
|
||
init() | ||
``` | ||
|
||
> By default, the path is `.` i.e. the current directory |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
# link | ||
|
||
Creates MlemLink for an `source` object and dumps it if `target` is provided | ||
|
||
Args: | ||
source (Union[str, MlemMeta]): The object to create link from. | ||
source_repo (str, optional): Path to mlem repo where to load obj from | ||
rev (str, optional): Revision if object is stored in git repo. | ||
target (str, optional): Where to store the link object. | ||
target_repo (str, optional): If provided, | ||
treat `target` as link name and dump link in MLEM DIR | ||
follow_links (bool): Whether to make link to the underlying object | ||
if `source` is itself a link. Defaults to True. | ||
external (bool): Whether to save link outside mlem dir | ||
absolute (bool): Whether to make link absolute or relative to mlem repo | ||
|
||
Returns: | ||
MlemLink: Link object to the `source`. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
# mlem.api.load() | ||
|
||
Load python object saved by MLEM | ||
|
||
```py | ||
def load( | ||
path: str, | ||
repo: Optional[str] = None, | ||
rev: Optional[str] = None, | ||
batch_size: Optional[int] = None, | ||
follow_links: bool = True, | ||
) -> Any | ||
``` | ||
|
||
### Usage: | ||
|
||
```py | ||
import os | ||
from mlem.api import load | ||
|
||
out_path = os.path.join(os.getcwd(), "saved-model") | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. not needed? |
||
loaded = load(out_path) | ||
``` | ||
|
||
## Description | ||
|
||
Loads a python object from a given path. The path can belong to different file | ||
systems (eg: `S3`). The function returns the underlying python object saved by | ||
MLEM. | ||
|
||
## Parameters | ||
|
||
- **`path`** (required) - Path to the object. Could be local path or path inside | ||
a git repo. | ||
- `repo` (optional) - URL to repo if object is located there. | ||
- `rev` (optional) - revision, could be git commit SHA, branch name or tag. | ||
- `follow_links` (optional) - If object we read is a MLEM link, whether to load | ||
the actual object link points to. Defaults to True. | ||
|
||
## Exceptions | ||
|
||
None | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Probably here we can actually not find the object :) or project There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. same for load_meta |
||
|
||
## Example: Load a trained model saved with MLEM | ||
|
||
```py | ||
import os | ||
from sklearn.datasets import load_iris | ||
from sklearn.tree import DecisionTreeClassifier | ||
from mlem.api import load | ||
|
||
path = os.path.join(os.getcwd(), "saved-model") | ||
|
||
model = load(path) | ||
assert isinstance(model, DecisionTreeClassifier) | ||
train, _ = load_iris(return_X_y=True) | ||
model.predict(train) | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,73 @@ | ||
# mlem.api.load_meta() | ||
|
||
Loads MlemObject from a given path | ||
|
||
```py | ||
def load_meta( | ||
path: str, | ||
repo: Optional[str] = None, | ||
rev: Optional[str] = None, | ||
follow_links: bool = True, | ||
load_value: bool = False, | ||
fs: Optional[AbstractFileSystem] = None, | ||
*, | ||
force_type: Optional[Type[T]] = None, | ||
) -> MlemObject | ||
``` | ||
|
||
### Usage: | ||
|
||
```py | ||
import os | ||
from mlem.api import load_meta | ||
|
||
out_path = os.path.join(os.getcwd(), "saved-model") | ||
loaded = load_meta(out_path) | ||
``` | ||
|
||
## Description | ||
|
||
Loads a [MlemObject](/doc/user-guide/basic-concepts#mlem-objects) from a given | ||
path. This differs from [load](/doc/api-reference/load) since the latter loads | ||
the actual python object incorporated within MlemObject. In fact, `load` uses | ||
`load_meta` beneath and uses its `get_value()` method to get the underlying | ||
python object. | ||
|
||
## Parameters | ||
|
||
- **`path`** (required) - Path to the object. Could be local path or path inside | ||
a git repo. | ||
- `repo` (optional) - URL to repo if object is located there. | ||
- `rev` (optional) - revision, could be git commit SHA, branch name or tag. | ||
- `follow_links` (optional) - If object we read is a MLEM link, whether to load | ||
the actual object link points to. Defaults to True. | ||
- `load_value` (optional) - Load actual python object incorporated in | ||
MlemObject. Defaults to False. | ||
- `fs` (optional) - filesystem to load from. If not provided, will be inferred | ||
from path | ||
- `force_type` (optional) - type of meta to be loaded. Defaults to MlemObject | ||
(any mlem meta) | ||
|
||
## Exceptions | ||
|
||
- `WrongMetaType` - Thrown if the loaded meta object has a different type than | ||
what is expected (force_type or MlemObject) | ||
|
||
## Examples | ||
|
||
```py | ||
import os | ||
from sklearn.datasets import load_iris | ||
from sklearn.tree import DecisionTreeClassifier | ||
|
||
from mlem.core.objects import MlemModel | ||
from mlem.api import load_meta | ||
|
||
train, _ = load_iris(return_X_y=True) | ||
out_path = os.path.join(os.getcwd(), "saved-model") | ||
meta = load_meta(out_path, load_value=True, force_type=MlemModel) | ||
|
||
model = meta.get_value() | ||
assert isinstance(model, DecisionTreeClassifier) | ||
model.predict(train) | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
# mlem.api.ls() | ||
|
||
Get a view of the MLEM repository by listing all of its MLEM Objects | ||
|
||
```py | ||
def ls( | ||
repo: str = ".", | ||
rev: Optional[str] = None, | ||
fs: Optional[AbstractFileSystem] = None, | ||
type_filter: Union[ | ||
Type[MlemObject], Iterable[Type[MlemObject]], None | ||
] = None, | ||
include_links: bool = True, | ||
) -> Dict[Type[MlemObject], List[MlemObject]] | ||
``` | ||
|
||
### Usage: | ||
|
||
```py | ||
from mlem.api import ls | ||
|
||
objects = ls(".", rev=None, type_filter=None, include_links=True) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. it's strange to set default values. Better to have more examples where those values are not default (with explanation what it will do and where) |
||
``` | ||
|
||
## Description | ||
|
||
Populates a dictionary where keys are different `types` of | ||
[MlemObjects](/doc/user-guide/basic-concepts#mlem-objects) and values are a | ||
collection of MlemObjects of that type. This API is internally used by the CLI | ||
command [list](/doc/command-reference/list). | ||
|
||
## Parameters | ||
|
||
- **`repo`** (required) - Path or URL to repo | ||
- `rev` (optional) - revision, could be git commit SHA, branch name or tag. | ||
- `fs` (optional) - filesystem to load from. If not provided, will be inferred | ||
from repo | ||
- `type_filter` (optional) - type of objects to be listed (eg: models / dataset | ||
/ etc.) | ||
- `include_links` (optional) - whether to include links while fetching the list | ||
of MlemObjects. Defaults to True | ||
|
||
## Exceptions | ||
|
||
None | ||
|
||
## Examples | ||
|
||
```py | ||
from mlem.api import ls | ||
|
||
objects = ls(".") | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
# MlemObject API | ||
|
||
[comment]: <> (list notable functions of `MlemMeta` subclasses ) | ||
|
||
- MlemMeta.read | ||
- MLemMeta.load_value | ||
- MlemMeta.get_value | ||
- MlemMeta.dump | ||
- MlemMeta.make_link | ||
- MlemMeta.clone | ||
- MlemMeta.update | ||
|
||
- MlemLink.load_link | ||
- MlemLink.parse_link | ||
- MlemLink.from_location | ||
|
||
- ModelMeta.from_obj | ||
- DatasetMeta.from_data | ||
|
||
- TargetEnvMeta.deploy | ||
- TargetEnvMeta.destroy | ||
- TargetEnvMeta.get_status | ||
|
||
Same for DeployMeta |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# pack | ||
|
||
Package a [MLEM model](/doc/user-guide/mlem-abcs#modeltype) in | ||
docker-build-ready folder or directly build a docker image. | ||
|
||
Args: | ||
packager (Union[str, Packager]): Packager to use. | ||
Out-of-the-box supported string values are "docker_dir" and "docker". | ||
model (Union[str, ModelMeta]): The model to pack. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add multiple examples with explanations like
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, this looks like examples actually. Let me check dvc docs to see what is the difference
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jorgeorpinel What is the purpose of usage section? We can see signature above and some examples in the end. For DVC python API usage has the same content as one of the examples in most cases