diff --git a/content/docs/api-reference/pack.md b/content/docs/api-reference/pack.md index 4bfa4806..54b82164 100644 --- a/content/docs/api-reference/pack.md +++ b/content/docs/api-reference/pack.md @@ -20,22 +20,6 @@ from mlem.api import pack pack("pip", "rf", target="build", package_name="example_mlem_get_started") ``` -> The extra kwargs supplied above can be seen from the output of -> `mlem types packager pip` which gives us -> -> ```py -> [required] package_name: str -> [required] target: str -> [not required] templates_dir: str = [] -> [not required] python_version: str = None -> [not required] short_description: str = "" -> [not required] url: str = "" -> [not required] email: str = "" -> [not required] author: str = "" -> [not required] version: str = "0.0.0" -> [not required] additional_setup_kwargs: typing.Any = {} -> ``` - ## Description This API is the underlying mechanism for the @@ -43,6 +27,26 @@ This API is the underlying mechanism for the programmatically create ship-able assets from MlemModels such as pip-ready packages, docker images, etc. + + +The arguments supplied to this method can be found with `mlem types`: + +```cli +$ mlem types packager pip +[required] package_name: str +[required] target: str +[not required] templates_dir: str = [] +[not required] python_version: str = None +[not required] short_description: str = "" +[not required] url: str = "" +[not required] email: str = "" +[not required] author: str = "" +[not required] version: str = "0.0.0" +[not required] additional_setup_kwargs: typing.Any = {} +``` + + + ## Parameters - **`packager`** (required) - Packager to use. Out-of-the-box supported string diff --git a/content/docs/command-reference/link.md b/content/docs/command-reference/link.md index 12111d50..c6a95981 100644 --- a/content/docs/command-reference/link.md +++ b/content/docs/command-reference/link.md @@ -44,8 +44,12 @@ Add a remote object to your local workspace (aka repo) without copying it $ mlem link rf --source-repo https://github.com/iterative/example-mlem-get-started remote_model ``` -> The remote model can now be served with the link created above, using the -> command `mlem serve remote_model fastapi`. + + +The remote model can now be served with the link created above, using the +command `mlem serve remote_model fastapi`. + + Alias a local object with a different name diff --git a/content/docs/command-reference/pprint.md b/content/docs/command-reference/pprint.md index 0cb70bd6..a6ff8d22 100644 --- a/content/docs/command-reference/pprint.md +++ b/content/docs/command-reference/pprint.md @@ -22,7 +22,11 @@ specific information such as `methods` for a `model` or `reader` for a Since only one specific object is printed, a `PATH` to the specific MLEM object is always required. -> You can use [`mlem list`](/doc/command-reference/list) to list MLEM objects. + + +You can use [`mlem list`](/doc/command-reference/list) to list MLEM objects. + + ## Options diff --git a/content/docs/contributing/core.md b/content/docs/contributing/core.md index 142192ea..0c8454c6 100644 --- a/content/docs/contributing/core.md +++ b/content/docs/contributing/core.md @@ -46,7 +46,11 @@ Make sure that you have Python 3.7 or higher installed. On macOS, we recommend using `brew` to install Python. For Windows, we recommend an official [python.org release](https://www.python.org/downloads/windows/). -> ℹ️ Note that `pip` version 20.3+ is required. + + +`pip` version 20.3+ is required. + + Install MLEM in editable mode with `pip install -e ".[tests]"`. But before we do that, we **strongly** recommend creating a diff --git a/content/docs/get-started/deploying.md b/content/docs/get-started/deploying.md index ce707539..23e4d5c3 100644 --- a/content/docs/get-started/deploying.md +++ b/content/docs/get-started/deploying.md @@ -4,12 +4,11 @@ You can create deployments in the cloud from your models. This uses packaging and serving functions under the hood. For example, Heroku deployment combines Docker image packaging with FastAPI serving. -> ⚠️ This functionality is experimental and is subject to change. We’ll add more -> target platforms in upcoming releases. + -Deployment often uses packaging and serving functionalities. For example, Heroku -deployment that is showcased in this section actually uses docker image -packaging with FastAPI serving. +This functionality is experimental and is subject to change. + + ## Defining target environment @@ -36,8 +35,12 @@ $ mlem create env heroku staging -c api_key= 💾 Saving env to .mlem/env/staging.mlem ``` -> Note that api_key argument is optional and MLEM will use `HEROKU_API_KEY` env -> variable if you don’t provide it via config. + + +MLEM will attempt to use the `HEROKU_API_KEY` environment variable if no +`api_key` argument is provided. + + ## Defining deployment @@ -92,10 +95,18 @@ $ mlem deploy create myservice ✅ Service example-mlem-get-started is up. You can check it out at https://my-mlem-service.herokuapp.com/ ``` -> 💡 You can also create and configure deployment on-the-fly using `-c` options -> for `mlem deploy create` command: -> -> `$ mlem deploy create service_name -m model -t staging -c app_name=example-mlem-get-started` + + +You can also define and run the deployment on-the-fly using `-c` options for +`mlem deploy create`, e.g.: + +```cli +$ mlem deploy create myservice \ + -m model -t staging \ + -c app_name=example-mlem-get-started +``` + +
@@ -145,9 +156,17 @@ $ mlem deploy apply myservice test_x.csv --json [1, 0, 2, 1, 1, 0, 1, 2, 1, 1, 2, 0, 0, 0, 0, 1, 2, 1, 1, 2, 0, 2, 0, 2, 2, 2, 2, 2, 0, 0, 0, 0, 1, 0, 0, 2, 1, 0] ``` -> 💡 As always, you don’t need to have deployment meta locally: -> -> `$ mlem deploy apply https://github.com/iterative/example-mlem-get-started/myservice https://github.com/iterative/example-mlem-get-started/test_x.csv --json` + + +You don’t even need to have the deployment metadata locally: + +```cli +$ mlem deploy apply --json \ + https://github.com/iterative/example-mlem-get-started/myservice \ + https://github.com/iterative/example-mlem-get-started/test_x.csv +``` + + ## Managing deployment diff --git a/content/docs/get-started/saving.md b/content/docs/get-started/saving.md index 62a4e159..376cd046 100644 --- a/content/docs/get-started/saving.md +++ b/content/docs/get-started/saving.md @@ -45,15 +45,21 @@ Now let's run this script and see how we save the model. ```cli $ python train.py +... + $ tree .mlem/model/ .mlem/model ├── rf └── rf.mlem ``` -> By default, MLEM saves your files to `.mlem/` directory, but that could be -> changed, see [project structure](/doc/user-guide/project-structure) for -> reference. + + +By default, MLEM saves your files to `.mlem/` directory, but that can be +changed. See [Project Structure](/doc/user-guide/project-structure) for more +details. + + The model was saved along with some metadata about it: `rf` containing the model binary and a `.mlem` metafile containing information about it. Let's take a look diff --git a/content/docs/index.md b/content/docs/index.md index 19216928..26b11f29 100644 --- a/content/docs/index.md +++ b/content/docs/index.md @@ -4,11 +4,15 @@ It seamlessly supports a variety of scenarios like real-time serving and batch processing. -> 💡 When combined with [GTO](https://github.com/iterative/gto), MLEM allows you -> to create a powerful Model Registry out of your Git repository! Such a -> registry serves as a centralized place to store and operationalize your models -> along with their metadata; manage model life-cycle, versions & releases, and -> easily automate tests and deployments using GitOps. + + +When combined with [GTO](https://github.com/iterative/gto), MLEM allows you to +create a powerful Model Registry out of your Git repository! Such a registry +serves as a centralized place to store and operationalize your models along with +their metadata; manage model life-cycle, versions & releases, and easily +automate tests and deployments using GitOps. + + diff --git a/content/docs/sidebar.json b/content/docs/sidebar.json index f2c7a376..531ed222 100644 --- a/content/docs/sidebar.json +++ b/content/docs/sidebar.json @@ -1,276 +1,284 @@ [ { "slug": "", + "source": "index.md", "label": "MLEM Documentation", "icon": "house" }, { "slug": "install", + "source": "install.md", "label": "Installation" }, { "slug": "get-started", "label": "Get Started", + "source": "get-started/index.md", "children": [ { "slug": "saving", "label": "Saving models", - "source": "get-started/saving.md" + "source": "saving.md" }, { "slug": "applying", "label": "Applying models", - "source": "get-started/applying.md" + "source": "applying.md" }, { "slug": "packaging", "label": "Packaging models", - "source": "get-started/packaging.md" + "source": "packaging.md" }, { "slug": "serving", "label": "Serving models", - "source": "get-started/serving.md" + "source": "serving.md" }, { "slug": "deploying", "label": "Deploying models", - "source": "get-started/deploying.md" + "source": "deploying.md" } ] }, { "slug": "use-cases", "label": "Use Cases", + "source": "use-cases/index.md", "children": [ { "slug": "dvc", "label": "Versioning MLEM objects with DVC", - "source": "use-cases/dvc.md" + "source": "dvc.md" }, { "slug": "mlem-mr", "label": "Pure Mlem Model Registry", - "source": "use-cases/mlem-mr.md" + "source": "mlem-mr.md" }, { "slug": "cicd", "label": "Using in CI/CD", - "source": "use-cases/cicd.md" + "source": "cicd.md" }, { "slug": "model-registry", "label": "Model Registry", - "source": "use-cases/model-registry.md" + "source": "model-registry.md" } ] }, { "slug": "user-guide", "label": "User Guide", + "source": "user-guide/index.md", "children": [ { "slug": "basic-concepts", "label": "Basic concepts", - "source": "user-guide/basic-concepts.md" + "source": "basic-concepts.md" }, { "slug": "project-structure", "label": "Project structure", - "source": "user-guide/project-structure.md" + "source": "project-structure.md" }, { "slug": "configuration", "label": "Configuration", - "source": "user-guide/configuration.md" + "source": "configuration.md" }, { "slug": "importing", "label": "Importing existing files", - "source": "user-guide/importing.md" + "source": "importing.md" }, { "slug": "linking", "label": "Links", - "source": "user-guide/linking.md" + "source": "linking.md" }, { "slug": "mlem-abcs", "label": "MLEM ABCs", - "source": "user-guide/mlem-abcs.md" + "source": "mlem-abcs.md" }, { "slug": "extending", "label": "Extending", - "source": "user-guide/extending.md" + "source": "extending.md" }, { "slug": "analytics", "label": "Anonymized Usage Analytics", - "source": "user-guide/analytics.md" + "source": "analytics.md" } ] }, { "slug": "command-reference", "label": "Command Reference", + "source": "command-reference/index.md", "children": [ { "slug": "init", "label": "init", - "source": "command-reference/init.md" + "source": "init.md" }, { "slug": "list", "label": "list", - "source": "command-reference/list.md" + "source": "list.md" }, { "slug": "pprint", "label": "pprint", - "source": "command-reference/pprint.md" + "source": "pprint.md" }, { "slug": "create", "label": "create", - "source": "command-reference/create.md" + "source": "create.md" }, { "slug": "serve", "label": "serve", - "source": "command-reference/serve.md" + "source": "serve.md" }, { "slug": "deploy", "label": "deploy", + "source": "deploy/index.md", "children": [ { "slug": "apply", "label": "deploy apply", - "source": "command-reference/deploy/apply.md" + "source": "apply.md" }, { "slug": "create", "label": "deploy create", - "source": "command-reference/deploy/create.md" + "source": "create.md" }, { "slug": "status", "label": "deploy status", - "source": "command-reference/deploy/status.md" + "source": "status.md" }, { "slug": "teardown", "label": "deploy teardown", - "source": "command-reference/deploy/teardown.md" + "source": "teardown.md" } ] }, { "slug": "types", "label": "types", - "source": "command-reference/types.md" + "source": "types.md" }, { "slug": "link", "label": "link", - "source": "command-reference/link.md" + "source": "link.md" }, { "slug": "clone", "label": "clone", - "source": "command-reference/clone.md" + "source": "clone.md" }, { "slug": "import", "label": "import", - "source": "command-reference/import.md" + "source": "import.md" }, { "slug": "pack", "label": "pack", - "source": "command-reference/pack.md" + "source": "pack.md" }, { "slug": "apply", "label": "apply", - "source": "command-reference/apply.md" + "source": "apply.md" }, { "slug": "apply-remote", "label": "apply-remote", - "source": "command-reference/apply-remote.md" + "source": "apply-remote.md" } ] }, { "slug": "api-reference", "label": "Python API Reference", + "source": "api-reference/index.md", "children": [ { "slug": "init", "label": "init()", - "source": "api-reference/init.md" + "source": "init.md" }, { "slug": "save", "label": "save()", - "source": "api-reference/save.md" + "source": "save.md" }, { "slug": "load", "label": "load()", - "source": "api-reference/load.md" + "source": "load.md" }, { "slug": "load_meta", "label": "load_meta()", - "source": "api-reference/load_meta.md" + "source": "load_meta.md" }, { "slug": "ls", "label": "ls()", - "source": "api-reference/ls.md" + "source": "ls.md" }, { "slug": "import_object", "label": "import_object()", - "source": "api-reference/import_object.md" + "source": "import_object.md" }, { "slug": "link", "label": "link()", - "source": "api-reference/link.md" + "source": "link.md" }, { "slug": "clone", "label": "clone()", - "source": "api-reference/clone.md" + "source": "clone.md" }, { "slug": "apply", "label": "apply()", - "source": "api-reference/apply.md" + "source": "apply.md" }, { "slug": "apply_remote", "label": "apply_remote()", - "source": "api-reference/apply_remote.md" + "source": "apply_remote.md" }, { "slug": "pack", "label": "pack()", - "source": "api-reference/pack.md" + "source": "pack.md" }, { "slug": "serve", "label": "serve()", - "source": "api-reference/serve.md" + "source": "serve.md" }, { "slug": "deploy", "label": "deploy()", - "source": "api-reference/deploy.md" + "source": "deploy.md" } ] }, @@ -282,12 +290,12 @@ { "slug": "core", "label": "MLEM Core Project", - "source": "contributing/core.md" + "source": "core.md" }, { "slug": "docs", "label": "Docs and Website", - "source": "contributing/docs.md" + "source": "docs.md" } ] } diff --git a/content/docs/user-guide/basic-concepts.md b/content/docs/user-guide/basic-concepts.md index c67907fa..1688d754 100644 --- a/content/docs/user-guide/basic-concepts.md +++ b/content/docs/user-guide/basic-concepts.md @@ -6,11 +6,12 @@ The most important concept in MLEM is **MLEM Object**. Basically, MLEM is a library to create, manage and use different **MLEM Objects**, such as models, datasets and other types you can read about below. -> So, when you use `save` API method, you create MLEM Object from an arbitrary -> supported Python object. + -> Also, MLEM Objects can be created with -> [`mlem create`](/doc/command-reference/create) CLI command +For example, when you use `mlem.api.save()`, you create a MLEM Object from a +supported Python structure. MLEM Objects can also be created with `mlem create`. + + MLEM Objects are saved as special _metafiles_ in YAML format with the `.mlem` extension. These may or may not have _artifacts_ (other files or directories) diff --git a/content/docs/user-guide/extending.md b/content/docs/user-guide/extending.md index 3f62fd03..d71c00df 100644 --- a/content/docs/user-guide/extending.md +++ b/content/docs/user-guide/extending.md @@ -66,8 +66,12 @@ dynamically, depending on what is imported in user's environment. For example, 2. After importing mlem, user imported `sklearn` 3. User loaded any object that uses any of `sklearn` extension implementation. -> Note that some of the fields in MlemObjects are lazy, which mean they will be -> loaded only if users accesses them. + + +Some of the fields in MLEM Objects are lazy, which means that they will be +loaded only if users accesses them. + + ## Subclassing MlemConfig diff --git a/content/docs/user-guide/importing.md b/content/docs/user-guide/importing.md index 9ca7b311..e6cbbaac 100644 --- a/content/docs/user-guide/importing.md +++ b/content/docs/user-guide/importing.md @@ -6,8 +6,12 @@ Objects, you can use [`mlem import`](/doc/command-reference/import) or They will try to load the path you provided and analyze the object saved there. -> Obviously, importing is more limited than `save` API, since MLEM do not have -> live python object to analyze and tries to recreate it, which may fail. + + +Importing is more limited than `mlem.api.save()`. This is because MLEM does not +have a live Python object to analyze and tries to recreate it, which may fail. + + You can see list of available import implementations [here](/doc/user-guide/mlem-abcs#importhook). diff --git a/content/docs/user-guide/linking.md b/content/docs/user-guide/linking.md index a25e6a8b..b9d7fa4e 100644 --- a/content/docs/user-guide/linking.md +++ b/content/docs/user-guide/linking.md @@ -5,10 +5,14 @@ represent MLEM Objects in other locations. You can [reference](/doc/user-guide/project-structure#referencing-mlem-objects) links anywhere you need to specify MLEM Object bot in API and CLI. -> Since MLEM Links are also a type of MLEM Objects, they share the same logic, -> for example they are saved under `.mlem/link` directory. To load and instance -> of `MlemLink` (and not the object it references) provide `follow_links=False` -> to `load_meta` method. + + +Since links are also a type of MLEM Object, they share the same internal logic. +For example, they are saved under the `.mlem/link` directory. To load and +instance of `MlemLink` (and not the object it references) provide +`follow_links=False` to `load_meta` method. + + ## Link structure @@ -27,8 +31,12 @@ Links can be created via [`mlem link`](/doc/command-reference/link) or [`mlem.api.link`](/doc/api-reference/link) commands, as well as the `MlemMeta.make_link()` method. -> You can create relative links inside the same repository, which will basically -> create an alias for that object. + + +You can create relative links inside the same repository, which will basically +create an alias for that object. + + Also, since links can target specific commits, tags or branches in a versioned repository, they can be used in a variety of different scenarios, for example to diff --git a/content/docs/user-guide/mlem-abcs.md b/content/docs/user-guide/mlem-abcs.md index 62cb4a76..df5d0eb8 100644 --- a/content/docs/user-guide/mlem-abcs.md +++ b/content/docs/user-guide/mlem-abcs.md @@ -17,9 +17,13 @@ via `-c` notation in CLI.
-> Fields marked as **transient** are used to hold some operational object and -> are not saved when you dump the objects. After loading objects with such -> fields they will be empty until you somehow "load" the object. + + +**Transient** fields are used to hold some operational object and are not saved +when an object is dumped. After opening objects with transient fields they will +be empty until you load the object. + +