Semi-autogenerated cli reference #172

mike0sv · 2022-09-03T08:58:06Z

First step for #171

This PR contains auto-generation machinery for CLI reference.

_generate_cli_spec.py generates spec.json file from mlem.cli.app object
_generate_options.py reads spec.json and for each command generates sections and inserts them into respective .md files

Auto-generated sections:

first paragraph (between cmd name and synopsys): mlem cli help message
usage
- usage text
- arguments with help
- subcommands (for groups)
options: list of cli options with help

There are also 2 not generated sections: Description and Examples.

Example: suppose we have a key in spec.json

"apply": {
    "args": {
      "args": [
        {
          "decl": "model",
          "help": "Path to model object  [required]"
        }
      ],
      "impls": null,
      "impl_metavar": null,
      "subcommands": null
    },
    "options": [
      {
        "decl": "-p, --project TEXT",
        "help": "Path to MLEM project  [default: (none)]"
      }
    ],
    "usage": "Usage: mlem apply [options] model data",
    "doc": "Apply a model to data. The result will be saved as a MLEM object to `output` if\nprovided. Otherwise, it will be printed to `stdout`."
  }

And apply.md

# apply

AAAA

## Synopsys

`` `usage
BBB
`` `

## Description

CCC

## Options

DDD

## Examples

EEE

It will be transformed in place into:

# apply

Apply a model to data. The result will be saved as a MLEM object to `output` if
provided. Otherwise, it will be printed to `stdout`.

## Synopsis

`` `usage
Usage: mlem apply [options] model data

Arguments:
- `model`: Path to model object  [required]
`` `

## Description

CCC

## Options

- `-p, --project TEXT`: Path to MLEM project  [default: (none)]

## Examples

EEE

Feel free to suggest changes to generated content formatting and to spec schema

github-actions · 2022-09-03T09:03:29Z

095acea

Link Check Report

All 3 links passed!

casperdcl · 2022-09-05T20:30:33Z

less hacky than https://github.com/tqdm/tqdm/blob/master/.meta/mkdocs.py :)

content/docs/command-reference/spec.json

content/docs/command-reference/build.md

shcheklein · 2022-09-06T04:28:19Z

content/docs/command-reference/build.md

-arguments:
-MODEL      Path to model  [required]
-[SUBTYPE]  Type of build. Choices: ['whl', 'pip', 'docker_dir', 'docker']
+Builtin builders:


MODEL -> model

required / optional check how it's done in other docs

list with options - it was better the way it was done (compact, easier to read, etc)

required and optional are different things: required means you always should provide it, optional means it can have None value. Required optional means you will get error if you don't provide it, but you can set it to None. Not required not optional means there is a default value which is not None and you can't set it to None

also cli changed a bit, now mlem build * are subcommands (instead of argument). Also mlem build is a separate command (for building from saved config using -l). I will update this soon when I decide how to better reflect this (same with serve and apply-remote)

shcheklein · 2022-09-06T04:32:44Z

content/docs/command-reference/checkenv.md

+
+## Options
+
+- `-p, --project TEXT`: Path to MLEM project [default: (none)]


style: -p <path>, --project <path> - path to MLEM ...

default - check how it's done in the docs

TEXT should be renamed into path, also in the CLI itself

done, but sometimes the duplication of metavar feels strange

content/docs/command-reference/types.md

jorgeorpinel

A few misc. Qs below.

I like the intent but I'm a bit skeptical about actually applying this as a process. Curious to see how it goes though, happy to help if I can. Lmk!

content/docs/command-reference/spec.json

content/docs/command-reference/apply.md

content/docs/command-reference/spec.json

content/docs/command-reference/checkenv.md

jorgeorpinel

I think that the automation effort is always great and this particular PR is def. an improvement and quite mergeable (after removing script and auxiliary files). Some minor comments below, but I'm not blocking it.

I'm not sure that we can continue to use this specific script though, maybe start only with overwriting usage blocks.

Cc @omesser FYI (rel. #171 (comment))

jorgeorpinel · 2022-09-14T16:59:14Z

content/docs/command-reference/checkenv.md

+
+## Examples


No need for these lines when there are no examples.

yeah, I added this to not forget to add them :)

content/docs/command-reference/config.md

content/docs/command-reference/init.md

jorgeorpinel · 2022-09-14T17:04:45Z

content/docs/command-reference/list.md

-List [MLEM objects](/doc/user-guide/basic-concepts#mlem-objects) inside a MLEM
-project (location should be [initialized](/doc/command-reference/init)).
-
-> Aliased to `mlem ls`
+List MLEM objects inside a [MLEM project](/doc/user-guide/project-structure).


We're losing a link and the alias note here.

jorgeorpinel · 2022-09-14T17:05:32Z

content/docs/command-reference/serve.md

 # serve

-Locally deploy the model using a server implementation and expose its methods as
-endpoints.
+Create an API from model methods using a server implementation.


Previous version may be better? (would need core change)

I changed this because you commented on core repo PR :) So should I return old doc?

Sorry, I don't remember the context. Can you link to the discussion? Up to you though, thanks.

jorgeorpinel · 2022-09-21T02:21:39Z

Should mlem dev be included @mike0sv? I'm curious why that command was not generated. Rel. #180.

Also, are there clear next steps to move this forward or are we stuck somewhere? Thanks

mike0sv · 2022-10-04T14:18:58Z

I'd say lets merge this! I created a separate release/0.3.0 branch just like in core repo so we can finalize bits of work on new release docs without going live with them

@mike0sv

This is a list of activities that should be done before release announce ## First, things before release ### Breaking changes - merge in this PR - [x] New deployment state management - [x] New CLI interface: breaking changes in `serve`, `apply-remote`, `build` and `declare` ### Other changes - merge in this PR - [x] SageMaker deployments - [x] K8s deployments - [x] `--quiet` #448 @mike0sv ### Docs - [x] Finish semi-generated API reference PR iterative/mlem.ai#172 - [x] Update Quick Start in this repo - [x] iterative/mlem.ai#197 - [ ] iterative/mlem.ai#188 ### Blog post - [ ] blog post re release @aguschin ### Example-repo - PR in `example-repos-dev` - [x] update `example-mlem-get-started` ### Update the website code animations - [x] https://mlem.ai ## Second, the release itself ### MLEM Release - [ ] merge this branch - [ ] release it on PyPi - [ ] merge docs PR - [ ] merge PR in `example-repos-dev` and update `iterative/example-mlem-get-started` ## Third, things after release - [ ] remove `xfail` mark from `tests/cli/test_mail.py::test_commands_docs_links`

semi-autogenerated cli reference

4840479

shcheklein temporarily deployed to mlem-ai-new-cli-qsq7njeufcmrzh September 3, 2022 08:58 Inactive

mike0sv requested review from jorgeorpinel, aguschin and omesser September 3, 2022 08:58

fix double spaces and impls order

2fa1e07

shcheklein temporarily deployed to mlem-ai-new-cli-qsq7njeufcmrzh September 5, 2022 06:03 Inactive

fix double spaces (for real)

7b83973

shcheklein temporarily deployed to mlem-ai-new-cli-qsq7njeufcmrzh September 5, 2022 06:07 Inactive

add abc subcommand args

d48ed0d

shcheklein temporarily deployed to mlem-ai-new-cli-qsq7njeufcmrzh September 5, 2022 06:20 Inactive

mike0sv mentioned this pull request Sep 5, 2022

Semi-autogenerated docs #171

Open

shcheklein reviewed Sep 6, 2022

View reviewed changes

content/docs/command-reference/spec.json Outdated Show resolved Hide resolved

shcheklein reviewed Sep 6, 2022

View reviewed changes

content/docs/command-reference/build.md Outdated Show resolved Hide resolved

shcheklein reviewed Sep 6, 2022

View reviewed changes

content/docs/command-reference/build.md Outdated Show resolved Hide resolved

shcheklein reviewed Sep 6, 2022

View reviewed changes

content/docs/command-reference/types.md Show resolved Hide resolved

jorgeorpinel reviewed Sep 7, 2022

View reviewed changes

address comments

96f2f7b

shcheklein temporarily deployed to mlem-ai-new-cli-qsq7njeufcmrzh September 9, 2022 03:05 Inactive

mike0sv added 2 commits September 9, 2022 06:21

prettier

a7085ab

prettier

de0b20a

shcheklein temporarily deployed to mlem-ai-new-cli-qsq7njeufcmrzh September 9, 2022 03:27 Inactive

mike0sv added 4 commits September 13, 2022 13:24

delete spec, update usage and options formatting

65f1588

lint?

e0c1b9d

lint?

7182940

lint?

5b221ba

lint

2ed3efa

shcheklein temporarily deployed to mlem-ai-new-cli-qsq7njeufcmrzh September 13, 2022 10:32 Inactive

feedback

e196fc4

shcheklein temporarily deployed to mlem-ai-new-cli-qsq7njeufcmrzh September 13, 2022 10:45 Inactive

revert yarn.lock

fabe888

shcheklein temporarily deployed to mlem-ai-new-cli-qsq7njeufcmrzh September 13, 2022 10:48 Inactive

jorgeorpinel reviewed Sep 14, 2022

View reviewed changes

move scripts, re-generate from latest docs

e669838

shcheklein temporarily deployed to mlem-ai-new-cli-qsq7njeufcmrzh September 19, 2022 12:45 Inactive

MLEM model and MLEM dataset

f254cb6

shcheklein temporarily deployed to mlem-ai-new-cli-qsq7njeufcmrzh September 19, 2022 12:50 Inactive

mike0sv changed the base branch from main to release/0.3.0 September 19, 2022 12:51

💅🏼

095acea

shcheklein temporarily deployed to mlem-ai-new-cli-qsq7njeufcmrzh September 19, 2022 12:58 Inactive

jorgeorpinel mentioned this pull request Sep 21, 2022

Docs for extensions #179

Closed

3 tasks

jorgeorpinel added A: docs Area: user documentation (gatsby-theme-iterative) C: ref Content of /doc/{command,api,object}-reference labels Sep 21, 2022

jorgeorpinel mentioned this pull request Sep 22, 2022

Document dvcfs API iterative/dvc.org#3927

Closed

aguschin mentioned this pull request Sep 23, 2022

Release 0.3.0 iterative/mlem#397

Merged

18 tasks

aguschin assigned mike0sv Oct 3, 2022

aguschin merged commit 0525959 into release/0.3.0 Oct 5, 2022

aguschin deleted the new-cli branch October 5, 2022 11:10

jorgeorpinel added the dependencies Pull requests that update a dependency file label Oct 30, 2022

jorgeorpinel added the ✨ epic Placeholder ticket for multi-sprint direction, use story, improvement label Dec 2, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Semi-autogenerated cli reference #172

Semi-autogenerated cli reference #172

mike0sv commented Sep 3, 2022

github-actions bot commented Sep 3, 2022 •

edited

Loading

casperdcl commented Sep 5, 2022

shcheklein Sep 6, 2022

mike0sv Sep 7, 2022

shcheklein Sep 6, 2022

mike0sv Sep 9, 2022

jorgeorpinel left a comment

jorgeorpinel left a comment •

edited

Loading

jorgeorpinel Sep 14, 2022

mike0sv Sep 15, 2022

jorgeorpinel Sep 14, 2022

jorgeorpinel Sep 14, 2022

mike0sv Sep 15, 2022

jorgeorpinel Sep 21, 2022

jorgeorpinel commented Sep 21, 2022

mike0sv commented Oct 4, 2022


		## Options

		- `-p, --project TEXT`: Path to MLEM project [default: (none)]

Semi-autogenerated cli reference #172

Semi-autogenerated cli reference #172

Conversation

mike0sv commented Sep 3, 2022

github-actions bot commented Sep 3, 2022 • edited Loading

Link Check Report

casperdcl commented Sep 5, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorgeorpinel left a comment

Choose a reason for hiding this comment

jorgeorpinel left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jorgeorpinel commented Sep 21, 2022

mike0sv commented Oct 4, 2022

github-actions bot commented Sep 3, 2022 •

edited

Loading

jorgeorpinel left a comment •

edited

Loading