Add mlem get started #109

mike0sv · 2022-04-19T20:20:31Z

No description provided.

mike0sv · 2022-04-19T20:24:01Z

Converting this to draft since actual docs are not live on mlem.ai, but any feedback is still welcomed

example-mlem-get-started/generate.sh

shcheklein · 2022-04-19T20:40:02Z

example-mlem-get-started/generate.sh

+dvc remote add myremote azure://example-mlem --default
+mlem config set default_storage.type dvc
+python train.py
+dvc add .mlem/model/rf .mlem/dataset/train.csv .mlem/dataset/test_x.csv .mlem/dataset/test_y.csv


that's not very common for DVC to track each independently (esp dataset) - is it intended?

there are .mlem files in this dir. Can we do something like echo *.mlem > .dvcignore && dvc add data?

the correct pattern was found and it is /**/?*.mlem. Btw, does dvcignore support ! negation?

this does not seem to work, since we cant dvc add .mlem/dataset/ since we lose .mlem files, even though they ignored. So going with dvc add .mlem/dataset/*.csv. Would be great to have some way to do "add everything except dvc-ignored"

Btw, does dvcignore support ! negation?

yes, it does support it

it should be working, otherwise it's a bug to my mind

$ cat .dvcignore /**/*.mlem !/.mlem/ $ dvc check-ignore .mlem -d .dvcignore:2:!/.mlem/ .mlem

I was trying to make an exception for .mlem dir in a root, but this is what I've got

q - why do you need an exception like this? (curious)

does the same combination work for gitignore?

My goal is to exclude all *.mlem files, but not .mlem dir in repo root. But simple /**/*.mlem adds .mlem dir to ignore list. With git this exclusion works works

example-mlem-get-started/generate.sh

shcheklein · 2022-04-19T20:47:44Z

example-mlem-get-started/generate.sh

+
+
+cp $HERE/code/src/requirements.txt .
+cp $HERE/code/src/prepare.py .


If you don it similar to DVC get-started you need to bundle this code and share from S3 so that people could curl or wget it

Why not get it from raw.github.com?

raw.github.com works as well, I don't see any particular problems with that .. I would bundle though everything as a single tar (or may github has a way to download revisions as a tar with curl, I don't know)

https://github.com/iterative/example-mlem-get-started/archive/refs/tags/5-switch-to-dvc.tar.gz for example

it contains some extra files that are not needed initially though?

it contains whole repo, yes. But I don't see any point of maintaining them separately, since they are very small and easy and also will be available in the docs themselves

it's not about the size, it's about the workflow. In the dvc.org get started you would have a command:

wget code.zip unzip code.zip git add -a -m "add code"

https://dvc.org/doc/start/data-pipelines#expand-to-download-example-code

including any extra files there would confuse and ruin the flow of the document.

so, your call here - if you want something similar to DVC then consider creating a clean code package.

I will consider this, thanks

thinking second time - I remember now why I didn't use GH (most likely) - I wanted to keep the command as short and nice as possible wget https://code.dvc.org/get-started/code.zip

example-mlem-get-started/code/README.md

shcheklein

Thanks a lot for codifying this 🙏

I put some comments here and there to address, but also:

should we create a branch in the regular example-get-started that converts models into mlem and uses gto + deploys it. That would be cool.
in this example - should we have something about client, deployment, gto? Just saving model might be not very interesting (not clear why would I want this at all)

mike0sv · 2022-04-20T13:33:04Z

Yes, but this could wait until MLEM release I guess
I am planning to add something like that later, this was for first two sections of get-started https://github.com/iterative/mlem.ai/pull/43/files#diff-1cd29580767fad4637f461df8460373ba6ec523dc598d7d3550657ed56cab101

shcheklein · 2022-04-20T17:38:50Z

Thanks, @mike0sv for addressing some things. I left a bit more comments, but nothing major.

example-mlem-get-started/generate.sh

Co-authored-by: Alexander Guschin <[email protected]>

Per #109 (comment) - we don't need gitpython anymore here

aguschin · 2022-07-08T09:36:26Z

Fixed and resolved some stuff, answered other comments. Waiting for your follow-up on the ones we need to address. @shcheklein

example-mlem-get-started/generate.sh

aguschin · 2022-07-11T09:07:20Z

Ok, now I think I've addressed everything we've discussed, @shcheklein :) If you agree, then please approve the PR.

example-mlem-get-started/code/README.md

shcheklein · 2022-07-12T00:08:30Z

example-mlem-get-started/code/README.md

+[Get Started](https://mlem.ai/doc/get-started). It is a step-by-step quick
+introduction into basic MLEM concepts.
+
+- The `main` branch contains only the code needed to start the Get Started. This


That's quite unusual - downside is that people who come to the repo won't see anything in it. This is not good to my mind. What is the purpose of keeping the main like this at all if we have the corresponding tags? Can it be done this way:

simple becomes main

dvc - can it be done in way that builds on top of simple - we can make a PR from dvc to main (ex-simple) to DVC-ify the project?

Did that and committed changes. Please see iterative/mlem.ai#132 for PR updating documentation.

example-mlem-get-started/generate.sh

shcheklein · 2022-07-12T00:11:39Z

@aguschin thanks! we are getting there :) I still have some questions though. Please see some unresolved comments.

example-mlem-get-started/code/README.md

shcheklein

Some minor comments left and I didn't try to run it.

Good progress, we are almost there.

Overall, still look too complicated for the MLEM get started. I think as we come to the updates on docs we can optimize this further.

aguschin · 2022-07-18T06:14:36Z

Thanks for the feedback. Fixed those @shcheklein.

shcheklein · 2022-07-18T23:45:12Z

@aguschin one unresolved item left - https://github.com/iterative/example-repos-dev/pull/109/files#r919031250 - up to you to address or merge! Good stuff.

Changes addressed :)

aguschin · 2022-07-21T11:14:08Z

All changes addressed. I've just updated https://github.com/iterative/example-mlem-get-started

Now merging iterative/mlem.ai#132 and #109

Thanks for your help and feedback @jorgeorpinel and @shcheklein !

add mlem get started

7993131

mike0sv requested a review from casperdcl April 19, 2022 20:20

mike0sv marked this pull request as draft April 19, 2022 20:22

shcheklein reviewed Apr 19, 2022

View reviewed changes

example-mlem-get-started/generate.sh Outdated Show resolved Hide resolved

shcheklein reviewed Apr 19, 2022

View reviewed changes

example-mlem-get-started/generate.sh Outdated Show resolved Hide resolved

shcheklein reviewed Apr 19, 2022

View reviewed changes

example-mlem-get-started/generate.sh Outdated Show resolved Hide resolved

shcheklein reviewed Apr 19, 2022

View reviewed changes

example-mlem-get-started/generate.sh Outdated Show resolved Hide resolved

shcheklein reviewed Apr 19, 2022

View reviewed changes

example-mlem-get-started/generate.sh Outdated Show resolved Hide resolved

shcheklein reviewed Apr 19, 2022

View reviewed changes

example-mlem-get-started/generate.sh Outdated Show resolved Hide resolved

shcheklein reviewed Apr 19, 2022

View reviewed changes

example-mlem-get-started/generate.sh Outdated Show resolved Hide resolved

shcheklein reviewed Apr 19, 2022

View reviewed changes

example-mlem-get-started/code/README.md Outdated Show resolved Hide resolved

shcheklein requested changes Apr 19, 2022

View reviewed changes

Update some stuff

e5bd46c

aguschin reviewed Apr 21, 2022

View reviewed changes

example-mlem-get-started/generate.sh Outdated Show resolved Hide resolved

aguschin reviewed Apr 21, 2022

View reviewed changes

example-mlem-get-started/generate.sh Outdated Show resolved Hide resolved

mike0sv and others added 2 commits April 21, 2022 16:11

Update example-mlem-get-started/generate.sh

234576f

Co-authored-by: Alexander Guschin <[email protected]>

remove commit

94424a0

shcheklein added a commit that referenced this pull request Apr 21, 2022

get-started: remove gitpython, add a comment

0c79ff8

Per #109 (comment) - we don't need gitpython anymore here

shcheklein mentioned this pull request Apr 21, 2022

get-started: remove gitpython, add a comment #111

Merged

shcheklein added a commit that referenced this pull request Apr 21, 2022

get-started: remove gitpython, add a comment (#111)

6818a5c

Per #109 (comment) - we don't need gitpython anymore here

mike0sv added 3 commits April 23, 2022 02:34

add pack step

ed79b4f

refactor tutorial

e75c3da

refactor tutorial

daaea8f

shcheklein changed the title ~~add mlem get started~~ Add mlem get started May 12, 2022

less datasets in tutorial

0571605

aguschin self-assigned this Jul 8, 2022

aguschin added 3 commits July 8, 2022 15:40

making dead code live; formatting

51f32c1

fix devcontainer

7ce9d77

fix devcontainer

ca0ae1d

shcheklein reviewed Jul 9, 2022

View reviewed changes

example-mlem-get-started/generate.sh Show resolved Hide resolved

add description for branches

0bd97e2

shcheklein reviewed Jul 12, 2022

View reviewed changes

example-mlem-get-started/code/README.md Outdated Show resolved Hide resolved

shcheklein reviewed Jul 12, 2022

View reviewed changes

example-mlem-get-started/generate.sh Outdated Show resolved Hide resolved

address minor feedback

20582ea

aguschin mentioned this pull request Jul 15, 2022

Docs: update DVC Use Case iterative/mlem.ai#132

Merged

aguschin added 2 commits July 15, 2022 17:40

remove simple branch; add pr to dvc-ify the project

8b33bd3

uncomment deploy step

e79b2ec

shcheklein reviewed Jul 16, 2022

View reviewed changes

example-mlem-get-started/code/README.md Outdated Show resolved Hide resolved

shcheklein reviewed Jul 16, 2022

View reviewed changes

example-mlem-get-started/code/README.md Outdated Show resolved Hide resolved

shcheklein previously requested changes Jul 16, 2022

View reviewed changes

remove gh usage

3c4ecd8

aguschin added 2 commits July 19, 2022 13:48

install DVC as MLEM's dependency

5cbdeb8

cleaning up requirements.txt

0b3fb35

aguschin merged commit d0b3914 into master Jul 21, 2022

aguschin deleted the add-mlem-get-started branch July 21, 2022 11:14

jorgeorpinel added A: docs Area: user documentation (related) A: example-mlem-get-started MLEM examples labels Jul 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add mlem get started #109

Add mlem get started #109

mike0sv commented Apr 19, 2022

mike0sv commented Apr 19, 2022

shcheklein Apr 19, 2022

mike0sv Apr 20, 2022

mike0sv Apr 20, 2022 •

edited

Loading

mike0sv Apr 20, 2022

shcheklein Apr 20, 2022

mike0sv Apr 20, 2022 •

edited

Loading

shcheklein Apr 20, 2022

mike0sv Apr 20, 2022 •

edited

Loading

shcheklein Apr 19, 2022

mike0sv Apr 20, 2022

shcheklein Apr 20, 2022

mike0sv Apr 20, 2022

shcheklein Apr 20, 2022

mike0sv Apr 20, 2022

shcheklein Apr 20, 2022

mike0sv Apr 21, 2022

shcheklein Apr 21, 2022 •

edited

Loading

shcheklein left a comment

mike0sv commented Apr 20, 2022

shcheklein commented Apr 20, 2022

aguschin commented Jul 8, 2022 •

edited

Loading

aguschin commented Jul 11, 2022

shcheklein Jul 12, 2022

aguschin Jul 15, 2022

shcheklein commented Jul 12, 2022

shcheklein left a comment

aguschin commented Jul 18, 2022

shcheklein commented Jul 18, 2022

aguschin commented Jul 21, 2022



		cp $HERE/code/src/requirements.txt .
		cp $HERE/code/src/prepare.py .

Add mlem get started #109

Add mlem get started #109

Conversation

mike0sv commented Apr 19, 2022

mike0sv commented Apr 19, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mike0sv Apr 20, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mike0sv Apr 20, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mike0sv Apr 20, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shcheklein Apr 21, 2022 • edited Loading

Choose a reason for hiding this comment

shcheklein left a comment

Choose a reason for hiding this comment

mike0sv commented Apr 20, 2022

shcheklein commented Apr 20, 2022

aguschin commented Jul 8, 2022 • edited Loading

aguschin commented Jul 11, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shcheklein commented Jul 12, 2022

shcheklein left a comment

Choose a reason for hiding this comment

aguschin commented Jul 18, 2022

shcheklein commented Jul 18, 2022

aguschin commented Jul 21, 2022

mike0sv Apr 20, 2022 •

edited

Loading

mike0sv Apr 20, 2022 •

edited

Loading

mike0sv Apr 20, 2022 •

edited

Loading

shcheklein Apr 21, 2022 •

edited

Loading

aguschin commented Jul 8, 2022 •

edited

Loading