-
Notifications
You must be signed in to change notification settings - Fork 44
MLEM index and .mlem/
folder
#381
Comments
I think we should not be doing |
Interestingly, rn |
I'd suggest to remove the dir with the ls command. If you find it useful it can be replaced by something like |
IMO, Also, what are the reasons for DVC to have |
related to #247 |
DVC has data management layer and And even in that case DVC still doesn't have an index for good or bad. |
MLEM also has a concept of project, which is tied with |
To summarize a bit What I changed in PR above: Pros:
Alternative: |
Thanks @mike0sv ! From reading this, it makes sense to simplify, drop it, simplify it. To clarify a few things:
it's just slower, right? (and there are ways to speedup it even if we decide not to keep an explicit directory like .mlem).
when using Git this whole directory is saved into Git, right?
we can rely on GTO (or DVC if we move index there) and show only registered objects? (I agree though that the full list would be good to have). Full scan is probably not a big issue (we do this already in DVC). At least for now.
that's a bit separate discussion, but may be we can move deployments / envs into a single file (mlem.yaml)? Pros/cons that come to my mind (not necessarily major, just everything from the top of my head):
will think more about this ⌛ |
@shcheklein thanks for the feedback! To what I know:
AFAIK, yes, it's slower. And rn we release 0.3.0 without
Yes.
Yes, as an option.
Do you mean moving ALL envs and ALL deployments to a single file (e.g. |
I'd say if we dont have index, dont act like we have one. We can implement repo scanning as internal api used only by studio and dont expose it to cli.
yes, unless you store artifacts with
This will mean that mlem only works with studio only if you use it with dvc/gto, which is very undesirable IMO. Again, for studio we can easily implement internal
From UX perspective I guess it's just different preferences. We can squash |
Thanks, @mike0sv @aguschin for clarifications.
that's a bit sad, but I feel fine to release w/o
thanks, Mike, that's helpful to know. Let's def not touch it now. Thought-process I had in mind though is that for certain object (deployments, env) config files can be simpler and a bit more opinionated. It's not about making everything a single files. It goes more in direction of making things similar to dvc remotes. (you don't create a separate file for it), or Git remotes. In our case envs could be defined like this, maybe deployments also. |
closed by #395 |
Feedback from @daavoo
This brings us back to the discussion of having an index in MLEM. I know @shcheklein and @mike0sv had conversations about this. So, have you decided something on this? From what I recall:
$ mlem init
to untie MLEM usage from MLEM project (we have a ticket for it in mlem.ai repo)mlem ls
without.mlem/
we need to implement repo parsing (just like DVC I suppose)The text was updated successfully, but these errors were encountered: