Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error if metadata matches existing keys #1313

Merged
merged 7 commits into from
Jul 1, 2024

Conversation

dakinggg
Copy link
Collaborator

@dakinggg dakinggg commented Jun 27, 2024

We flatten the metadata so that it is easier to parse out of mlflow/run metadata. Because of this, a user's metadata key could overwrite the real key in the config that gets logged. This PR fixes that by simple erroring and asking the user to rename their metadata key. It also adds flatten_metadata to allow not flattening the metadata.

Original run that overwrites the key: metadata-with-error-4-G9ps3T
Run with duplicate key that now errors: metadata-unflat-1-ltIBGL
Run without flattening metadata: metadata-unflat-2-VoxpxS

@josejg
Copy link
Contributor

josejg commented Jun 28, 2024

Could we prefix metadata_ in the metadata keys for clarity ?

@dakinggg dakinggg marked this pull request as ready for review June 29, 2024 00:08
@dakinggg dakinggg requested a review from a team as a code owner June 29, 2024 00:08
@dakinggg dakinggg requested review from josejg and irenedea June 29, 2024 00:08
@dakinggg dakinggg enabled auto-merge (squash) June 29, 2024 00:54
@josejg
Copy link
Contributor

josejg commented Jun 29, 2024

looks good to me, but I'll give it a try just in case

scripts/train/train.py Outdated Show resolved Hide resolved
@dakinggg dakinggg merged commit 1993e7f into mosaicml:main Jul 1, 2024
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants