Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
How to access user data
Problem: it isn't clear what the interface is to access the values from inside the model code.
Users can call their own arbitrary user_data from the Experiment Configuration file. This would be things like data sources and containers or even credentials (which are obscurable). To do this, the user uses
data
(notmetadata.data
) in the Experiment configuration yaml file. Which is documented here. However, it is not clear how Determined accesses the values of this user data from inside the model code.Solution:
Help users understand that the way to access the data:
https://docs.determined.ai/latest/reference/experiment-config-reference.html#data ...
is here
https://docs.determined.ai/latest/reference/training/api-det-reference.html#determined.ClusterInfo.user_data
another way
another way is just the xpath, for example, under the "username" section we could have task_container_defaults.registry_auth.username
there are 3 ways to retrieve data config and one way to get all of it:
From DET_EXPERIMENT_CONFIG env var like:
expconf = os.environ.get("DET_EXPERIMENT_CONFIG")
From context
data_config = train_context.get_data_config()
From cluster_info
data = det.get_cluster_info().user_data