Filtering by hyperparameters in ml-dash

[btickell]

I'm trying to revamp my workflow and this package seems to be a good starting point. I was wondering if it is possible to filter results based on parameters configs. I am running experiments with doodad and as such I have several seeds under a single experiment-name. I want to grab values based on either unique parameter configurations or results that share the same value (lr=.01 for example). Is this currently available?

[geyang]

Yes, you can take a look at the examples here:

https://ml-logger.readthedocs.io/en/latest/modindex.html?highlight=get_parameters#ml_logger.ML_Logger.get_parameters

experiments = logger.glob("some_prefix/blah/*/parameters.pkl")
for exp_key in experiments:
      params = logger.get_params(key=exp_key)

I just typed this up by hand, you can check the docs for details.

[btickell]

Ok this looks good, can I get plots from multiple runs based on this filter or is dynamic plotting something that needs to be done on top?

[geyang]

@btickell Sorry for this late reply! A lot happened in the past year. In case this is helpful, this does return results from all of the experiments that match that glob string. I believe everything is combined in a single DataFrame object.