W&B prompt when running an experiment

[bhigy]

Whenever I try to run an experiment I get the following prompt:

wandb: (1) Create a W&B account
wandb: (2) Use an existing W&B account
wandb: (3) Don't visualize my results
wandb: Enter your choice:

By default, I would prefer if the user doesn't have to do anything. W&B could be inactive or use choice (3) above.

[cwmeijer]

I agree, we should check how to make it be quiet by default.

[cwmeijer]

According to (wandb/wandb#764 (comment)), you can completely disable & silence wandb, making all calls to the package a noop without raising exceptions. This can be done by running:
wandb disabled
I think this should be enough. It doesn't seem simple to somehow make disabling wandb part of Platalea's installation procedure in order to make it disabled by default. It would make sense to make it part of the documentation I guess or to point to the command above in a print message. I'm not sure what would make the most sense. Any ideas? (adding help wanted label)

[cwmeijer]

Let's implement both options ot inform user of how to disable wandb:

• comment in readme
• print in output when running the code

[bhigy]

I don't know how I missed that but according to wandb/wandb#764 (comment), it seems possilbe to set the mode in the code when initializing wandb. It would be nice to add q parameter to set this which would be set to disabled by default. I am using platalea for another project and asking people to disable wandb manually is cumbersome.

I'll see if I can work on that today.

[bhigy]

One limit of using a command line parameter (e.g. wandb_mode) is that we override wandb functionalities (such as using the command wandb offline or setting the environment variable WANDB_MODE. I need to see whether there is a better way to handle this.

[cwmeijer]

Overriding is maybe not too bad. I would be fine with just adding '--wand_mode on' with every run.

[bhigy]

I tried to find a way to just set disabled as the default instead of online but there is no way to know for sure whether the value online is the default and can be overwritten or has been set explicitly by the user. Also having a global option --wandb_mode seems redundant.

One solution that seems better to me is to transfer the responsability of this to the experiment script (e.g. basic.py). By adding a parameter to the experiment function, we can decide how to handle this for each experiment (keep the current behavior, disable completely wandb, add an experiment-specific switch). This way, in the experiment I will use for the zerospeech challenge I can just disable it by default.

[bhigy]

Maybe also allow running experiments without installing it.

[egpbos]

Maybe also allow running experiments without installing it.

We can make it an optional dependency. Then, instead of importing, we can do something like this (wrapped in a function perhaps so we don't have to repeat everywhere):

try:
    import wandb
except ImportError:
    class PassClass:
        def __getattr__(self, name):
            def dummy_func(*args, **kwargs):
                pass
            return dummy_func
    wandb = PassClass()

This way, when the module is not installed, wandb.anything(something, whatever=bla) will just pass, i.e. do nothing. So all the wandb.function() calls can just stay in and don't have to be wrapped in if statements.

[bhigy]

First part (disabling by default) has been implemented in the branch zerospeech21-vg.