Figuring out the log files

[jmamath]

I am trying to understand the log output.
After running a training command for instance python examples/run_expt.py --dataset camlyon17 --algorithm ERM --root_dir data I will get a log folder with many files. What is the difference between test_algo.csv and test_eval.csv. I have seen that they are related to two loggers:

datasets[split]['eval_logger'] = BatchLogger(
            os.path.join(config.log_dir, f'{split}_eval.csv'), mode=mode, use_wandb=(config.use_wandb and verbose))
datasets[split]['algo_logger'] = BatchLogger(
            os.path.join(config.log_dir, f'{split}_algo.csv'), mode=mode, use_wandb=(config.use_wandb and verbose))

What is the difference between algo and eval ?

[kohpangwei]

Hi Jean-Michel,

Thanks for the question! The *_algo.csv files log the outputs of the algorithm (e.g., the model loss), while the *_eval.csv files log the evaluation results (e.g., the accuracy of the predictions), which are independent of which algorithm is used to train the model. Does that make sense?

[jmamath]

Ok, I get it, thank you.