Proposal for Aim Visualizer Backend

[GeorgePearse]

What is the feature?

Aim is a great experiment tracker.

Currently no visualizer implemented for it.

Not thinking the below is complete (I use the datetime to create the experiment name), but may be useful for other users. Happy to make changes to actually get it merged in.

import aim
import datetime

@VISBACKENDS.register_module()
class AimVisBackend(BaseVisBackend):
    """
    Based on https://github.com/open-mmlab/mmengine/blob/main/mmengine/visualization/vis_backend.py 
    """

    def __init__(
            self,
            save_dir: str,  
        ):
        super().__init__(save_dir)
        self._init_env()

        
    def _init_env(self):
        """Setup env for MLflow."""

        try:
            import aim
        except ImportError:
            raise ImportError(
                'Please run "pip install aim" to install aim'
            )  # type: ignore
            
        clean_datetime = str(datetime.now()).replace(' ','_').replace(':','-')
        self._aim_run = aim.Run(
            experiment = clean_datetime,
        )
        
    @property  # type: ignore
    @force_init_env
    def experiment(self):
        """Return MLflow object."""
        return self._aim_run
    
    @force_init_env
    def add_config(self, config, **kwargs) -> None:
        """Record the config to mlflow.
        Args:
            config (Config): The Config object
        """
        # works perfectly
        self._aim_run['hparams'] = dict(config)
        
        # looks a bit clumsy in Aim UI
        aim_text = aim.Text(str(config))
        self._aim_run.track(aim_text, name='Config', step=0)
        
    @force_init_env
    def add_scalar(self,
                   name: str,
                   value: Union[int, float, torch.Tensor, np.ndarray],
                   step: int = 0,
                   **kwargs) -> None:
        """Record the scalar data to mlflow.
        Args:
            name (str): The scalar identifier.
            value (int, float, torch.Tensor, np.ndarray): Value to save.
            step (int): Global step value to record. Default to 0.
        """
        self._aim_run.track(name=name, value=value, step=step)
        
        
    @force_init_env
    def add_scalars(self,
                    scalar_dict: dict,
                    step: int = 0,
                    file_path: Optional[str] = None,
                    **kwargs) -> None:
        """Record the scalar's data to wandb.
        Args:
            scalar_dict (dict): Key-value pair storing the tag and
                corresponding values.
            step (int): Useless parameter. Wandb does not
                need this parameter. Defaults to 0.
            file_path (str, optional): Useless parameter. Just for
                interface unification. Defaults to None.
        """
        for k in scalar_dict:
            self._aim_run.track(name=k, value=scalar_dict[k], step=step)

Any other context?

The use of 'save_dir' as the kwarg fed through to the VisualizerBackend seems a little odd,
feels like it comes from the Local Visualizer but has no analogy for most other forms, e.g. MLFLow, Neptune, W and B.

TypeError: class `DetLocalVisualizer` in mmdet/visualization/local_visualizer.py: class `AimVisBackend` in viz_backend.py: __init__() got an unexpected keyword argument 'save_dir'

Surely a better core would be experiment_name or experiment_id?

[HAOCHENYE]

Thank you very much for your suggestion, and I am really looking forward to your submission of a PR to implement this feature 😆 !

As far as I know, MLFlow and W&B do indeed utilize this parameter, although it's true that it is not essential for some visualization backends. Regardless, it is recommended to maintain the save_dir for now. The experiment_name parameter is indeed more common, and I suggest adding it to each Visbackend, which will also make it convenient for the Runner to pass in.