[<Library component: Models|Core|etc...>] Adding ModernTCN, Mamba request

[skmanzg]

Description

1. Adding model(ModernTCN, Mamba) request
2. Question about the adding method
3. review comment

Use case

1. I would like to ask to add model ModernTCN(IRIC 2024), and Mamba.

Modern TCN Github repo
This model uses CNN but the architecture is Transformer.

Simple Mamba for Time Series forecasting
Mamba for Time Series forecasting
Mamba was released as a 'Transformer substitute.'

2. I have tried to add ModernTCN in my local environment but I found that the document 'Adding models To NeuralForecast' is quite outdated and limited. Nonetheless, I tried to refer existed ipynb files in 'nbs.' According to the document, forward function must be added like this below:

    def forward(self, windows_batch): # <<--- Receives windows_batch dictionary
        # Parse windows_batch
        insample_y = windows_batch['insample_y'].clone()
        # MLP
        y_pred = self.mlp(insample_y)
        # Reshape and map to loss domain
        y_pred = y_pred.reshape(batch_size, self.h, self.loss.outputsize_multiplier)
        y_pred = self.loss.domain_map(y_pred)
        return y_pred

The problem is that MordernTCN class has already this function, and I think most model would be like that.

    def forward(self, x, te=None):

        # instance norm
        if self.revin:
            x = x.permute(0, 2, 1)
            x = self.revin_layer(x, 'norm')
            x = x.permute(0, 2, 1)
        x = self.forward_feature(x,te)
        x = self.head(x)
        # de-instance norm
        if self.revin:
            x = x.permute(0, 2, 1)
            x = self.revin_layer(x, 'denorm')
            x = x.permute(0, 2, 1)
        return x

I wonder how to fix the function in this case? and when I look at the TimeLLM and TimeMixer ipynb files, their foward functions in the model name class were different from the guide document.

3. The inverse transformation for mixture of sliding window, scaling and complicated dimensions is quite difficult than I thought when I tried to do it alone only to fail and waste time. One of the reasons that I prefer to use this library is that it provides inverse transformation of the outputs while time-series library does not.

[marcopeix]

Hello! Didn't look into it in much detail (I will get more time soon), but basically, your forward function should take self and windows_batch and then the x in the forward function is windows_batch['insample_y']. Maybe this can help you get unstuck.

Otherwise, thanks for raising this issue! You're right we should:

• Update the documentation on adding a model
• Include Mamba (I think it's a good addition to the library)
• Read the paper on ModernTCN and see if it's worth including