WPMixer: Efficient Multi-Resolution Mixing for Long-Term Time Series Forecasting

---

@misc{murad2024wpmixerefficientmultiresolutionmixing,
      title={WPMixer: Efficient Multi-Resolution Mixing for Long-Term Time Series Forecasting}, 
      author={Md Mahmuddun Nabi Murad and Mehmet Aktukmak and Yasin Yilmaz},
      year={2024},
      eprint={2412.17176},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2412.17176}, 
}

Get started

Follow these steps to get started with WPMixer:

1. Install Requirements

Install Python 3.10 and the necessary dependencies.

pip install -r requirements.txt

2. Download Data

Process-1: Download the zip file of the datasets from the link. Paste the zip file inside the root folder and extract. Now you will have ./data/ folder containing all the datasets.

Or, Process-2: Download the data and locate them in the ./data/ folder. You can download all data from the public GitHub repo: Autoformer or TimeMixer. All the datasets are well-pre-processed and can be used easily. To place and rename the datasets file, check the following folder tree,

Figure: Folder Tree

3. Train the model

We provide the experiment scripts of all benchmarks under the folder ./scripts/ to reproduce the results. Running those scripts by the following commands will generate logs in the ./logs/WPMixer/ folder.

Multivariate long-term forecasting results with full hyperparameter search settings (Table-2):

bash ./scripts/Full_HyperSearch/ETTh1_full_hyp.sh
bash ./scripts/Full_HyperSearch/ETTh2_full_hyp.sh
bash ./scripts/Full_HyperSearch/ETTm1_full_hyp.sh
bash ./scripts/Full_HyperSearch/ETTm2_full_hyp.sh
bash ./scripts/Full_HyperSearch/Weather_full_hyp.sh
bash ./scripts/Full_HyperSearch/Electricity_full_hyp.sh
bash ./scripts/Full_HyperSearch/Traffic_full_hyp.sh

Multivariate long-term forecasting results with unified settings (Table-9 in Supplementary):

bash ./scripts/Unified/ETTh1_Unified_setup.sh
bash ./scripts/Unified/ETTh2_Unified_setup.sh
bash ./scripts/Unified/ETTm1_Unified_setup.sh
bash ./scripts/Unified/ETTm2_Unified_setup.sh
bash ./scripts/Unified/Weather_Unified_setup.sh
bash ./scripts/Unified/Electricity_Unified_setup.sh
bash ./scripts/Unified/Traffic_Unified_setup.sh

Univariate long-term forecasting results (Table-10 in Supplementary):

bash ./scripts/Univariate/ETTh1_univariate.sh
bash ./scripts/Univariate/ETTh2_univariate.sh
bash ./scripts/Univariate/ETTm1_univariate.sh
bash ./scripts/Univariate/ETTm2_univariate.sh

---

Brief Overview of the Paper

---

Abstract

Time series forecasting is crucial for various applications, such as weather forecasting, power load forecasting, and financial analysis. In recent studies, MLP-mixer models for time series forecasting have been shown as a promising alternative to transformer-based models. However, the performance of these models is still yet to reach its potential. In this paper, we propose Wavelet Patch Mixer (WPMixer), a novel MLP-based model, for long-term time series forecasting, which leverages the benefits of patching, multi-resolution wavelet decomposition, and mixing. Our model is based on three key components: (i) multi-resolution wavelet decomposition, (ii) patching and embedding, and (iii) MLP mixing. Multi-resolution wavelet decomposition efficiently extracts information in both the frequency and time domains. Patching allows the model to capture an extended history with a look-back window and enhances capturing local information while MLP mixing incorporates global information. Our model significantly outperforms state-of-the-art MLP-based and transformer-based models for long-term time series forecasting in a computationally efficient way, demonstrating its efficacy and potential for practical applications.

Model Architecture:

Multivariate Long-Term Forecasting Results with full hyperparameter searching:

Multivariate Long-Term Forecasting under Unified Setting:

Univariate Long-term forecasting result: