[DOC] Add details of deep learning classifiers in the deep learning notebook

README.md

@@ -27,7 +27,7 @@ does not apply:
 - `similarity_search`
 - `visualisation`
 
-| Overview        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
+| Overview        |                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
 |-----------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | **CI/CD**       | [![github-actions-release](https://img.shields.io/github/actions/workflow/status/aeon-toolkit/aeon/release.yml?logo=github&label=build%20%28release%29)](https://github.com/aeon-toolkit/aeon/actions/workflows/release.yml) [![github-actions-main](https://img.shields.io/github/actions/workflow/status/aeon-toolkit/aeon/pr_pytest.yml?logo=github&branch=main&label=build%20%28main%29)](https://github.com/aeon-toolkit/aeon/actions/workflows/pr_pytest.yml) [![github-actions-nightly](https://img.shields.io/github/actions/workflow/status/aeon-toolkit/aeon/periodic_tests.yml?logo=github&label=build%20%28nightly%29)](https://github.com/aeon-toolkit/aeon/actions/workflows/periodic_tests.yml) [![docs-main](https://img.shields.io/readthedocs/aeon-toolkit/stable?logo=readthedocs&label=docs%20%28stable%29)](https://www.aeon-toolkit.org/en/stable/) [![docs-main](https://img.shields.io/readthedocs/aeon-toolkit/latest?logo=readthedocs&label=docs%20%28latest%29)](https://www.aeon-toolkit.org/en/latest/) [![!codecov](https://img.shields.io/codecov/c/github/aeon-toolkit/aeon?label=codecov&logo=codecov)](https://codecov.io/gh/aeon-toolkit/aeon) [![openssf-scorecard](https://api.scorecard.dev/projects/github.com/aeon-toolkit/aeon/badge)](https://img.shields.io/ossf-scorecard/github.com/aeon-toolkit/aeon?label=openssf%20scorecard&style=flat) |
 | **Code**        | [![!pypi](https://img.shields.io/pypi/v/aeon?logo=pypi&color=blue)](https://pypi.org/project/aeon/) [![!conda](https://img.shields.io/conda/vn/conda-forge/aeon?logo=anaconda&color=blue)](https://anaconda.org/conda-forge/aeon) [![!python-versions](https://img.shields.io/pypi/pyversions/aeon?logo=python)](https://www.python.org/) [![!black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![license](https://img.shields.io/badge/license-BSD%203--Clause-green?logo=style)](https://github.com/aeon-toolkit/aeon/blob/main/LICENSE) [![binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/aeon-toolkit/aeon/main?filepath=examples)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |

examples/classification/deep_learning.ipynb

@@ -25,42 +25,25 @@
     "although it is worth checking the output.\n"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "metadata": {},
-   "source": []
-  },
   {
    "cell_type": "code",
-   "execution_count": 1,
+   "execution_count": 3,
    "metadata": {},
    "outputs": [
-    {
-     "name": "stderr",
-     "output_type": "stream",
-     "text": [
-      "2024-11-12 08:25:34.582860: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.\n",
-      "2024-11-12 08:25:34.583988: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.\n",
-      "2024-11-12 08:25:34.626288: I external/local_tsl/tsl/cuda/cudart_stub.cc:32] Could not find cuda drivers on your machine, GPU will not be used.\n",
-      "2024-11-12 08:25:34.772595: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
-      "To enable the following instructions: AVX2 AVX512F AVX512_VNNI FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
-      "2024-11-12 08:25:35.886826: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT\n"
-     ]
-    },
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "\u001B[1m65/65\u001B[0m \u001B[32m━━━━━━━━━━━━━━━━━━━━\u001B[0m\u001B[37m\u001B[0m \u001B[1m0s\u001B[0m 2ms/step\n"
+      "\u001b[1m65/65\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m0s\u001b[0m 4ms/step\n"
      ]
     },
     {
      "data": {
       "text/plain": [
-       "0.49854227405247814"
+       "0.5014577259475219"
       ]
      },
-     "execution_count": 1,
+     "execution_count": 3,
      "metadata": {},
      "output_type": "execute_result"
     }
@@ -106,7 +89,33 @@
     "\n",
     "An ``IndividualInceptionClassifier`` is structured as follows.\n",
     "\n",
-    "<img src=\"./img/inception.png\" width=\"600\" alt=\"ROCKET.\">"
+    "<img src=\"./img/inception.png\" width=\"600\" alt=\"ROCKET.\">\n",
+    "\n",
+    "The ``BaseDeepClassifier`` serves as an abstract foundation for deep learning-based time series classifiers. It provides default implementations for prediction methods and requires defining a custom model-building process. Supporting multivariate inputs and leveraging TensorFlow, it includes key parameters like batch size, random state, and output file names, emphasizing flexibility and extensibility for deep learning tasks.\n",
+    "\n",
+    "The ``TimeCNNClassifier`` is a deep learning model designed for time-series classification using convolutional neural networks (CNN), as described in [5]. It consists of multiple convolution layers with customizable kernel sizes, number of filters, activation functions, and pooling layers. The model supports various configurations for padding, strides, and dilation rates.  The model's adaptability allows it to handle noise and achieve superior classification accuracy. Customizable parameters include kernel sizes, activation functions, and pooling layers, supporting versatile applications across various domains.\n",
+    "The CNN architecture for three-variate time series classification is structured follows.\n",
+    "\n",
+    "<img src=\"./img/TimeCNNClassifier.png\" width=\"600\" alt=\"ROCKET\">\n",
+    "\n",
+    "The ``FCNClassifier`` differs from the ``TimeCNNClassifier`` primarily in its deeper architecture with three convolutional layers, larger default filters, and kernel sizes designed for time-series feature extraction [5]. While both utilize convolutional operations, FCN focuses on categorical cross-entropy loss, whereas TimeCNN emphasizes mean-squared-error loss, making FCN more suited for classification tasks.\n",
+    "\n",
+    "The ``DisjointCNNClassifier`` is a deep learning model designed for time-series classification using a unique 1+1D convolutional architecture [6]. It features four convolutional layers, customizable filter sizes, ELU activations, and a max-pooling layer. The model integrates fully connected layers and optimizes using categorical cross-entropy loss, focusing on flexibility and accuracy for diverse time-series datasets.\n",
+    "The overall architecture of the disjointCNN model and 1+1D block is as follows.\n",
+    "\n",
+    "<img src=\"./img/DisjointCNN.png\" width=\"600\" alt=\"ROCKET\">\n",
+    "\n",
+    "The ``LITETimeClassifier`` is an ensemble classifier combining multiple ``IndividualLITETimeClassifier`` models to handle univariate and multivariate time-series data [7]. It uses configurable layers, kernels, and activations, supporting LITE or LITEMV variants for diverse tasks. Key features include categorical cross-entropy loss, adjustable filters, and extensive model-saving options for flexibility and robustness in time-series classification.\n",
+    "The LITE architecture is as follows.\n",
+    "\n",
+    "<img src=\"./img/LITE architecture.png\" width=\"600\" alt=\"ROCKET\">\n",
+    "\n",
+    "HInception is a hierarchical variant of Inception modules, designed to process multi-scale data with an emphasis on capturing hierarchical relationships in time-series datasets [8].\n",
+    "\n",
+    "The ``EncoderClassifier`` is a neural network model tailored for time-series classification. It features 1D convolution layers with customizable kernel sizes, filters, and padding [9]. Incorporating max pooling, dropout, and a fully connected layer, it uses sigmoid activation and categorical cross-entropy loss. Designed for flexibility, it supports various optimizers, callbacks, and saving mechanisms, offering robust performance for diverse datasets.\n",
+    "The architecture diagram of the encoder(top) and the convolutional block(bottom left) is as follows.\n",
+    "\n",
+    "<img src=\"./img/EncoderClassifier.png\" width=\"600\" alt=\"ROCKET\">\n"
    ]
   },
   {
@@ -283,21 +292,28 @@
     "[3] Foumani et al. (2023) \"Deep Learning for Time Series Classification and Extrinsic\n",
     "Regression: A Current Survey\" ArXiv https://arxiv.org/pdf/2302.02515.pdf\n",
     "\n",
-    "[4]  https://github.com/MSD-IRIMAS/CF-4-TSC"
+    "[4]  https://github.com/MSD-IRIMAS/CF-4-TSC\n",
+    "\n",
+    "[5] Zhao et. al, Convolutional neural networks for time series classification,\n",
+    "    Journal of Systems Engineering and Electronics, 28(1):2017\n",
+    "\n",
+    "[6] Foumani, Seyed Navid Mohammadi, Chang Wei Tan, and Mahsa Salehi.\n",
+    "    \"Disjoint-cnn for multivariate time series classification\n",
+    "\n",
+    "[7] Ismail-Fawaz et al. LITE: Light Inception with boosTing\n",
+    "    tEchniques for Time Series Classification, IEEE International\n",
+    "    Conference on Data Science and Advanced Analytics, 2023\n",
+    "\n",
+    "[8] Mancuso, P., Piccialli, V. and Sudoso, A.M., 2021. A machine learning approach for forecasting hierarchical time series. Expert Systems with Applications, 182, p.115102.\n",
+    "\n",
+    "[9] Serrà et al. Towards a Universal Neural Network Encoder for Time Series In proceedings International Conference of the Catalan Association for Artificial Intelligence, 120--129 2018"
    ]
-  },
-  {
-   "cell_type": "markdown",
-   "metadata": {
-    "collapsed": false
-   },
-   "source": []
   }
  ],
  "metadata": {
   "file_extension": ".py",
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "base",
    "language": "python",
    "name": "python3"
   },
@@ -311,7 +327,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.12"
+   "version": "3.11.7"
   },
   "mimetype": "text/x-python",
   "name": "python",