This repository contains the code and experiments for the ALTEGRAD 2024 project by Rachida Saroui, focusing on using transfer learning to improve text classification tasks. This project utilizes pre-trained transformer models and custom architectures such as AttentionBiGRU, HAN (Hierarchical Attention Networks), and TimeDistributed layers to build powerful classifiers for document classification tasks.
Key Features:
- Transfer Learning: Leveraging pre-trained transformer models for efficient training on a smaller dataset.
- Custom Architectures: Includes AttentionBiGRU and Hierarchical Attention Networks (HAN) for better document classification performance.
- Dynamic Dataset Handling: A custom PyTorch dataset loader to efficiently manage and preprocess documents and labels.
To run this project, you'll need the following dependencies:
- Python 3.8+
- PyTorch 1.10+
- Transformers 4.3.0+
- NumPy
- Pandas
- Matplotlib (optional, for plotting)
- scikit-learn (for evaluation metrics)