├── .gitignore ├── Forecasting-using-MAE-Loss.ipynb ├── Forecasting-using-MAPE-Loss.ipynb ├── Forecasting-using-MASE-Loss.ipynb ├── Forecasting-using-MSE-Loss.ipynb ├── Forecasting-using-MSLE-Loss.ipynb ├── Forecasting-using-Quantile-Loss.ipynb ├── Forecasting-using-RMSE-Loss.ipynb ├── Forecasting-using-SMAPE-Loss.ipynb └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | /.ipynb_checkpoints/ 2 | /.idea/ -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Regression Loss Functions Performance Evaluation in Time Series Forecasting using Temporal Fusion Transformers 2 | [![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7542536.svg)](https://doi.org/10.5281/zenodo.7542536) 3 | 4 | ``` 5 | This repository compares the performance of 8 different regression loss functions used 6 | in Time Series Forecasting using Temporal Fusion Transformers. Summary of experiment with 7 | instructions on how to replicate this experiment can be find below. 8 | ``` 9 | 10 | ## About Temporal Fusion Transformers 11 | 12 | Paper Link: https://arxiv.org/pdf/1912.09363.pdf 13 | 14 | Authors: Bryan Lim, Sercan Arik, Nicolas Loeff and Tomas Pfister 15 | 16 | > Abstract - Multi-horizon forecasting problems often contain a complex mix of inputs -- including static (i.e. time-invariant) 17 | > covariates, known future inputs, and other exogenous time series that are only observed historically -- without any 18 | > prior information on how they interact with the target. While several deep learning models have been proposed for 19 | > multi-step prediction, they typically comprise black-box models which do not account for the full range of inputs 20 | > present in common scenarios. In this paper, we introduce the Temporal Fusion Transformer (TFT) -- a novel 21 | > attention-based architecture which combines high-performance multi-horizon forecasting with interpretable insights 22 | > into temporal dynamics. To learn temporal relationships at different scales, the TFT utilizes recurrent layers for 23 | > local processing and interpretable self-attention layers for learning long-term dependencies. 24 | > The TFT also uses specialized components for the judicious selection of relevant features and a series of gating layers 25 | > to suppress unnecessary components, enabling high performance in a wide range of regimes. On a variety of real-world datasets, 26 | > we demonstrate significant performance improvements over existing benchmarks, and showcase three practical 27 | > interpretability use-cases of TFT. 28 | 29 | ## Experiments Summary and Our Paper 30 | 31 | ### Cite Our Paper 32 | 33 | ``` 34 | @inproceedings{jadon2024comprehensive, 35 | title={A comprehensive survey of regression-based loss functions for time series forecasting}, 36 | author={Jadon, Aryan and Patil, Avinash and Jadon, Shruti}, 37 | booktitle={International Conference on Data Management, Analytics \& Innovation}, 38 | pages={117--147}, 39 | year={2024}, 40 | organization={Springer} 41 | } 42 | ``` 43 | 44 | ### Paper Links 45 | 1. https://link.springer.com/chapter/10.1007/978-981-97-3245-6_9 46 | 2. https://arxiv.org/abs/2211.02989 47 | 48 | ![Summary of Loss Functions](https://github.com/aryan-jadon/Regression-Loss-Functions-in-Time-Series-Forecasting-Tensorflow/blob/main/loss_functions_plots/Loss-Functions-Summary.png) 49 | 50 | ### Dataset Used for this experiment - https://www.kaggle.com/datasets/utathya/future-volume-prediction 51 | 52 | Parameters Used During Experiments - 53 | 54 | | Parameters | Value | 55 | |:------------------------------------------------------|:-----:| 56 | | learning_rate | 0.03 | 57 | | hidden_size | 16 | 58 | | attention_head_size | 1 | 59 | | dropout | 0.1 | 60 | | hidden_continuous_size | 8 | 61 | 62 | 63 | | Loss Function | Value | 64 | |:-----------------------------------------------|:-----------:| 65 | | Mean Absolute Percentage Error Loss | 258170080.0 | 66 | | Mean Squared Error Loss | 501227.06 | 67 | | Quantile Loss | 1462.5258 | 68 | | Root Mean Square Error Loss | 683.93 | 69 | | Mean Absolute Scaled Error Loss | 288.15 | 70 | | Mean Absolute Error Loss | 264.46 | 71 | | Symmetric Mean Absolute Percentage Loss | 0.40 | 72 | | Mean Squared Log Error Loss | 0.34 | 73 | 74 | 75 | ## Replicate This Experiment 76 | 77 | ### Step 1: Install the Requirements 78 | 79 | Installing Pytorch Forecasting 80 | 81 | If you are working windows, you need to first install PyTorch with 82 | ```bash 83 | pip install torch -f https://download.pytorch.org/whl/torch_stable.html. 84 | ``` 85 | 86 | Otherwise, you can proceed with 87 | ```bash 88 | pip install pytorch-forecasting 89 | ``` 90 | 91 | Alternatively, to install the package via conda: 92 | ```bash 93 | conda install pytorch-forecasting pytorch>=1.7 -c pytorch -c conda-forge 94 | ``` 95 | 96 | Installing TorchMetrics 97 | 98 | You can install TorchMetrics using pip or conda: 99 | 100 | Python Package Index (PyPI) 101 | ```bash 102 | pip install torchmetrics 103 | ``` 104 | 105 | Conda 106 | ```bash 107 | conda install -c conda-forge torchmetrics 108 | ``` 109 | 110 | ### Step 2: Running Experiment Notebooks 111 | 112 | ```bash 113 | jupyter notebook 114 | ``` 115 | 116 | --------------------------------------------------------------------------------