├── LAB TEST.ipynb ├── LSTM-AE Outlier Detection.ipynb ├── Outlier_Algorithm.ipynb ├── README.md ├── Reference ├── A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data.pdf ├── A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise.pdf ├── Algorithm optimization and AD simulation based on extended clustering and OD.pdf ├── Deep-Compact-Clustering Based Anomaly Detection Applied to Electromechanical Industrial Systems.pdf ├── Ensemble method based on ANN to estimate air pollution health risks.pdf ├── Forecasting and AD approaches using LSTM and LSTM AE wirh the app in supply chain management.pdf ├── FuseAD Unsupervised Anomaly Detection in Streaming Sensors Data by Fusing Statistical and Deep Learning Models.pdf ├── Minimum volume ellipsoid classification model for contamination event detection in water distribution systems.pdf ├── Patient classfication as an OD problem an app of the OC-SVM.pdf ├── Sequential_Fault_Diagnosis_Based_on_LSTM_Neural_Network.pdf ├── Soft clustering using weighted oc-svm.pdf ├── Unsupervised Anomaly Detection Approach for Time-Series in Multi-Domains Using Deep Reconstruction Error.pdf ├── Unsupervised Anomaly Detection Baed on Deep Autoencoding and Clustering.pdf ├── Unsupervised_Anomaly_Detection_With_LSTM_Neural_Networks.pdf └── Water_MDPI_Change Point Enhanced Anomaly Detection for IoT Time.pdf ├── Simulation Preprocessing.ipynb ├── data ├── data ├── device11_5_1_5_25.csv ├── device16_0707_0708.csv ├── device16_0713_0720.csv ├── device16_5_1_5_25.csv ├── simul_imputed.csv ├── simul_label.csv ├── simul_no_label.csv └── simul_preprocessed.csv └── figure ├── figure10_Latent Feature Space OC-SVM prediction Result Train data & Training and test data.png ├── figure11_CO2 plot over time actual outliers & reconstruction error-based model & OC-SVM-based model.png ├── figure12_Hard-voting-based CO2 plot.png ├── figure13_Soft-voting-based CO2 plot.png ├── figure1_Framework of the proposed method. Flowchart for outlier detection.png ├── figure2_IoT environmental sensor device.png ├── figure3_ Verification of sensor stability sensor installation at home & test bed location configuration.png ├── figure4_ Verification of sensor reliability actual sensor device installation & experimental environment configuration.png ├── figure5_LSTM -AE architecture.png ├── figure6_Pearson Correlation between environmental substances..png ├── figure7_Reconstruction Error Scatter Plot & Distribution Plot.png ├── figure8_Truth and Pred Error comparation of Train & Test Set.png ├── figure9_DBSCAN clustering results total data points & normal data cluster points.png ├── table1_Measurements of performance characteristics of the sensor devices.png ├── table2_Augmented Dicky–Fuller test at various levels.png ├── table3_Results of laboratory test for model validation.png └── table4_Results of the laboratory test for model validation.png /LAB TEST.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/LAB TEST.ipynb -------------------------------------------------------------------------------- /LSTM-AE Outlier Detection.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/LSTM-AE Outlier Detection.ipynb -------------------------------------------------------------------------------- /Outlier_Algorithm.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Outlier_Algorithm.ipynb -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/README.md -------------------------------------------------------------------------------- /Reference/A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data.pdf -------------------------------------------------------------------------------- /Reference/A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise.pdf -------------------------------------------------------------------------------- /Reference/Algorithm optimization and AD simulation based on extended clustering and OD.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/Algorithm optimization and AD simulation based on extended clustering and OD.pdf -------------------------------------------------------------------------------- /Reference/Deep-Compact-Clustering Based Anomaly Detection Applied to Electromechanical Industrial Systems.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/Deep-Compact-Clustering Based Anomaly Detection Applied to Electromechanical Industrial Systems.pdf -------------------------------------------------------------------------------- /Reference/Ensemble method based on ANN to estimate air pollution health risks.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/Ensemble method based on ANN to estimate air pollution health risks.pdf -------------------------------------------------------------------------------- /Reference/Forecasting and AD approaches using LSTM and LSTM AE wirh the app in supply chain management.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/Forecasting and AD approaches using LSTM and LSTM AE wirh the app in supply chain management.pdf -------------------------------------------------------------------------------- /Reference/FuseAD Unsupervised Anomaly Detection in Streaming Sensors Data by Fusing Statistical and Deep Learning Models.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/FuseAD Unsupervised Anomaly Detection in Streaming Sensors Data by Fusing Statistical and Deep Learning Models.pdf -------------------------------------------------------------------------------- /Reference/Minimum volume ellipsoid classification model for contamination event detection in water distribution systems.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/Minimum volume ellipsoid classification model for contamination event detection in water distribution systems.pdf -------------------------------------------------------------------------------- /Reference/Patient classfication as an OD problem an app of the OC-SVM.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/Patient classfication as an OD problem an app of the OC-SVM.pdf -------------------------------------------------------------------------------- /Reference/Sequential_Fault_Diagnosis_Based_on_LSTM_Neural_Network.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/Sequential_Fault_Diagnosis_Based_on_LSTM_Neural_Network.pdf -------------------------------------------------------------------------------- /Reference/Soft clustering using weighted oc-svm.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/Soft clustering using weighted oc-svm.pdf -------------------------------------------------------------------------------- /Reference/Unsupervised Anomaly Detection Approach for Time-Series in Multi-Domains Using Deep Reconstruction Error.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/Unsupervised Anomaly Detection Approach for Time-Series in Multi-Domains Using Deep Reconstruction Error.pdf -------------------------------------------------------------------------------- /Reference/Unsupervised Anomaly Detection Baed on Deep Autoencoding and Clustering.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/Unsupervised Anomaly Detection Baed on Deep Autoencoding and Clustering.pdf -------------------------------------------------------------------------------- /Reference/Unsupervised_Anomaly_Detection_With_LSTM_Neural_Networks.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/Unsupervised_Anomaly_Detection_With_LSTM_Neural_Networks.pdf -------------------------------------------------------------------------------- /Reference/Water_MDPI_Change Point Enhanced Anomaly Detection for IoT Time.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Reference/Water_MDPI_Change Point Enhanced Anomaly Detection for IoT Time.pdf -------------------------------------------------------------------------------- /Simulation Preprocessing.ipynb: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/Simulation Preprocessing.ipynb -------------------------------------------------------------------------------- /data/data: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /data/device11_5_1_5_25.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/data/device11_5_1_5_25.csv -------------------------------------------------------------------------------- /data/device16_0707_0708.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/data/device16_0707_0708.csv -------------------------------------------------------------------------------- /data/device16_0713_0720.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/data/device16_0713_0720.csv -------------------------------------------------------------------------------- /data/device16_5_1_5_25.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/data/device16_5_1_5_25.csv -------------------------------------------------------------------------------- /data/simul_imputed.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/data/simul_imputed.csv -------------------------------------------------------------------------------- /data/simul_label.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/data/simul_label.csv -------------------------------------------------------------------------------- /data/simul_no_label.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/data/simul_no_label.csv -------------------------------------------------------------------------------- /data/simul_preprocessed.csv: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/data/simul_preprocessed.csv -------------------------------------------------------------------------------- /figure/figure10_Latent Feature Space OC-SVM prediction Result Train data & Training and test data.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/figure10_Latent Feature Space OC-SVM prediction Result Train data & Training and test data.png -------------------------------------------------------------------------------- /figure/figure11_CO2 plot over time actual outliers & reconstruction error-based model & OC-SVM-based model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/figure11_CO2 plot over time actual outliers & reconstruction error-based model & OC-SVM-based model.png -------------------------------------------------------------------------------- /figure/figure12_Hard-voting-based CO2 plot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/figure12_Hard-voting-based CO2 plot.png -------------------------------------------------------------------------------- /figure/figure13_Soft-voting-based CO2 plot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/figure13_Soft-voting-based CO2 plot.png -------------------------------------------------------------------------------- /figure/figure1_Framework of the proposed method. Flowchart for outlier detection.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/figure1_Framework of the proposed method. Flowchart for outlier detection.png -------------------------------------------------------------------------------- /figure/figure2_IoT environmental sensor device.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/figure2_IoT environmental sensor device.png -------------------------------------------------------------------------------- /figure/figure3_ Verification of sensor stability sensor installation at home & test bed location configuration.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/figure3_ Verification of sensor stability sensor installation at home & test bed location configuration.png -------------------------------------------------------------------------------- /figure/figure4_ Verification of sensor reliability actual sensor device installation & experimental environment configuration.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/figure4_ Verification of sensor reliability actual sensor device installation & experimental environment configuration.png -------------------------------------------------------------------------------- /figure/figure5_LSTM -AE architecture.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/figure5_LSTM -AE architecture.png -------------------------------------------------------------------------------- /figure/figure6_Pearson Correlation between environmental substances..png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/figure6_Pearson Correlation between environmental substances..png -------------------------------------------------------------------------------- /figure/figure7_Reconstruction Error Scatter Plot & Distribution Plot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/figure7_Reconstruction Error Scatter Plot & Distribution Plot.png -------------------------------------------------------------------------------- /figure/figure8_Truth and Pred Error comparation of Train & Test Set.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/figure8_Truth and Pred Error comparation of Train & Test Set.png -------------------------------------------------------------------------------- /figure/figure9_DBSCAN clustering results total data points & normal data cluster points.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/figure9_DBSCAN clustering results total data points & normal data cluster points.png -------------------------------------------------------------------------------- /figure/table1_Measurements of performance characteristics of the sensor devices.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/table1_Measurements of performance characteristics of the sensor devices.png -------------------------------------------------------------------------------- /figure/table2_Augmented Dicky–Fuller test at various levels.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/table2_Augmented Dicky–Fuller test at various levels.png -------------------------------------------------------------------------------- /figure/table3_Results of laboratory test for model validation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/table3_Results of laboratory test for model validation.png -------------------------------------------------------------------------------- /figure/table4_Results of the laboratory test for model validation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/rootofdata/LSTM-AE_for_Unsupervised_Outlier_Detection/HEAD/figure/table4_Results of the laboratory test for model validation.png --------------------------------------------------------------------------------