├── Deep neural network aided CCA.ipynb ├── LICENSE ├── README.md ├── lib ├── generate_data.py ├── lcca_detect.py ├── model_attention.py ├── model_cnn.py ├── model_gru.py ├── model_lstm.py ├── myplot.py └── run_dcca.py └── requirements.txt /Deep neural network aided CCA.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Deep neural network aided CCA - Case Study I\n" 8 | ] 9 | }, 10 | { 11 | "cell_type": "markdown", 12 | "metadata": {}, 13 | "source": [ 14 | "## 1 Linear CCA\n", 15 | "\n", 16 | "The following command executes *lcca_detect.py* in the \"lib\" folder, performing linear CCA for process monitoring" 17 | ] 18 | }, 19 | { 20 | "cell_type": "code", 21 | "execution_count": null, 22 | "metadata": { 23 | "ExecuteTime": { 24 | "end_time": "2021-04-02T03:40:25.986212Z", 25 | "start_time": "2021-04-02T03:40:23.964455Z" 26 | } 27 | }, 28 | "outputs": [], 29 | "source": [ 30 | "%run lib/lcca_detect.py" 31 | ] 32 | }, 33 | { 34 | "cell_type": "markdown", 35 | "metadata": {}, 36 | "source": [ 37 | "## 2 Deep neural network aided CCA\n", 38 | "\n", 39 | "This is a script to run the deep neural network aided CCA model, it controls the running of the model by executing lib/run_dcca.py. Parameters include:\n", 40 | "\n", 41 | "- model: which neural network to run, optional among {\"cnn\", \"lstm\", \"gru\", \"attention\"}\n", 42 | "- train: whether to retrain the model, can be \"True\" or \"False\"; if \"False\", the model will be tested with results saved in \"result\" folder.\n", 43 | "- epoch: iteration times of training dataset\n", 44 | "- window-len: length of sliding window\n", 45 | "- output-dim: dimensions of neural network output vector\n", 46 | "- model-index: label assigned to a particular model\n", 47 | "\n", 48 | "\n", 49 | "\n", 50 | "\n", 51 | "**Note:** This notebook should only be used to adjust hyperparameters and test the designed model. If you need to modify the model structure, find the corresponding file in the \"lib\" folder." 52 | ] 53 | }, 54 | { 55 | "cell_type": "markdown", 56 | "metadata": {}, 57 | "source": [ 58 | "*The following code can be executed repeatedly, the given parameters are not optimal. Please refer to the paper for more details.*" 59 | ] 60 | }, 61 | { 62 | "cell_type": "code", 63 | "execution_count": null, 64 | "metadata": { 65 | "ExecuteTime": { 66 | "end_time": "2021-04-02T03:41:25.166646Z", 67 | "start_time": "2021-04-02T03:40:33.019704Z" 68 | } 69 | }, 70 | "outputs": [], 71 | "source": [ 72 | "%run lib/run_dcca.py --model=cnn --train=False --epoch=100 --window-len=70 --output-dim=30 --model-index=1" 73 | ] 74 | }, 75 | { 76 | "cell_type": "code", 77 | "execution_count": null, 78 | "metadata": {}, 79 | "outputs": [], 80 | "source": [] 81 | } 82 | ], 83 | "metadata": { 84 | "kernelspec": { 85 | "display_name": "Python 3", 86 | "language": "python", 87 | "name": "python3" 88 | }, 89 | "language_info": { 90 | "codemirror_mode": { 91 | "name": "ipython", 92 | "version": 3 93 | }, 94 | "file_extension": ".py", 95 | "mimetype": "text/x-python", 96 | "name": "python", 97 | "nbconvert_exporter": "python", 98 | "pygments_lexer": "ipython3", 99 | "version": "3.7.6" 100 | } 101 | }, 102 | "nbformat": 4, 103 | "nbformat_minor": 4 104 | } 105 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 CSU-IILab 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # DNN-CCA 2 | 3 | Deep neural network aided canonical correlation analysis (DNN-CCA) in Tensorflow and Keras 4 | 5 | 6 | 7 | This repository contains code for case study I in paper 8 | 9 | > Z. Chen, K. Liang, S. X. Ding, C. Yang, T. Peng and X. Yuan, "A comparative study of deep neural network aided canonical correlation analysis-based process monitoring and fault detection methods," *IEEE Transactions on Neural Networks and Learning Systems*, 2021, doi:10.1109/TNNLS.2021.3072491. 10 | 11 | 12 | 13 | ## 1 Getting Started 14 | 15 | ### 1.1 Installation 16 | 17 | Python3.6 and Tensorflow1.15 are required and should be installed on the host machine following the official guide. 18 | 19 | 1. Clone this repository 20 | 21 | ``` 22 | git clone https://github.com/CSU-IILab/DNN-CCA 23 | ``` 24 | 25 | 2. Install the required packages 26 | 27 | ``` 28 | pip install -r requirements.txt 29 | ``` 30 | 31 | 32 | 33 | ## 2 Instructions 34 | 35 | This repository provides the complete code for building deep neural network aided CCA, and a Jupyter Notebook for model training and testing. 36 | 37 | ### 2.1 Model definition 38 | 39 | - Model structures are defined in *lib/model_xxx.py*, no hyperparameter included. 40 | 41 | - All the models can be run via *lib/run_dcca.py* 42 | - The dataset is generated by simple numerical equation of random variables, consistent with the paper. 43 | 44 | ### 2.2 Deep neural network aided CCA Notebook 45 | 46 | - It's a script to run the Linear CCA and deep neural network aided CCA, it controls the running of the model by executing *lib/lcca_detect.py* and *lib/run_dcca.py*. 47 | - All the hyperparameters can be set by using this notebook, as well as train and test the model. 48 | 49 | 50 | 51 | ## 3 Citation 52 | 53 | Please cite our paper if you use this code or any of the models. 54 | 55 | ``` 56 | @article{chen2021dnncca, 57 | title={A comparative study of deep neural network aided canonical correlation analysis-based process monitoring and fault detection methods}, 58 | author={Chen, Zhiwen and Liang, Ketian and Ding, Steven X and Yang, Chao and Peng, Tao and Yuan, Xiaofeng}, 59 | journal={IEEE Transactions on Neural Networks and Learning Systems}, 60 | volume={}, 61 | number={}, 62 | pages={}, 63 | year={}, 64 | publisher={IEEE}, 65 | doi={10.1109/TNNLS.2021.3072491} 66 | } 67 | ``` 68 | 69 | 70 | 71 | ## 4 License 72 | 73 | MIT License 74 | 75 | 76 | 77 | ## 5 Related works 78 | 79 | - Canonical correlation analysis-based fault detection and process monitoring ([Matlab source code](https://ww2.mathworks.cn/matlabcentral/fileexchange/66947-canonical-correlation-analysis-based-fault-detection-and-process-monitoring-algorithm)) 80 | > Z. Chen, S. X. Ding, T. Peng, C. Yang and W. Gui, "Fault detection for non-Gaussian process using generalized canonical correlation analysis and randomized algorithms," *IEEE Transactions on Industrial Electronics*, vol. 65, no. 2, pp. 1559-1567, 2018. 81 | 82 | 83 | 84 | - Distributed CCA-based fault detection ([Matlab source code](https://www.mathworks.com/matlabcentral/fileexchange/89278-distributed-cca-based-fault-detection-method)) 85 | > Z. Chen, Y. Cao, S. X. Ding, K. Zhang, T. Koenings, T. Peng, C. Yang and W. Gui, "A Distributed Canonical Correlation Analysis-Based Fault Detection Method for Plant-Wide Process Monitoring," *IEEE Transactions on Industrial Informatics*, vol. 15, no. 5, pp. 2710-2720, 2019. 86 | 87 | 88 | 89 | -------------------------------------------------------------------------------- /lib/generate_data.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | 4 | def generate_data_ramp(fig): 5 | t1 = 1.99 * np.random.random((2000, 1)) + 0.01 6 | t2 = 1.99 * np.random.random((2000, 1)) + 0.01 7 | 8 | e1 = np.random.normal(0, 0.1, (2000, 1)) 9 | e2 = np.random.normal(0, 0.1, (2000, 1)) 10 | e3 = np.random.normal(0, 0.1, (2000, 1)) 11 | e4 = np.random.normal(0, 0.1, (2000, 1)) 12 | e5 = np.random.normal(0, 0.1, (2000, 1)) 13 | 14 | input1 = np.sin(t1) + e1 15 | input2 = t1 ** 2 - 3 * t1 + 4 + e2 16 | input3 = t1 ** 3 + 3 * t2 + e3 17 | output1 = input1 ** 2 + input1 * input2 + input1 + e4 18 | output2 = input1 * input3 + input2 + np.sin(input3) + e5 19 | 20 | output1f = output1 21 | # 产生故障 22 | fault = fig 23 | output2f = np.zeros_like(output2) 24 | output2f[0: 1000, :] = output2[0: 1000, :] 25 | output2f[1000: 2000, :] = output2[1000: 2000, :] + fault 26 | 27 | input_free = np.concatenate((input1, input2, input3), 1) 28 | output_free = np.concatenate((output1, output2), 1) 29 | input_fault = input_free 30 | output_fault = np.concatenate((output1f, output2f), 1) 31 | return input_free.T, output_free.T, input_fault.T, output_fault.T # (3 * 2000) 32 | 33 | 34 | 35 | def generate_batch(input_free, output_free, input_fault, output_fault, batch_len): 36 | series_len = input_free.shape[0] 37 | input_free_batch = [] 38 | output_free_batch = [] 39 | input_fault_batch = [] 40 | output_fault_batch = [] 41 | for i in range(series_len - batch_len + 1): 42 | input_free_batch.append(input_free[i: (i + batch_len)]) 43 | output_free_batch.append(output_free[i: (i + batch_len)]) 44 | input_fault_batch.append(input_fault[i: (i + batch_len)]) 45 | output_fault_batch.append(output_fault[i: (i + batch_len)]) 46 | return np.array(input_free_batch), np.array(output_free_batch), np.array(input_fault_batch), np.array(output_fault_batch) 47 | 48 | 49 | 50 | 51 | -------------------------------------------------------------------------------- /lib/lcca_detect.py: -------------------------------------------------------------------------------- 1 | from matplotlib import pyplot as plt 2 | import scipy.stats as ss 3 | import numpy as np 4 | from myplot import * 5 | from generate_data import generate_data_ramp 6 | 7 | 8 | def Linear_CCA(input_free, output_free, input_fault, output_fault): 9 | 10 | fault_index = 1000 11 | 12 | num_sample = input_free.shape[1] 13 | num_samplef = input_fault.shape[1] 14 | dim_input = input_free.shape[0] 15 | dim_output = output_free.shape[0] 16 | 17 | mean_in = np.mean(input_free, axis=1) 18 | mean_out = np.mean(output_free, axis=1) 19 | norm_input = (input_free.T - np.tile(mean_in, (num_sample, 1))).T 20 | norm_output = (output_free.T - np.tile(mean_out, (num_sample, 1))).T 21 | 22 | 23 | norm_inputf = (input_fault.T - np.tile(mean_in, (num_samplef, 1))).T 24 | norm_outputf = (output_fault.T - np.tile(mean_out, (num_samplef, 1))).T 25 | 26 | SigmaHat12 = np.dot(norm_input, norm_output.T) # 6*3 27 | SigmaHat11 = np.dot(norm_input, norm_input.T) 28 | SigmaHat22 = np.dot(norm_output, norm_output.T) 29 | 30 | [D1, V1] = np.linalg.eigh(SigmaHat11) 31 | [D2, V2] = np.linalg.eigh(SigmaHat22) 32 | SigmaHat11RootInv = np.dot(np.dot(V1, np.diag(D1 ** -0.5)), V1.T) # 6*6 33 | SigmaHat22RootInv = np.dot(np.dot(V2, np.diag(D2 ** -0.5)), V2.T) # 3*3 34 | 35 | Tval = np.dot(np.dot(SigmaHat11RootInv, SigmaHat12), SigmaHat22RootInv) 36 | [U, S_2, V_] = np.linalg.svd(Tval) 37 | V = V_.T 38 | S = np.zeros([dim_input, dim_output]) # 3 * 4 39 | for i in range(min(dim_input, dim_output)): 40 | S[i, i] = S_2[i] 41 | rank_S = S_2.shape[0] 42 | 43 | 44 | P = np.dot(SigmaHat11RootInv, U[:, :rank_S]) 45 | P_res = np.dot(SigmaHat11RootInv, U[:, rank_S:]) 46 | L = np.dot(SigmaHat22RootInv, V[:, :rank_S]) 47 | L_res = np.dot(SigmaHat22RootInv, V[:, rank_S:]) 48 | P_2 = np.concatenate((P, P_res), 1).T 49 | L_2 = np.concatenate((L, L_res), 1).T 50 | 51 | 52 | Inv_input = np.linalg.inv((np.identity(dim_input) - np.dot(S, S.T))/(num_samplef-1)) 53 | T2_in = [] 54 | res1 = [] 55 | Q2_1 = [] 56 | for i in range(num_sample): 57 | te1 = np.dot(P_2, norm_inputf[:, i]) - np.dot(np.dot(S, L_2), norm_outputf[:, i]) 58 | te2 = np.dot(np.dot(te1.T, Inv_input), te1) 59 | T2_in.append(te2) 60 | res1.append(te1) 61 | q2_1 = np.dot(te1.T, te1) 62 | Q2_1.append(q2_1) 63 | T2_in = np.array(T2_in) 64 | res1 = np.array(res1) 65 | Q2_1 = np.array(Q2_1) 66 | 67 | 68 | Inv_output = np.linalg.inv((np.identity(dim_output) - np.dot(S.T, S))/(num_samplef-1)) 69 | T2_in2 = [] 70 | res2 = [] 71 | Q2_2 = [] 72 | for i in range(num_sample): 73 | te1 = np.dot(L_2, norm_outputf[:, i]) - np.dot(np.dot(S.T, P_2), norm_inputf[:, i]) 74 | te2 = np.dot(np.dot(te1.T, Inv_output), te1) 75 | T2_in2.append(te2) 76 | res2.append(te1) 77 | q2_2 = np.dot(te1.T, te1) 78 | Q2_2.append(q2_2) 79 | T2_in2 = np.array(T2_in2) 80 | res2 = np.array(res2) 81 | Q2_2 = np.array(Q2_2) 82 | 83 | alpha = 0.05 84 | thre1_T = ss.chi2.isf(alpha, Inv_input.shape[1]) 85 | thre2_T = ss.chi2.isf(alpha, Inv_output.shape[1]) 86 | 87 | rate_false1, rate_mdr1 = far_mdr_compute(T2_in, thre1_T, fault_index) 88 | rate_false2, rate_mdr2 = far_mdr_compute(T2_in2, thre2_T, fault_index) 89 | fdd = fdd_compute(T2_in2, thre2_T, fault_index) 90 | 91 | t = np.linspace(0, num_sample, num_sample) 92 | smin = np.min(T2_in2) 93 | smax = np.max(T2_in2) 94 | length = len(T2_in2) 95 | plt.rcParams['font.sans-serif'] = ['Times New Roman'] 96 | plt.rcParams['font.size'] = 40 97 | plt.figure(figsize=(16,8)) 98 | plt.yscale('log') 99 | plt.plot(np.arange(length), T2_in2, linewidth=4, alpha=0.7, zorder=0) 100 | plt.vlines(x=fault_index, ymin=smin, ymax=smax, colors='r', label='fault injection', linestyles='-.', linewidth=8, zorder=2) 101 | plt.hlines(xmin=0, xmax=length, y=thre2_T, colors='orange', label='detection threshold', linestyles='--', linewidth=8, zorder=3) 102 | print(f'T2_2, thred:{thre2_T:.3f}, FAR:{rate_false2:.2%}, MDR:{rate_mdr2:.2%}, FDD:{fdd}') 103 | plt.ylim([smin, smax]) 104 | plt.legend(loc=4) 105 | plt.xlabel('Sample') 106 | plt.ylabel('Test statistic') 107 | plt.subplots_adjust(top=1, bottom=0, left=0, right=1, hspace=0, wspace=0) 108 | plt.margins(0, 0) 109 | saveRes('LinearCCA', T2_in2, thre2_T, fault_index) 110 | 111 | plt.show() 112 | 113 | 114 | 115 | 116 | if __name__ == "__main__": 117 | input_free, output_free, input_fault, output_fault = generate_data_ramp(2) 118 | input_free = input_free.T 119 | output_free = output_free.T 120 | input_fault = input_fault.T 121 | output_fault = output_fault.T 122 | Linear_CCA(input_free.T, output_free.T, input_fault.T, output_fault.T) 123 | 124 | -------------------------------------------------------------------------------- /lib/model_attention.py: -------------------------------------------------------------------------------- 1 | from keras.layers import Input, Dense, Flatten, BatchNormalization, Dropout 2 | from keras.regularizers import l2 3 | from keras import backend as K 4 | import tensorflow as tf 5 | import math 6 | import numpy as np 7 | from keras.layers.core import Permute 8 | from keras_self_attention import SeqSelfAttention 9 | 10 | 11 | def my_act(x): 12 | return (x ** 3) / 3 + x 13 | 14 | 15 | def my_init_sigmoid(shape, dtype=None): 16 | rnd = K.random_uniform( 17 | shape, 0., 1., dtype) 18 | from keras.initializers import _compute_fans 19 | fan_in, fan_out = _compute_fans(shape) 20 | return 8. * (rnd - 0.5) * math.sqrt(6) / math.sqrt(fan_in + fan_out) 21 | 22 | 23 | def my_init_others(shape, dtype=None): 24 | rnd = K.random_uniform( 25 | shape, 0., 1., dtype) 26 | from keras.initializers import _compute_fans 27 | fan_in, fan_out = _compute_fans(shape) 28 | return 2. * (rnd - 0.5) / math.sqrt(fan_in) 29 | 30 | 31 | class DeepCCA_attention(): 32 | def __init__(self, input_size1, 33 | input_size2, outdim_size, reg_par, time_step): 34 | self.input_size1 = input_size1 35 | self.input_size2 = input_size2 36 | self.outdim_size = outdim_size 37 | self.reg_par = reg_par 38 | self.time_step = time_step 39 | 40 | self.input_view1 = tf.placeholder(tf.float32, [None, time_step, input_size1]) 41 | self.input_view2 = tf.placeholder(tf.float32, [None, time_step, input_size2]) 42 | self.rate = tf.placeholder(tf.float32) 43 | 44 | self.output_view1 = self.layers1() 45 | self.output_view2 = self.layers2() 46 | 47 | self.neg_corr, self.S = self.neg_correlation(self.output_view1, self.output_view2) 48 | 49 | def PositionalEncoding(self, max_len, d_emb): 50 | pos_enc = np.array([ 51 | [pos / np.power(10000, 2 * (j // 2) / d_emb) for j in range(d_emb)] 52 | if pos != 0 else np.zeros(d_emb) for pos in range(max_len)]) 53 | pos_enc[1:, 0::2] = np.sin(pos_enc[1:, 0::2]) # dim 2i 54 | pos_enc[1:, 1::2] = np.cos(pos_enc[1:, 1::2]) # dim 2i+1 55 | return pos_enc 56 | 57 | def FFN(self, inputs): 58 | layer_ = Dense(64, activation=tf.nn.relu, kernel_initializer=my_init_sigmoid, 59 | kernel_regularizer=l2(self.reg_par))(inputs) 60 | layer__ = Dense(64, kernel_initializer=my_init_sigmoid, 61 | kernel_regularizer=l2(self.reg_par))(tf.concat([inputs, layer_], axis=2)) 62 | layer__ = Dropout(0.2)(layer__) 63 | return layer__ 64 | 65 | 66 | 67 | def layers1(self): 68 | input1 = self.input_view1 69 | input1 = input1 + self.PositionalEncoding(self.time_step, self.input_size1) 70 | layer1_1 = SeqSelfAttention(16, attention_type='additive')(input1) 71 | layer1_1 = BatchNormalization(axis=2)(layer1_1) 72 | layer1_1 = self.FFN(layer1_1) 73 | layer1_6 = Flatten()(layer1_1) 74 | out1 = Dense(self.outdim_size, activation=None, kernel_initializer=my_init_others, 75 | kernel_regularizer=l2(self.reg_par))(layer1_6) 76 | return out1 77 | 78 | def layers2(self): 79 | input2 = self.input_view2 80 | input2 = input2 + self.PositionalEncoding(self.time_step, self.input_size2) 81 | layer2_1 = SeqSelfAttention(16, attention_type='additive')(input2) 82 | layer2_1 = BatchNormalization(axis=2)(layer2_1) 83 | layer2_1 = self.FFN(layer2_1) 84 | layer2_3 = self.FFN(layer2_1) 85 | layer2_6 = Flatten()(layer2_3) 86 | out2 = Dense(self.outdim_size, activation=None, kernel_initializer=my_init_others, 87 | kernel_regularizer=l2(self.reg_par))(layer2_6) 88 | return out2 89 | 90 | def neg_correlation(self, output1, output2): 91 | r1 = 1e-3 92 | r2 = 1e-3 93 | eps = 1e-10 94 | 95 | H1 = tf.transpose(output1) 96 | H2 = tf.transpose(output2) 97 | 98 | m = tf.shape(H1)[1] 99 | 100 | H1bar = H1 - (1.0 / tf.cast(m, tf.float32)) * tf.matmul(H1, tf.ones([m, m])) 101 | H2bar = H2 - (1.0 / tf.cast(m, tf.float32)) * tf.matmul(H2, tf.ones([m, m])) 102 | 103 | SigmaHat12 = (1.0 / (tf.cast(m, tf.float32) - 1)) * tf.matmul(H1bar, tf.transpose(H2bar)) 104 | SigmaHat11 = (1.0 / (tf.cast(m, tf.float32) - 1)) * tf.matmul(H1bar, tf.transpose(H1bar)) + r1 * tf.eye( 105 | self.outdim_size) 106 | SigmaHat22 = (1.0 / (tf.cast(m, tf.float32) - 1)) * tf.matmul(H2bar, tf.transpose(H2bar)) + r2 * tf.eye( 107 | self.outdim_size) 108 | 109 | # Calculating the root inverse of covariance matrices by using eigen decomposition 110 | # [D1, V1] = tf.linalg.eigh(SigmaHat11) 111 | # [D2, V2] = tf.linalg.eigh(SigmaHat22) 112 | 113 | [D1, V1] = tf.self_adjoint_eig(SigmaHat11) 114 | [D2, V2] = tf.self_adjoint_eig(SigmaHat22) 115 | 116 | # Added to increase stability 117 | posInd1 = tf.where(tf.greater(D1, eps)) 118 | posInd1 = tf.reshape(posInd1, [-1, tf.shape(posInd1)[0]])[0] 119 | D1 = tf.gather(D1, posInd1) 120 | V1 = tf.gather(V1, posInd1) 121 | 122 | posInd2 = tf.where(tf.greater(D2, eps)) 123 | posInd2 = tf.reshape(posInd2, [-1, tf.shape(posInd2)[0]])[0] 124 | D2 = tf.gather(D2, posInd2) 125 | V2 = tf.gather(V2, posInd2) 126 | 127 | SigmaHat11RootInv = tf.matmul(tf.matmul(V1, tf.linalg.diag(D1 ** -0.5)), tf.transpose(V1)) 128 | SigmaHat22RootInv = tf.matmul(tf.matmul(V2, tf.linalg.diag(D2 ** -0.5)), tf.transpose(V2)) 129 | 130 | Tval = tf.matmul(tf.matmul(SigmaHat11RootInv, SigmaHat12), SigmaHat22RootInv) 131 | Tval.set_shape([self.outdim_size, self.outdim_size]) 132 | s = tf.svd(Tval, compute_uv=False) 133 | corr = tf.reduce_sum(s) 134 | 135 | return -corr, s 136 | -------------------------------------------------------------------------------- /lib/model_cnn.py: -------------------------------------------------------------------------------- 1 | from keras.layers import Input, Dense, Conv1D, Dropout, MaxPooling1D, Flatten 2 | from keras.regularizers import l2 3 | from keras import backend as K 4 | from keras import models 5 | import tensorflow as tf 6 | import numpy as np 7 | import math 8 | 9 | 10 | def my_act(x): 11 | return (x ** 3) / 3 + x 12 | 13 | 14 | def my_init_sigmoid(shape, dtype=None): 15 | rnd = K.random_uniform( 16 | shape, 0., 1., dtype) 17 | from keras.initializers import _compute_fans 18 | fan_in, fan_out = _compute_fans(shape) 19 | return 8. * (rnd - 0.5) * math.sqrt(6) / math.sqrt(fan_in + fan_out) 20 | 21 | 22 | def my_init_others(shape, dtype=None): 23 | rnd = K.random_uniform( 24 | shape, 0., 1., dtype) 25 | from keras.initializers import _compute_fans 26 | fan_in, fan_out = _compute_fans(shape) 27 | return 2. * (rnd - 0.5) / math.sqrt(fan_in) 28 | 29 | 30 | class DeepCCA_cnn(): 31 | def __init__(self, input_size1, 32 | input_size2, outdim_size, reg_par, input_2d): 33 | self.input_size1 = input_size1 34 | self.input_size2 = input_size2 35 | self.outdim_size = outdim_size 36 | self.reg_par = reg_par 37 | 38 | self.input_view1 = tf.placeholder(tf.float32, [None, input_2d, input_size1]) 39 | self.input_view2 = tf.placeholder(tf.float32, [None, input_2d, input_size2]) 40 | self.rate = tf.placeholder(tf.float32) 41 | 42 | self.output_view1 = self.layers1() 43 | self.output_view2 = self.layers2() 44 | 45 | self.neg_corr, self.S = self.neg_correlation(self.output_view1, self.output_view2) 46 | 47 | def layers1(self): 48 | input1 = self.input_view1 49 | layer1_1 = Conv1D(filters=64, kernel_size=3, activation='relu')(input1) 50 | layer1_3 = Conv1D(filters=64, kernel_size=3, activation='relu')(layer1_1) 51 | layer1_3 = Conv1D(filters=64, kernel_size=3, activation='relu')(layer1_3) 52 | layer1_3 = Conv1D(filters=64, kernel_size=3, activation='relu')(layer1_3) 53 | layer1_3 = Conv1D(filters=64, kernel_size=3, activation='relu')(layer1_3) 54 | layer1_5 = Flatten()(layer1_3) 55 | layer1_6 = Dense(64, activation=tf.nn.relu, kernel_initializer=my_init_sigmoid, kernel_regularizer=l2(self.reg_par))(layer1_5) 56 | out1 = Dense(self.outdim_size, activation=None, kernel_initializer=my_init_others, kernel_regularizer=l2(self.reg_par))(layer1_6) 57 | return out1 58 | 59 | def layers2(self): 60 | input2 = self.input_view2 61 | layer2_1 = Conv1D(filters=64, kernel_size=3, activation='relu')(input2) 62 | layer2_3 = Conv1D(filters=64, kernel_size=3, activation='relu')(layer2_1) 63 | layer2_3 = Conv1D(filters=64, kernel_size=3, activation='relu')(layer2_3) 64 | layer2_3 = Conv1D(filters=64, kernel_size=3, activation='relu')(layer2_3) 65 | layer2_3 = Conv1D(filters=64, kernel_size=3, activation='relu')(layer2_3) 66 | layer2_5 = Flatten()(layer2_3) 67 | layer2_6 = Dense(64, activation=tf.nn.relu, kernel_initializer=my_init_sigmoid, kernel_regularizer=l2(self.reg_par))(layer2_5) 68 | out2 = Dense(self.outdim_size, activation=None, kernel_initializer=my_init_others, kernel_regularizer=l2(self.reg_par))(layer2_6) 69 | return out2 70 | 71 | def neg_correlation(self, output1, output2): 72 | r1 = 1e-3 73 | r2 = 1e-3 74 | eps = 1e-10 75 | 76 | H1 = tf.transpose(output1) 77 | H2 = tf.transpose(output2) 78 | 79 | m = tf.shape(H1)[1] 80 | 81 | H1bar = H1 - (1.0 / tf.cast(m, tf.float32)) * tf.matmul(H1, tf.ones([m, m])) 82 | H2bar = H2 - (1.0 / tf.cast(m, tf.float32)) * tf.matmul(H2, tf.ones([m, m])) 83 | 84 | SigmaHat12 = (1.0 / (tf.cast(m, tf.float32) - 1)) * tf.matmul(H1bar, tf.transpose(H2bar)) 85 | SigmaHat11 = (1.0 / (tf.cast(m, tf.float32) - 1)) * tf.matmul(H1bar, tf.transpose(H1bar)) + r1 * tf.eye(self.outdim_size) 86 | SigmaHat22 = (1.0 / (tf.cast(m, tf.float32) - 1)) * tf.matmul(H2bar, tf.transpose(H2bar)) + r2 * tf.eye(self.outdim_size) 87 | 88 | 89 | [D1, V1] = tf.self_adjoint_eig(SigmaHat11) 90 | [D2, V2] = tf.self_adjoint_eig(SigmaHat22) 91 | 92 | 93 | posInd1 = tf.where(tf.greater(D1, eps)) 94 | posInd1 = tf.reshape(posInd1, [-1, tf.shape(posInd1)[0]])[0] 95 | D1 = tf.gather(D1, posInd1) 96 | V1 = tf.gather(V1, posInd1) 97 | 98 | posInd2 = tf.where(tf.greater(D2, eps)) 99 | posInd2 = tf.reshape(posInd2, [-1, tf.shape(posInd2)[0]])[0] 100 | D2 = tf.gather(D2, posInd2) 101 | V2 = tf.gather(V2, posInd2) 102 | 103 | SigmaHat11RootInv = tf.matmul(tf.matmul(V1, tf.linalg.diag(D1 ** -0.5)), tf.transpose(V1)) 104 | SigmaHat22RootInv = tf.matmul(tf.matmul(V2, tf.linalg.diag(D2 ** -0.5)), tf.transpose(V2)) 105 | 106 | Tval = tf.matmul(tf.matmul(SigmaHat11RootInv, SigmaHat12), SigmaHat22RootInv) 107 | Tval.set_shape([self.outdim_size, self.outdim_size]) 108 | s = tf.svd(Tval, compute_uv=False) 109 | corr = tf.reduce_sum(s) 110 | 111 | return -corr, s 112 | 113 | 114 | 115 | 116 | 117 | -------------------------------------------------------------------------------- /lib/model_gru.py: -------------------------------------------------------------------------------- 1 | from keras.layers import Input, Dense, Dropout, Flatten, GRU, BatchNormalization 2 | from keras.regularizers import l2 3 | from keras import backend as K 4 | import tensorflow as tf 5 | import math 6 | 7 | 8 | def my_act(x): 9 | return (x ** 3) / 3 + x 10 | 11 | 12 | def my_init_sigmoid(shape, dtype=None): 13 | rnd = K.random_uniform( 14 | shape, 0., 1., dtype) 15 | from keras.initializers import _compute_fans 16 | fan_in, fan_out = _compute_fans(shape) 17 | return 8. * (rnd - 0.5) * math.sqrt(6) / math.sqrt(fan_in + fan_out) 18 | 19 | 20 | def my_init_others(shape, dtype=None): 21 | rnd = K.random_uniform( 22 | shape, 0., 1., dtype) 23 | from keras.initializers import _compute_fans 24 | fan_in, fan_out = _compute_fans(shape) 25 | return 2. * (rnd - 0.5) / math.sqrt(fan_in) 26 | 27 | 28 | class DeepCCA_gru(): 29 | def __init__(self, input_size1, 30 | input_size2, outdim_size, reg_par, time_step): 31 | self.input_size1 = input_size1 32 | self.input_size2 = input_size2 33 | self.outdim_size = outdim_size 34 | self.reg_par = reg_par 35 | self.time_step = time_step 36 | 37 | self.input_view1 = tf.placeholder(tf.float32, [None, time_step, input_size1]) 38 | self.input_view2 = tf.placeholder(tf.float32, [None, time_step, input_size2]) 39 | self.rate = tf.placeholder(tf.float32) 40 | 41 | self.output_view1 = self.layers1() 42 | self.output_view2 = self.layers2() 43 | 44 | self.neg_corr, self.S = self.neg_correlation(self.output_view1, self.output_view2) 45 | 46 | 47 | def layers1(self): 48 | input1 = self.input_view1 49 | layer1_1 = GRU(64, return_sequences=True)(input1) 50 | layer1_3 = GRU(64, return_sequences=True)(layer1_1) 51 | layer1_3 = GRU(64, return_sequences=True)(layer1_3) 52 | layer1_3 = GRU(64, return_sequences=True)(layer1_3) 53 | 54 | layer1_5 = Flatten()(layer1_3) 55 | layer1_6 = Dense(64, activation=tf.nn.relu, kernel_initializer=my_init_sigmoid, 56 | kernel_regularizer=l2(self.reg_par))(layer1_5) 57 | out1 = Dense(self.outdim_size, activation=None, kernel_initializer=my_init_others, 58 | kernel_regularizer=l2(self.reg_par))(layer1_6) 59 | return out1 60 | 61 | def layers2(self): 62 | input2 = self.input_view2 63 | layer2_1 = GRU(64, return_sequences=True)(input2) 64 | layer2_3 = GRU(64, return_sequences=True)(layer2_1) 65 | layer2_3 = GRU(64, return_sequences=True)(layer2_3) 66 | layer2_3 = GRU(64, return_sequences=True)(layer2_3) 67 | layer2_5 = Flatten()(layer2_3) 68 | layer2_6 = Dense(64, activation=tf.nn.relu, kernel_initializer=my_init_sigmoid, 69 | kernel_regularizer=l2(self.reg_par))(layer2_5) 70 | out2 = Dense(self.outdim_size, activation=None, kernel_initializer=my_init_others, 71 | kernel_regularizer=l2(self.reg_par))(layer2_6) 72 | return out2 73 | 74 | def neg_correlation(self, output1, output2): 75 | r1 = 1e-3 76 | r2 = 1e-3 77 | eps = 1e-10 78 | 79 | H1 = tf.transpose(output1) 80 | H2 = tf.transpose(output2) 81 | 82 | m = tf.shape(H1)[1] 83 | 84 | H1bar = H1 - (1.0 / tf.cast(m, tf.float32)) * tf.matmul(H1, tf.ones([m, m])) 85 | H2bar = H2 - (1.0 / tf.cast(m, tf.float32)) * tf.matmul(H2, tf.ones([m, m])) 86 | 87 | SigmaHat12 = (1.0 / (tf.cast(m, tf.float32) - 1)) * tf.matmul(H1bar, tf.transpose(H2bar)) 88 | SigmaHat11 = (1.0 / (tf.cast(m, tf.float32) - 1)) * tf.matmul(H1bar, tf.transpose(H1bar)) + r1 * tf.eye( 89 | self.outdim_size) 90 | SigmaHat22 = (1.0 / (tf.cast(m, tf.float32) - 1)) * tf.matmul(H2bar, tf.transpose(H2bar)) + r2 * tf.eye( 91 | self.outdim_size) 92 | 93 | # Calculating the root inverse of covariance matrices by using eigen decomposition 94 | # [D1, V1] = tf.linalg.eigh(SigmaHat11) 95 | # [D2, V2] = tf.linalg.eigh(SigmaHat22) 96 | 97 | [D1, V1] = tf.self_adjoint_eig(SigmaHat11) 98 | [D2, V2] = tf.self_adjoint_eig(SigmaHat22) 99 | 100 | # Added to increase stability 101 | posInd1 = tf.where(tf.greater(D1, eps)) 102 | posInd1 = tf.reshape(posInd1, [-1, tf.shape(posInd1)[0]])[0] 103 | D1 = tf.gather(D1, posInd1) 104 | V1 = tf.gather(V1, posInd1) 105 | 106 | posInd2 = tf.where(tf.greater(D2, eps)) 107 | posInd2 = tf.reshape(posInd2, [-1, tf.shape(posInd2)[0]])[0] 108 | D2 = tf.gather(D2, posInd2) 109 | V2 = tf.gather(V2, posInd2) 110 | 111 | SigmaHat11RootInv = tf.matmul(tf.matmul(V1, tf.linalg.diag(D1 ** -0.5)), tf.transpose(V1)) 112 | SigmaHat22RootInv = tf.matmul(tf.matmul(V2, tf.linalg.diag(D2 ** -0.5)), tf.transpose(V2)) 113 | 114 | Tval = tf.matmul(tf.matmul(SigmaHat11RootInv, SigmaHat12), SigmaHat22RootInv) 115 | Tval.set_shape([self.outdim_size, self.outdim_size]) 116 | s = tf.svd(Tval, compute_uv=False) 117 | corr = tf.reduce_sum(s) 118 | 119 | return -corr, s 120 | -------------------------------------------------------------------------------- /lib/model_lstm.py: -------------------------------------------------------------------------------- 1 | from keras.layers import Input, LSTM, Dense, Dropout, Flatten, BatchNormalization 2 | from keras.regularizers import l2 3 | from keras import backend as K 4 | import tensorflow as tf 5 | import math 6 | 7 | 8 | def my_act(x): 9 | return (x ** 3) / 3 + x 10 | 11 | 12 | def my_init_sigmoid(shape, dtype=None): 13 | rnd = K.random_uniform( 14 | shape, 0., 1., dtype) 15 | from keras.initializers import _compute_fans 16 | fan_in, fan_out = _compute_fans(shape) 17 | return 8. * (rnd - 0.5) * math.sqrt(6) / math.sqrt(fan_in + fan_out) 18 | 19 | 20 | def my_init_others(shape, dtype=None): 21 | rnd = K.random_uniform( 22 | shape, 0., 1., dtype) 23 | from keras.initializers import _compute_fans 24 | fan_in, fan_out = _compute_fans(shape) 25 | return 2. * (rnd - 0.5) / math.sqrt(fan_in) 26 | 27 | 28 | class DeepCCA_lstm(): 29 | def __init__(self, input_size1, 30 | input_size2, outdim_size, reg_par, time_step): 31 | self.input_size1 = input_size1 32 | self.input_size2 = input_size2 33 | self.outdim_size = outdim_size 34 | self.reg_par = reg_par 35 | self.time_step = time_step 36 | 37 | self.input_view1 = tf.placeholder(tf.float32, [None, time_step, input_size1]) 38 | self.input_view2 = tf.placeholder(tf.float32, [None, time_step, input_size2]) 39 | self.rate = tf.placeholder(tf.float32) 40 | 41 | self.output_view1 = self.layers1() 42 | self.output_view2 = self.layers2() 43 | 44 | self.neg_corr, self.S = self.neg_correlation(self.output_view1, self.output_view2) 45 | 46 | 47 | 48 | 49 | def layers1(self): 50 | input1 = self.input_view1 51 | layer1_1 = LSTM(64, return_sequences=True)(input1) 52 | layer1_3 = LSTM(64, return_sequences=True)(layer1_1) 53 | layer1_5 = Flatten()(layer1_3) 54 | layer1_6 = Dense(64, activation=tf.nn.relu, kernel_initializer=my_init_sigmoid, 55 | kernel_regularizer=l2(self.reg_par))(layer1_5) 56 | 57 | out1 = Dense(self.outdim_size, activation=None, kernel_initializer=my_init_others, 58 | kernel_regularizer=l2(self.reg_par))(layer1_6) 59 | return out1 60 | 61 | def layers2(self): 62 | input2 = self.input_view2 63 | layer2_1 = LSTM(64, return_sequences=True)(input2) 64 | layer2_3 = LSTM(64, return_sequences=True)(layer2_1) 65 | layer2_5 = Flatten()(layer2_3) 66 | print(layer2_5) 67 | layer2_6 = Dense(64, activation=tf.nn.relu, kernel_initializer=my_init_sigmoid, 68 | kernel_regularizer=l2(self.reg_par))(layer2_5) 69 | out2 = Dense(self.outdim_size, activation=None, kernel_initializer=my_init_others, 70 | kernel_regularizer=l2(self.reg_par))(layer2_6) 71 | return out2 72 | 73 | def neg_correlation(self, output1, output2): 74 | r1 = 1e-3 75 | r2 = 1e-3 76 | eps = 1e-10 77 | 78 | H1 = tf.transpose(output1) 79 | H2 = tf.transpose(output2) 80 | 81 | m = tf.shape(H1)[1] 82 | 83 | H1bar = H1 - (1.0 / tf.cast(m, tf.float32)) * tf.matmul(H1, tf.ones([m, m])) 84 | H2bar = H2 - (1.0 / tf.cast(m, tf.float32)) * tf.matmul(H2, tf.ones([m, m])) 85 | 86 | SigmaHat12 = (1.0 / (tf.cast(m, tf.float32) - 1)) * tf.matmul(H1bar, tf.transpose(H2bar)) 87 | SigmaHat11 = (1.0 / (tf.cast(m, tf.float32) - 1)) * tf.matmul(H1bar, tf.transpose(H1bar)) + r1 * tf.eye( 88 | self.outdim_size) 89 | SigmaHat22 = (1.0 / (tf.cast(m, tf.float32) - 1)) * tf.matmul(H2bar, tf.transpose(H2bar)) + r2 * tf.eye( 90 | self.outdim_size) 91 | 92 | # Calculating the root inverse of covariance matrices by using eigen decomposition 93 | # [D1, V1] = tf.linalg.eigh(SigmaHat11) 94 | # [D2, V2] = tf.linalg.eigh(SigmaHat22) 95 | 96 | [D1, V1] = tf.self_adjoint_eig(SigmaHat11) 97 | [D2, V2] = tf.self_adjoint_eig(SigmaHat22) 98 | 99 | # Added to increase stability 100 | posInd1 = tf.where(tf.greater(D1, eps)) 101 | posInd1 = tf.reshape(posInd1, [-1, tf.shape(posInd1)[0]])[0] 102 | D1 = tf.gather(D1, posInd1) 103 | V1 = tf.gather(V1, posInd1) 104 | 105 | posInd2 = tf.where(tf.greater(D2, eps)) 106 | posInd2 = tf.reshape(posInd2, [-1, tf.shape(posInd2)[0]])[0] 107 | D2 = tf.gather(D2, posInd2) 108 | V2 = tf.gather(V2, posInd2) 109 | 110 | SigmaHat11RootInv = tf.matmul(tf.matmul(V1, tf.linalg.diag(D1 ** -0.5)), tf.transpose(V1)) 111 | SigmaHat22RootInv = tf.matmul(tf.matmul(V2, tf.linalg.diag(D2 ** -0.5)), tf.transpose(V2)) 112 | 113 | Tval = tf.matmul(tf.matmul(SigmaHat11RootInv, SigmaHat12), SigmaHat22RootInv) 114 | Tval.set_shape([self.outdim_size, self.outdim_size]) 115 | s = tf.svd(Tval, compute_uv=False) 116 | corr = tf.reduce_sum(s) 117 | 118 | return -corr, s 119 | 120 | -------------------------------------------------------------------------------- /lib/myplot.py: -------------------------------------------------------------------------------- 1 | import os 2 | import numpy as np 3 | import matplotlib.pyplot as plt 4 | from sklearn.neighbors import KernelDensity 5 | from scipy.io import savemat 6 | 7 | 8 | def generate_batch(input_free, output_free, input_fault, output_fault, batch_len): 9 | series_len = input_free.shape[0] 10 | input_free_batch = [] 11 | output_free_batch = [] 12 | input_fault_batch = [] 13 | output_fault_batch = [] 14 | for i in range(series_len - batch_len + 1): 15 | input_free_batch.append(input_free[i: (i + batch_len)]) 16 | output_free_batch.append(output_free[i: (i + batch_len)]) 17 | input_fault_batch.append(input_fault[i: (i + batch_len)]) 18 | output_fault_batch.append(output_fault[i: (i + batch_len)]) 19 | return np.array(input_free_batch), np.array(output_free_batch), np.array(input_fault_batch), np.array(output_fault_batch) 20 | 21 | 22 | def far_mdr_compute(sta, thre, index_thre): 23 | rate_false = (sta[:index_thre] > thre).tolist().count(True) / index_thre 24 | rate_mdr = (sta[index_thre:] < thre).tolist().count(True) / (sta.shape[0] - index_thre) 25 | return rate_false, rate_mdr 26 | 27 | 28 | 29 | def saveRes(name, statistics, threshold, fault_index): 30 | if not os.path.exists('./result/fig/'): 31 | os.makedirs('./result/fig/') 32 | savemat('./result/'+name+'.mat', 33 | {'statistics':statistics, 34 | 'threshold':threshold, 35 | 'fault_index':fault_index, 36 | }) 37 | plt.savefig('./result/fig/'+name+'.png', dpi=300) 38 | 39 | 40 | def fdd_compute(data, threshold, fault_index): 41 | L = len(data) 42 | fd_index = L 43 | for i in range(L-1,fault_index,-1): 44 | if(data[i] > threshold): 45 | fd_index = i-fault_index 46 | return fd_index 47 | 48 | def threshold_compute(statistics): 49 | from sklearn.neighbors import KernelDensity 50 | from sklearn.preprocessing import MinMaxScaler 51 | 52 | staMin = np.min(statistics.ravel()) 53 | staMax = np.max(statistics.ravel()) 54 | staBand = (staMax - staMin) * 0.5 55 | ts = np.linspace(staMin-staBand, staMax+staBand, 1000).reshape(-1,1) 56 | kde = np.exp(KernelDensity().fit(statistics.reshape(-1,1)).score_samples(ts)) 57 | distribution = MinMaxScaler(feature_range=(0,1)).fit_transform(np.cumsum(kde/1000).reshape(-1,1)) 58 | 59 | return ts[1000-len(distribution[distribution>0.95])] 60 | 61 | 62 | def plot_ts(test_corr_up, S_up, hidden_view1_up, hidden_view2_up, test_corr_dw, S_dw, 63 | hidden_view1_dw, hidden_view2_dw, outdim_size, pt_num, num_sample, saveName='None'): 64 | print(f'DCCA for free data: {-test_corr_up}, DCCA for fault data: {-test_corr_dw}') 65 | 66 | # up_1 67 | S2_up = np.diag(S_up) 68 | res1_up = [] 69 | T2_up = [] 70 | Q2_up = [] 71 | Inv_output_up = np.linalg.inv((np.identity(outdim_size) - np.dot(S2_up, S2_up.T)) / (num_sample - 1)) 72 | for i in range(pt_num): 73 | # compute the residual 74 | te1_up = hidden_view1_up[i] - np.dot(S2_up, hidden_view2_up[i]) 75 | res1_up.append(te1_up) 76 | # compute the T2 77 | te2_up = np.dot(np.dot(te1_up.T, Inv_output_up), te1_up) 78 | T2_up.append(te2_up) 79 | # compute the Q2 80 | q2_up = np.dot(te1_up.T, te1_up) 81 | Q2_up.append(q2_up) 82 | res1_up = np.array(res1_up) 83 | T2_up = np.array(T2_up) 84 | Q2_up = np.array(Q2_up) 85 | threshold1_T = get_thred(T2_up) 86 | # threshold1_Q = get_thred(Q2_up) 87 | 88 | # up_2 89 | S2_up = np.diag(S_up) 90 | res2_up = [] 91 | T2_2_up = [] 92 | Q2_2_up = [] 93 | Inv_output_up = np.linalg.inv((np.identity(outdim_size) - np.dot(S2_up, S2_up.T)) / (num_sample - 1)) 94 | for i in range(pt_num): 95 | # compute the residual 96 | te1_up = hidden_view2_up[i] - np.dot(S2_up, hidden_view1_up[i]) 97 | res2_up.append(te1_up) 98 | # compute the T2 99 | te2_up = np.dot(np.dot(te1_up.T, Inv_output_up), te1_up) 100 | T2_2_up.append(te2_up) 101 | # compute the Q2 102 | q2_up = np.dot(te1_up.T, te1_up) 103 | Q2_2_up.append(q2_up) 104 | res2_up = np.array(res2_up) 105 | T2_2_up = np.array(T2_2_up) 106 | Q2_2_up = np.array(Q2_2_up) 107 | # threshold2_T = get_thred(T2_2_up) 108 | # threshold2_Q = get_thred(Q2_2_up) 109 | 110 | # dw_1 111 | S2_dw = np.diag(S_dw) 112 | res1_dw = [] 113 | T2_dw = [] 114 | Q2_dw = [] 115 | Inv_output_dw = np.linalg.inv((np.identity(outdim_size) - np.dot(S2_dw, S2_dw.T)) / (num_sample - 1)) 116 | for i in range(pt_num): 117 | # compute the residual 118 | te1_dw = hidden_view1_dw[i] - np.dot(S2_dw, hidden_view2_dw[i]) 119 | res1_dw.append(te1_dw) 120 | # compute the T2 121 | te2_dw = np.dot(np.dot(te1_dw.T, Inv_output_dw), te1_dw) 122 | T2_dw.append(te2_dw) 123 | # compute the Q2 124 | q2_dw = np.dot(te1_dw.T, te1_dw) 125 | Q2_dw.append(q2_dw) 126 | res1_dw = np.array(res1_dw) 127 | T2_dw = np.array(T2_dw) 128 | Q2_dw = np.array(Q2_dw) 129 | 130 | # dw_2 131 | S2_dw = np.diag(S_dw) 132 | res2_dw = [] 133 | T2_2_dw = [] 134 | Q2_2_dw = [] 135 | Inv_output_dw = np.linalg.inv((np.identity(outdim_size) - np.dot(S2_dw, S2_dw.T)) / (num_sample - 1)) 136 | for i in range(pt_num): 137 | # compute the residual 138 | te1_dw = hidden_view2_dw[i] - np.dot(S2_dw, hidden_view1_dw[i]) 139 | res2_dw.append(te1_dw) 140 | # compute the T2 141 | te2_dw = np.dot(np.dot(te1_dw.T, Inv_output_dw), te1_dw) 142 | T2_2_dw.append(te2_dw) 143 | # compute the Q2 144 | q2_dw = np.dot(te1_dw.T, te1_dw) 145 | Q2_2_dw.append(q2_dw) 146 | res2_dw = np.array(res2_dw) 147 | T2_2_dw = np.array(T2_2_dw) 148 | Q2_2_dw = np.array(Q2_2_dw) 149 | 150 | 151 | fault_index = num_sample - 1000 + 1 152 | threshold2_T = get_thred(T2_2_dw[0:fault_index]) 153 | # print('fault index =', fault_index) 154 | # print('threshold =', threshold2_T) 155 | 156 | 157 | rate_false_T1, rate_mdr_T1 = far_mdr_compute(T2_dw, threshold1_T, fault_index) 158 | rate_false_T2, rate_mdr_T2 = far_mdr_compute(T2_2_dw, threshold2_T, fault_index) 159 | 160 | t = np.linspace(0, pt_num, pt_num) 161 | fdd = fdd_compute(T2_2_dw, threshold2_T, fault_index) 162 | smin = np.min(T2_2_dw) 163 | smax = np.max(T2_2_dw) 164 | length = len(T2_2_dw) 165 | 166 | 167 | plt.rcParams['font.sans-serif'] = ['Times New Roman'] 168 | plt.rcParams['font.size'] = 40 169 | plt.figure(figsize=(16,8)) 170 | plt.yscale('log') 171 | plt.plot(np.arange(length), T2_2_dw[:], linewidth=4, alpha=0.7, zorder=0) 172 | plt.vlines(x=fault_index, ymin=smin, ymax=smax, colors='r', label='fault injection', linestyles='-.', linewidth=8, zorder=2) 173 | plt.hlines(xmin=0, xmax=length, y=threshold2_T, colors='orange', label='detection threshold', linestyles='--', linewidth=8, zorder=3) 174 | print(f'T2_2, thred:{threshold2_T:.3f}, FAR:{rate_false_T2:.2%}, MDR:{rate_mdr_T2:.2%}, FDD:{fdd}') 175 | plt.ylim([smin, smax]) 176 | plt.legend(loc=4) 177 | plt.xlabel('Sample') 178 | plt.ylabel('Test statistic') 179 | plt.subplots_adjust(top=1, bottom=0, left=0, right=1, hspace=0, wspace=0) 180 | plt.margins(0, 0) 181 | if not (saveName == 'None'): 182 | saveRes(saveName, T2_2_dw, threshold2_T, fault_index) 183 | 184 | plt.show() 185 | 186 | 187 | 188 | def get_thred(T2_statistic, alpha=0.95): 189 | # T2_statistic (1, None) 190 | data = T2_statistic.reshape(-1, 1) 191 | Min = np.min(data) 192 | Max = np.max(data) 193 | Range = Max - Min 194 | x_start = Min - Range 195 | x_end = Max + Range 196 | nums = 2 ** 12 197 | dx = (x_end-x_start)/(nums-1) 198 | data_plot = np.linspace(x_start, x_end, nums)[:, np.newaxis] 199 | 200 | # choose the best bandwidth 201 | data_median = np.median(data) 202 | new_median = np.median(np.abs(data - data_median)) / 0.6745 203 | bw = new_median * ((4 / (3 * data.shape[0])) ** 0.2) 204 | 205 | kde = KernelDensity(kernel='gaussian', bandwidth=bw).fit(data) 206 | log_dens = kde.score_samples(data_plot) 207 | pdf = np.exp(log_dens).reshape(-1, 1) 208 | 209 | CDF = 0 210 | index = 0 211 | while CDF <= alpha: 212 | CDF += pdf[index]*dx 213 | index += 1 214 | return np.squeeze(data_plot[index]) -------------------------------------------------------------------------------- /lib/run_dcca.py: -------------------------------------------------------------------------------- 1 | import os 2 | import tensorflow as tf 3 | import time 4 | import random 5 | from keras import backend as K 6 | from myplot import * 7 | from generate_data import generate_data_ramp 8 | import shutil 9 | from scipy.io import savemat 10 | 11 | import argparse 12 | parser = argparse.ArgumentParser(description='train model') 13 | parser.add_argument('--model', type=str, default = 'cnn') 14 | parser.add_argument('--train', type=str, default = 'False') 15 | parser.add_argument('--epoch', type=int, default = 100) 16 | parser.add_argument('--window-len', type=int, default = 70) 17 | parser.add_argument('--output-dim', type=int, default = 30) 18 | parser.add_argument('--model-index', type=int, default = 1) 19 | args = parser.parse_args() 20 | model = args.model 21 | retrain = True if args.train == 'True' else False 22 | n_epochs = args.epoch 23 | batch_len = args.window_len 24 | outdim_size = args.output_dim 25 | model_index = args.model_index 26 | 27 | if model == 'cnn': 28 | from model_cnn import DeepCCA_cnn as DeepCCA_model 29 | elif model == 'lstm': 30 | from model_lstm import DeepCCA_lstm as DeepCCA_model 31 | elif model == 'gru': 32 | from model_gru import DeepCCA_gru as DeepCCA_model 33 | elif model == 'attention': 34 | from model_attention import DeepCCA_attention as DeepCCA_model 35 | 36 | 37 | saveName = model 38 | model_save_rootpath = './model' 39 | dir_path = os.path.join(model_save_rootpath, f'{model}_{batch_len}_{outdim_size}_{model_index}') 40 | if (retrain == True): 41 | try: 42 | shutil.rmtree(dir_path) 43 | except: 44 | pass 45 | model_save_path = os.path.join(dir_path, "best.ckpt") 46 | if not os.path.exists(dir_path): 47 | os.makedirs(dir_path) 48 | 49 | log_save_rootpath = './model/' 50 | dir_path = os.path.join(log_save_rootpath, f'log_{model}_{batch_len}_{outdim_size}_{model_index}') 51 | if (retrain == True): 52 | try: 53 | shutil.rmtree(dir_path) 54 | except: 55 | pass 56 | log_save_path = dir_path 57 | if not os.path.exists(dir_path): 58 | os.makedirs(dir_path) 59 | 60 | 61 | learning_rate = 0.01 62 | momentum = 0.5 63 | batch_size = 256 64 | num_sample = 2000 - batch_len + 1 65 | input_size1 = 3 66 | input_size2 = 2 67 | input_2d = batch_len 68 | reg_par = 2 69 | use_all_singular_values = True 70 | 71 | input_free, output_free, input_fault, output_fault = generate_data_ramp(2) 72 | input_free = input_free.T 73 | output_free = output_free.T 74 | input_fault = input_fault.T 75 | output_fault = output_fault.T 76 | 77 | 78 | num_sam = input_free.shape[0] 79 | num_samf = input_fault.shape[0] 80 | mean_in = np.mean(input_free, axis=0) 81 | mean_out = np.mean(output_free, axis=0) 82 | input_free = input_free - np.tile(mean_in, (num_sam, 1)) 83 | output_free = output_free - np.tile(mean_out, (num_sam, 1)) 84 | input_fault = input_fault - np.tile(mean_in, (num_samf, 1)) 85 | output_fault = output_fault - np.tile(mean_out, (num_samf, 1)) 86 | 87 | 88 | input_free_batch, output_free_batch, input_fault_batch, output_fault_batch\ 89 | = generate_batch(input_free, output_free, input_fault, output_fault, batch_len) 90 | 91 | tf.reset_default_graph() 92 | rand_num = random.randint(0, 1000) 93 | K.set_learning_phase(1) 94 | dcca_model = DeepCCA_model(input_size1, input_size2, outdim_size, reg_par, input_2d) 95 | 96 | input_view1 = dcca_model.input_view1 97 | input_view2 = dcca_model.input_view2 98 | hidden_view1 = dcca_model.output_view1 99 | hidden_view2 = dcca_model.output_view2 100 | neg_corr = dcca_model.neg_corr 101 | S = dcca_model.S 102 | rate_ = dcca_model.rate 103 | 104 | 105 | tf.summary.scalar('correlation', -neg_corr) 106 | merged = tf.summary.merge_all() 107 | 108 | # gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=0.333) 109 | train_op = tf.train.AdamOptimizer().minimize(neg_corr, 110 | var_list=tf.trainable_variables()) 111 | iterations = 0 112 | 113 | 114 | saver = tf.train.Saver() 115 | loss = 0 116 | last_loss = 0 117 | loss_his = [] 118 | corr1 = [] 119 | corr2 = [] 120 | config = tf.ConfigProto() 121 | config.gpu_options.per_process_gpu_memory_fraction = 0.3 122 | with tf.Session(config=config) as sess: 123 | 124 | if not os.path.exists(model_save_path + ".index"): 125 | print("Learning begins...") 126 | sess.run(tf.global_variables_initializer()) 127 | train_writer = tf.summary.FileWriter(log_save_path, sess.graph) 128 | 129 | 130 | start_time = time.time() 131 | for epoch in range(n_epochs): 132 | for batch in range(int(input_free_batch.shape[0]/batch_size)): 133 | 134 | X1_batch = input_free_batch[(batch * batch_size): ((batch + 1) * batch_size)] 135 | X2_batch = output_free_batch[(batch * batch_size): ((batch + 1) * batch_size)] 136 | 137 | _, neg_corr_val, summary = sess.run([train_op, neg_corr, merged], 138 | feed_dict={input_view1: X1_batch, input_view2: X2_batch 139 | }) 140 | train_writer.add_summary(summary, batch + epoch * batch_size) 141 | 142 | loss = neg_corr_val 143 | loss_his.append(loss) 144 | if last_loss > loss: 145 | last_loss = loss 146 | if not os.path.exists(model_save_path): 147 | os.mkdir(model_save_path) 148 | print("create the directory: %s" % model_save_path) 149 | saver.save(sess, model_save_path) 150 | 151 | 152 | if iterations % 100 == 0: 153 | time_node = time.time() 154 | print("epoch: " + str(epoch) + " correlation for train:", -neg_corr_val/outdim_size, " time costs " + str(time_node - start_time) + "s") 155 | 156 | test_corr, S_, hidden_view1_, hidden_view2_ = sess.run([neg_corr, S, hidden_view1, hidden_view2], feed_dict= 157 | {input_view1: input_free_batch, input_view2: output_free_batch}) 158 | print("correlation of normal data: ", -test_corr/outdim_size) 159 | corr1.append(-test_corr) 160 | 161 | test_corr3, S_3, hidden_view1_3, hidden_view2_3 = sess.run([neg_corr, S, hidden_view1, hidden_view2], feed_dict= 162 | {input_view1: input_fault_batch, input_view2: output_fault_batch}) 163 | print("correlation of faulty data: ", -test_corr3/outdim_size) 164 | corr2.append(-test_corr3) 165 | print("\n") 166 | 167 | iterations += 1 168 | # savemat('corr.mat', {'corr1':corr1, 'corr2':corr2}) 169 | 170 | 171 | 172 | else: 173 | K.set_learning_phase(0) 174 | pt_num = 2000 - batch_len + 1 175 | saver.restore(sess, model_save_path) 176 | test_corr_up, S_up, hidden_view1_up, hidden_view2_up = sess.run([neg_corr, S, hidden_view1, hidden_view2], feed_dict= 177 | {input_view1: input_free_batch, input_view2: output_free_batch, rate_: 1}) 178 | test_corr_dw, S_dw, hidden_view1_dw, hidden_view2_dw = sess.run([neg_corr, S, hidden_view1, hidden_view2], feed_dict= 179 | {input_view1: input_fault_batch, input_view2: output_fault_batch, rate_: 1}) 180 | plot_ts(test_corr_up, S_up, hidden_view1_up, hidden_view2_up, test_corr_dw, S_dw, 181 | hidden_view1_dw, hidden_view2_dw, outdim_size, pt_num, num_sample, saveName) 182 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | scipy 2 | numpy 3 | matplotlib 4 | jupyter 5 | scikit-learn 6 | tensorflow==1.15.0 7 | Keras==2.2.4 8 | keras_self_attention --------------------------------------------------------------------------------