├── README.md ├── Requirements └── SCNN-Tensorflow ├── lane-detection-model ├── .gitignore ├── GUI │ ├── 111111.png │ ├── 1212.png │ ├── 3333.png │ ├── A cup of coffee.ico │ ├── Balance.ico │ └── algorithm.ico ├── Settings │ ├── config.JSON │ ├── config_2.JSON │ └── mask.png ├── config │ ├── .gitignore │ └── global_config.py ├── data_provider │ ├── .gitignore │ ├── lanenet_data_processor.py │ └── lanenet_data_processor_test.py ├── demo_file │ ├── test_img.txt │ ├── train_gt.txt │ └── val_gt.txt ├── encoder_decoder_model │ ├── .gitignore │ ├── cnn_basenet.py │ └── vgg_encoder.py ├── lanenet_model │ ├── .gitignore │ └── lanenet_merge_model.py ├── prob2lines │ ├── __pycache__ │ │ └── getLane.cpython-37.pyc │ └── calTotal.py ├── requirements.txt └── tools │ ├── __pycache__ │ ├── advanced_lane.cpython-37.pyc │ ├── cnn_basenet.cpython-37.pyc │ ├── getLane.cpython-37.pyc │ ├── global_config.cpython-37.pyc │ ├── global_lane.cpython-37.pyc │ ├── lanenet_data_processor_test.cpython-37.pyc │ ├── lanenet_merge_model.cpython-37.pyc │ ├── main.cpython-37.pyc │ ├── main1.cpython-37.pyc │ ├── perspective_regionofint_main.cpython-37.pyc │ ├── sliding_main.cpython-37.pyc │ ├── test_lanenet.cpython-37.pyc │ ├── thresholding_main.cpython-37.pyc │ └── vgg_encoder.cpython-37.pyc │ ├── advanced_lane.py │ ├── calibration_main.py │ ├── cnn_basenet.py │ ├── computer_vison_1.py │ ├── getLane.py │ ├── global_config.py │ ├── global_lane.py │ ├── lanenet_data_processor_test.py │ ├── lanenet_merge_model.py │ ├── main.py │ ├── main1.py │ ├── perspective_regionofint_main.py │ ├── sliding_main.py │ ├── test_lanenet.py │ ├── thresholding_main.py │ ├── train_lanenet.py │ └── vgg_encoder.py └── lane_evaluation ├── .gitignore ├── Makefile ├── Run.sh ├── calTotal.m ├── evl ├── evaluate.exe ├── results.txt └── run.bat ├── include ├── counter.hpp ├── hungarianGraph.hpp ├── lane_compare.hpp └── spline.hpp ├── run2.sh └── src ├── counter.cpp ├── evaluate.cpp ├── lane_compare.cpp └── spline.cpp /README.md: -------------------------------------------------------------------------------- 1 |

2 | Self Driving Car 3 |

4 | 5 | 6 | 7 | # Lane Line Detection using Image Processing vs Deep Learning 8 | Lane line detection technique is used in many self-driving autonomous vehicles as well as line-following robots,our project is to develop a software to Detect lane lines in a variety of conditions, including changing road surfaces, curved roads, and variable lighting. An image processing based detection and a deep learning AI detection were to be implemented, evaluated and compared in the scope of this project. 9 | [Paper](https://www.researchgate.net/publication/344123734_Lane_Line_Detection_using_Image_Processing_Deep_Learning_comparative_study) for this project. 10 | # Structure 11 | An easy to use sandbox for lane detection in the proposed methods, which is an easy-to-use graphical user interface (GUI) that contains two sections: 12 | 1. Image processing section. 13 | 2. Deep Learning section. 14 | # Requirements 15 | 1) Hardware: For testing, GPU with 3G memory suffices. 16 | 2) Anaconda 17 | 3) Spyder IDE 18 | 4) Pycharm 19 | 5) Opencv (for tools/lane_evaluation), version 3.2.0.7 (2.4.x should also work). 20 | 6) Matplotlib 21 | 7) Numpy==1.13.1 22 | 8) easydict==1.6 23 | 9) matplotlib==2.0.2 24 | 10) glog==0.3.1 25 | 11) scikit_learn==0.19.1 26 | # Befor we start let's get to know the Data set 27 |

28 | 29 | 30 | [Xingang](https://github.com/XingangPan) is the researcher who built this data set. 31 | CULane is a large scale challenging dataset for academic research on traffic lane detection. It is collected by cameras mounted on six different vehicles driven by different drivers in Beijing. More than 55 hours of videos were collected and 133,235 frames were extracted. Data examples are shown above. We have divided the dataset into 88880 for training set, 9675 for validation set, and 34680 for test set. The test set is divided into normal and 8 challenging categories, which correspond to the 9 examples above. 32 |
33 | For each frame, we manually annotate the traffic lanes with cubic splines. For cases where lane markings are occluded by vehicles or are unseen, we still annotate the lanes according to the context, as shown in (2). We also hope that algorithms could distinguish barriers on the road, like the one in (1). Thus the lanes on the other side of the barrier are not annotated. In this dataset we focus our attention on the detection of four lane markings, which are paid most attention to in real applications. Other lane markings are not annotated. 34 | The whole dataset is available at [CULane](https://xingangpan.github.io/projects/CULane.html). 35 | # To run this project: 36 | Open computer_vison_1.py in spyder or pycharm and run it. 37 | ## First: Image processing section 38 | ### This section consists of the following stages: 39 | 1. Camera Calibration & Distortion Correction Stage. 40 | 2. Creating a Thresholded binary image using color transforms and gradients. 41 | 3. Apply a perspective transform to rectify binary image ("birds-eye view"). 42 | 4. Detect lane pixels and fit a polynomial expression to find the lane boundary. 43 | 5. Determine the curvature of the lane and vehicle position with respect to center. 44 | 6. Overlay the detected lane boundaries back onto the original image. 45 | ### Let us now discuss each of these section stages in detail. 46 | # Step 1: Camera Calibration Stage & Distortion Correction Stage 47 | Real cameras use curved lenses to form an image, and light rays often bend a little too much or too little at the edges of these lenses. This creates an effect that distorts the edges of images, so that lines or objects appear more or less curved than they actually are. This is called radial distortion, which is the most common type of distortion. 48 | There are three coefficients needed to correct radial distortion: k1, k2, and k3. To correct the appearance of radially distorted points in an image, one can use a correction formula mentioned below. 49 | ![Camera Calibration](https://user-images.githubusercontent.com/66889657/92038347-ecedf700-ed7b-11ea-9b0c-49ba9b6ae774.JPG) 50 | In the following equations, (x,y) is a point in a distorted image, to undistort these points, OpenCV calculates r, which is the known distance between a point in an undistorted (corrected) image (xcorrected ,ycorrected) and the center of the image distortion, which is often the center of that image (xc ,yc ).
51 | This center point (xc ,yc) is sometimes referred to as the distortion center, these points are pictured above. 52 |
53 | The ``` do_calibration() ``` function performs the following operations: 54 |
55 | 1. Read chessboad images and convert to gray scale 56 | 2. Find the chessboard corners. 57 | 3. Perform the ``` cv2.calibrateCamera() ``` to compute the distortion co-efficients and camera matrix that we need to transform the 3d object points to 2d image points. 58 | 4. Store the calibration values to use it later. 59 | ## Now Distortion Correction: 60 | Using the distortion co-efficients and camera matrix obtained from the camera calibration stage we undistort the images using the ```cv2.undistort``` function. 61 | ### A sample chessboard image and corresponding undistorted image is shown below: 62 | ![Distortion Correction](https://user-images.githubusercontent.com/66889657/92039345-95e92180-ed7d-11ea-8797-2b400a364774.JPG) 63 | ###### By perfoming the distortion correction we see that the chessboard lines appear to be parallel compared to the original raw image.
64 | ### Another sample image and corresponding undistorted image is shown below: 65 | ![Distortion Correction ](https://user-images.githubusercontent.com/66889657/92040062-ced5c600-ed7e-11ea-8fb7-e3c5553bbdc9.JPG) 66 | # Note : we didn't apply this step because the data set image were already un-distorted. 67 | # Step 2: Create a Thresholded binary image using color transforms and gradients 68 | In the thresholding binary image stage, multiple transformations were applied and later combined to get the best binary image for lane detection, various thresholding operations used are explained below.
69 | 1. Saturation thresholding : The images are transformed to HSL color space to obtain the saturation values, the yellow color lanes are best detected in the saturation color space. 70 | 2. Histogram equalized thresholding: The images are transformed to gray scale and histogram is equalized using the opencv library functions, the white color lanes are best detected using this operation. 71 | 3. Gradient Thresholding : The Sobel operator is applied to get the gradients in the x and y direction which are also used to get the magnitude and direction thresholded images. 72 | 4. Region of Interest : Region of Interest operation is a process of masking unwanted portions of an image, thus keeping only essential part of the image Combining the above thresholding step to get the best binary image for lane detection: To obtain the clear distinctive lanes in the binary image, threshold parameters for the above operation have to be fine tuned. 73 | ![Thresholded binary image](https://user-images.githubusercontent.com/66889657/92040862-cf229100-ed7f-11ea-9fc2-4fd397f19def.JPG) 74 | 75 | ### in the previous figure: 76 | + image 1: Image sample. 77 | + image 2: Sobel-X. 78 | + image 3: Sobel-Y. 79 | + image 4: magintude. 80 | + image 5: direction. 81 | + image 6: Using mask. 82 | + image 7: The final output of this stage. 83 | ## The thresholds are as follows: 84 |

85 | 86 | 87 | # Step 3: Perspective transformation 88 | A perspective transform maps the points in a given image to different, desired, image points with a new perspective. For this project, perspective transformation is applied to get a bird’s-eye view like transform, that let’s us view a lane from above; this will be useful for calculating the lane curvature later on.
89 | The source and destination points for the perspective transformation were taken through experimentation on data samples. 90 | ## The following images shows the perspective transformation from source to destination. 91 | + Image 1 - Image with parallel lanes. 92 | --- 93 | ![perspective transformation](https://user-images.githubusercontent.com/66889657/92042269-85877580-ed82-11ea-99fe-5d24d63f51f7.JPG) 94 | --- 95 | + Image 2 - Image with lanes, here lanes appear parallel in normal view, but on perspective transformation we can clearly see that the lanes are curved. 96 | --- 97 | ![perspective transformation](https://user-images.githubusercontent.com/66889657/92042358-b667aa80-ed82-11ea-8d58-4d29895affc6.JPG) 98 | --- 99 | # Step 4: Detect lane pixels and fit to find the lane boundary. 100 | After applying calibration, thresholding, and a perspective transform to a road image, we have a binary image where the lane lines stand out clearly as shown above. Next a polynomial curve is fitted to the lanes.
This is defined in the function ```for_sliding_window()``` included in the file ```sliding_main.py``` 101 | For this, we first take a histogram along all the columns in the lower half of the image.
102 | The histogram plot is shown below. 103 |

104 | 105 |

106 |
107 | With this histogram,we are adding up the pixel values along each column in the image.
In the thresholded binary image, pixels are either 0 or 1, so the two most prominent peaks in this histogram will be good indicators of the x-position of the base of the lane lines.
108 | we use that as a starting point to search for the lines.
109 | From that point, we use a sliding window, placed around the line centers, to find and follow the lines up to the top of the frame. The sliding window technique can be shown as in the below image: 110 |

111 | 112 | 113 | In the above image the sliding windows are shown in green, left lanes are red colored, right lanes are blue colored and the polynomial fits are yellow lines. 114 | # Step 5: Determine the curvature of the lane and vehicle position with respect to center.
115 | The curvature of the lanes f(y) are calculated by using the formulae R(curve) 116 |

117 | 118 | 119 | The vehicle position is calculated as the difference between the image center and the lane center. 120 |
121 | # Step 6: Overlay the detected lane boundaries back onto the original image.
122 | Now we overlay the detected lanes on the original images using inverse perspective transform. 123 | 124 | # Second Deep Learning Section 125 | ## Not: This Section contains Tensorflow implementation of "Spatial As Deep: Spatial CNN for Traffic Scene Understanding" (SCNN-Tensorflow), you can find The original repo for the Section [here](https://github.com/cardwing/Codes-for-Lane-Detection) 126 | ### This section consists of Simple steps: 127 | 1. After we install the necessary packages in the [Requirements](https://github.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/blob/master/Requirements) 128 | 2. Download VGG-16:
129 | Download the vgg.npy [here](https://github.com/machrisaa/tensorflow-vgg) and put it in SCNN-Tensorflow/lane-detection-model/data. 130 | 3. Pre-trained model for testing:
131 | Download it from [here](https://drive.google.com/file/d/1-E0Bws7-v35vOVfqEXDTJdfovUTQ2sf5/view). 132 | # Results of applying the previous two sections and comparing between them 133 |
134 | Finally you've to run the file computer_vison_1.py and the following GUI will appear: 135 |

136 | 137 |

138 | In order to apply Image processing you must press the Computer vision button and the following GUI will appear: 139 |

140 | 141 |

142 | Now to apply image processing technology, First choose an image: 143 |

144 | 145 |

146 | after that we can see the result as the following image shows: 147 |

148 | 149 |

150 | Now to apply the SCNN we go back to the first GUI and press the Deep learning button, then the following GUI appears: 151 |

152 | 153 |

154 | After that you must choose an image to apply SCNN and the result'll be as the following image: 155 |

156 | 157 |

158 | If you want to visually compare the two technologies, we can go back to the original interface and choose the compare button, and the following interface appears: 159 |

160 | 161 |

162 | Then you must choose an image from the data set to apply the two technologies and the result'll be as the following image: 163 |

164 | 165 |

166 | Now we will compare the SCNN output and the Image processing output using some scenarios as the following figure: 167 |

168 | 169 |

170 | From the previous output images we can say the following: 171 | we could say that both SCNN and image processing gave us an acceptable output on the previous scenarios, but notice that the SCNN has outperformed Image processing. 172 | 173 | # Discussion 174 | The results clearly show that SCNN network has outperformed image processing significantly in most of the scenarios we have in the dataset, in the scenario of images that contain a deviation we noticed that the SCNN gave much better results than Image processing on the same images, the same is true for scenarios that contain shadows, night, Since the image processing suffers from several problems, the most important of which are shadows and the sunshine, which the image processing technique couldn’t overcome, but the SCNN did. 175 | # Shortcomings in Image Processing 176 | we have observed some problems with the current Image Processing Implementation:
177 | + when the lane is covered by some shadow, our code originally failed to detect it, we managed to fix this issue by applying the HSL color filtering as another pre-processing step, but it didn't give good results as we expected. 178 | + Straight lines do not work when there are curves on the road, so to solve this problem we used Bird Eye Transform and it was difficult to get the correct parameters, we are not sure that we got the best settings. 179 | + For the sake of comparison, we wanted to use Intersection-Over-Union (Iou) and F1 Score , we succeeded applying them on SCNN, but in terms of image processing, we tried in various ways, but we could not achieve Iou and F-Score on this section, so we decided the comparison should be visual. 180 | 181 | # Conclusion 182 | This was an exciting and challenging first project that got us to understand a lot more about color spaces, image processing and revise some linear algebra too, for the image processing section on one hand and using SCNN for lane detection on the other hand.
183 | We are looking forward to even more challenging projects that stretch our knowledge in all these fields and more, and help us in understanding how a self-driving car is built! 184 | # Credits & References: 185 | [Codes-for-Lane-Detection](https://github.com/cardwing/Codes-for-Lane-Detection) 186 |
187 |
188 | [SCNN](https://github.com/XingangPan/SCNN) 189 |
190 |
191 | [CULane Dataset](https://xingangpan.github.io/projects/CULane.html) 192 |
193 |
194 | [Mastering OpenCV 4 - Third Edition](https://www.packtpub.com/free-ebooks/application-development/mastering-opencv-4-third-edition/9781789533576) 195 |
196 |
197 | [Learning OpenCV 3](https://www.oreilly.com/library/view/learning-opencv-3/9781491937983) 198 |
199 |
200 | [Udacity Advance Lane-Detection of the Road in Autonomous Driving](https://mc.ai/udacity-advance-lane-detection-of-the-road-in-autonomous-driving) 201 |
202 |
203 | -------------------------------------------------------------------------------- /Requirements: -------------------------------------------------------------------------------- 1 | Requirements: 2 | 1) Hardware: For testing, GPU with 3G memory suffices. For training, we recommend 4xGPU with 12G memory. 3 | 2) Anaconda 4 | 3) Spyder IDE 5 | 4) Pycharm 6 | 5) Opencv (for tools/lane_evaluation), version 3.2.0.7 (2.4.x should also work). 7 | 6) Matplotlib 8 | 7) Numpy==1.13.1 9 | 8) easydict==1.6 10 | 9) matplotlib==2.0.2 11 | 10) glog==0.3.1 12 | 11) scikit_learn==0.19.1 13 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/.gitignore: -------------------------------------------------------------------------------- 1 | zip/ 2 | Visual_output/ 3 | train_images/ 4 | test_images/ 5 | model_culane-71-3/ 6 | data/ 7 | camera_cal/ -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/GUI/111111.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/GUI/111111.png -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/GUI/1212.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/GUI/1212.png -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/GUI/3333.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/GUI/3333.png -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/GUI/A cup of coffee.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/GUI/A cup of coffee.ico -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/GUI/Balance.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/GUI/Balance.ico -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/GUI/algorithm.ico: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/GUI/algorithm.ico -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/Settings/config.JSON: -------------------------------------------------------------------------------- 1 | { 2 | "grad_thx_min":60, 3 | "grad_thx_max":255, 4 | "grad_thy_min":180, 5 | "grad_thy_max":255, 6 | "mag_th_min":52, 7 | "mag_th_max":130, 8 | "dir_th_min":0.7, 9 | "dir_th_max":1.3, 10 | "s_threshold_min":220, 11 | "s_threshold_max":255, 12 | "l_threshold_min":220, 13 | "l_threshold_max":233, 14 | "k_size":15, 15 | "adp_thr":240 16 | } -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/Settings/config_2.JSON: -------------------------------------------------------------------------------- 1 | { 2 | "grad_thx_min":160, 3 | "grad_thx_max":255, 4 | "grad_thy_min":120, 5 | "grad_thy_max":255, 6 | "mag_th_min":70, 7 | "mag_th_max":200, 8 | "dir_th_min":0.87, 9 | "dir_th_max":1.3, 10 | "s_threshold_min":180, 11 | "s_threshold_max":255, 12 | "l_threshold_min":160, 13 | "l_threshold_max":233, 14 | "k_size":15, 15 | "adp_thr":224 16 | } -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/Settings/mask.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/Settings/mask.png -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/config/.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/config/global_config.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Time : 18-1-31 上午11:21 4 | # @Author : Luo Yao 5 | # @Site : http://icode.baidu.com/repos/baidu/personal-code/Luoyao 6 | # @File : global_config.py 7 | # @IDE: PyCharm Community Edition 8 | """ 9 | 设置全局变量 10 | """ 11 | from easydict import EasyDict as edict 12 | 13 | __C = edict() 14 | # Consumers can get config by: from config import cfg 15 | 16 | cfg = __C 17 | 18 | # Train options 19 | __C.TRAIN = edict() 20 | 21 | # Set the shadownet training epochs 22 | __C.TRAIN.EPOCHS = 90100 # 200010 23 | # Set the display step 24 | __C.TRAIN.DISPLAY_STEP = 1 25 | # Set the test display step during training process 26 | __C.TRAIN.TEST_DISPLAY_STEP = 1000 27 | # Set the momentum parameter of the optimizer 28 | __C.TRAIN.MOMENTUM = 0.9 29 | # Set the initial learning rate 30 | __C.TRAIN.LEARNING_RATE = 0.01 # 0.0005 31 | # Set the GPU resource used during training process 32 | __C.TRAIN.GPU_MEMORY_FRACTION = 0.85 33 | # Set the GPU allow growth parameter during tensorflow training process 34 | __C.TRAIN.TF_ALLOW_GROWTH = True 35 | # Set the shadownet training batch size 36 | __C.TRAIN.BATCH_SIZE = 8 # 4 37 | # Set the shadownet validation batch size 38 | __C.TRAIN.VAL_BATCH_SIZE = 8 # 4 39 | # Set the learning rate decay steps 40 | __C.TRAIN.LR_DECAY_STEPS = 210000 41 | # Set the learning rate decay rate 42 | __C.TRAIN.LR_DECAY_RATE = 0.1 43 | # Set the class numbers 44 | __C.TRAIN.CLASSES_NUMS = 2 45 | # Set the image height 46 | __C.TRAIN.IMG_HEIGHT = 288 # 256 47 | # Set the image width 48 | __C.TRAIN.IMG_WIDTH = 800 # 512 49 | # Set GPU number 50 | __C.TRAIN.GPU_NUM = 4 # 8 51 | # Set CPU thread number 52 | __C.TRAIN.CPU_NUM = 4 # 53 | 54 | # Test options 55 | __C.TEST = edict() 56 | 57 | # Set the GPU resource used during testing process 58 | __C.TEST.GPU_MEMORY_FRACTION = 0.8 59 | # Set the GPU allow growth parameter during tensorflow testing process 60 | __C.TEST.TF_ALLOW_GROWTH = True 61 | # Set the test batch size 62 | __C.TEST.BATCH_SIZE = 8 63 | # Set the test CPU thread number 64 | __C.TEST.CPU_NUM = 8 65 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/data_provider/.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/data_provider/lanenet_data_processor.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Time : 18-5-11 下午4:58 4 | # @Author : Luo Yao 5 | # @Site : http://icode.baidu.com/repos/baidu/personal-code/Luoyao 6 | # @File : lanenet_data_processor.py 7 | # @IDE: PyCharm Community Edition 8 | """ 9 | 实现LaneNet的数据解析类 10 | """ 11 | import tensorflow as tf 12 | 13 | from config import global_config 14 | 15 | CFG = global_config.cfg 16 | VGG_MEAN = [123.68, 116.779, 103.939] 17 | 18 | 19 | class DataSet(object): 20 | """ 21 | 实现数据集类 22 | """ 23 | 24 | def __init__(self, dataset_info_file): 25 | """ 26 | :param dataset_info_file: 27 | """ 28 | self._len = 0 29 | self.dataset_info_file = dataset_info_file 30 | self._img, self._label_instance, self._label_existence = self._init_dataset() 31 | 32 | def __len__(self): 33 | return self._len 34 | 35 | @staticmethod 36 | def process_img(img_queue): 37 | img_raw = tf.read_file(img_queue) 38 | img_decoded = tf.image.decode_jpeg(img_raw, channels=3) 39 | img_resized = tf.image.resize_images(img_decoded, [CFG.TRAIN.IMG_HEIGHT, CFG.TRAIN.IMG_WIDTH], 40 | method=tf.image.ResizeMethod.BICUBIC) 41 | img_casted = tf.cast(img_resized, tf.float32) 42 | return tf.subtract(img_casted, VGG_MEAN) 43 | 44 | @staticmethod 45 | def process_label_instance(label_instance_queue): 46 | label_instance_raw = tf.read_file(label_instance_queue) 47 | label_instance_decoded = tf.image.decode_png(label_instance_raw, channels=1) 48 | label_instance_resized = tf.image.resize_images(label_instance_decoded, 49 | [CFG.TRAIN.IMG_HEIGHT, CFG.TRAIN.IMG_WIDTH], 50 | method=tf.image.ResizeMethod.NEAREST_NEIGHBOR) 51 | label_instance_resized = tf.reshape(label_instance_resized, [CFG.TRAIN.IMG_HEIGHT, CFG.TRAIN.IMG_WIDTH]) 52 | return tf.cast(label_instance_resized, tf.int32) 53 | 54 | @staticmethod 55 | def process_label_existence(label_existence_queue): 56 | return tf.cast(label_existence_queue, tf.float32) 57 | 58 | def _init_dataset(self): 59 | """ 60 | :return: 61 | """ 62 | if not tf.gfile.Exists(self.dataset_info_file): 63 | raise ValueError('Failed to find file: ' + self.dataset_info_file) 64 | 65 | img_list = [] 66 | label_instance_list = [] 67 | label_existence_list = [] 68 | 69 | with open(self.dataset_info_file, 'r') as file: 70 | for _info in file: 71 | info_tmp = _info.strip(' ').split() 72 | 73 | img_list.append(info_tmp[0][1:]) 74 | label_instance_list.append(info_tmp[1][1:]) 75 | label_existence_list.append([int(info_tmp[2]), int(info_tmp[3]), int(info_tmp[4]), int(info_tmp[5])]) 76 | 77 | self._len = len(img_list) 78 | # img_queue = tf.train.string_input_producer(img_list) 79 | # label_instance_queue = tf.train.string_input_producer(label_instance_list) 80 | with tf.name_scope('data_augmentation'): 81 | image_tensor = tf.convert_to_tensor(img_list) 82 | label_instance_tensor = tf.convert_to_tensor(label_instance_list) 83 | label_existence_tensor = tf.convert_to_tensor(label_existence_list) 84 | input_queue = tf.train.slice_input_producer([image_tensor, label_instance_tensor, label_existence_tensor]) 85 | img = self.process_img(input_queue[0]) 86 | label_instance = self.process_label_instance(input_queue[1]) 87 | label_existence = self.process_label_existence(input_queue[2]) 88 | 89 | return img, label_instance, label_existence 90 | 91 | def next_batch(self, batch_size): 92 | return tf.train.batch([self._img, self._label_instance, self._label_existence], batch_size=batch_size, 93 | num_threads=CFG.TRAIN.CPU_NUM) 94 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/data_provider/lanenet_data_processor_test.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Time : 18-5-11 下午4:58 4 | # @Author : Luo Yao 5 | # @Site : http://icode.baidu.com/repos/baidu/personal-code/Luoyao 6 | # @File : lanenet_data_processor.py 7 | # @IDE: PyCharm Community Edition 8 | """ 9 | 实现LaneNet的数据解析类 10 | """ 11 | import tensorflow as tf 12 | 13 | from config import global_config 14 | 15 | CFG = global_config.cfg 16 | VGG_MEAN = [123.68, 116.779, 103.939] 17 | 18 | 19 | class DataSet(object): 20 | """ 21 | 实现数据集类 22 | """ 23 | 24 | def __init__(self, dataset_info_file, batch_size): 25 | """ 26 | 27 | :param dataset_info_file: 28 | """ 29 | self._dataset_info_file = dataset_info_file 30 | self._batch_size = batch_size 31 | self._img_list = self._init_dataset() 32 | self._next_batch_loop_count = 0 33 | 34 | def __len__(self): 35 | return self._len 36 | 37 | def _init_dataset(self): 38 | """ 39 | :return: 40 | """ 41 | img_list = [] 42 | 43 | if not tf.gfile.Exists(self._dataset_info_file): 44 | raise ValueError('Failed to find file: ' + self._dataset_info_file) 45 | 46 | with open(self._dataset_info_file, 'r') as file: 47 | for _info in file: 48 | info_tmp = _info.strip(' ').split() 49 | img_list.append(info_tmp[0][:]) # changed this for windows path 50 | 51 | self._len = len(img_list) 52 | 53 | return img_list 54 | 55 | @staticmethod 56 | def process_img(img_path): 57 | img_raw = tf.read_file(img_path) 58 | img_decoded = tf.image.decode_jpeg(img_raw, channels=3) 59 | img_resized = tf.image.resize_images(img_decoded, [CFG.TRAIN.IMG_HEIGHT, CFG.TRAIN.IMG_WIDTH], 60 | method=tf.image.ResizeMethod.BICUBIC) 61 | img_casted = tf.cast(img_resized, tf.float32) 62 | return tf.subtract(img_casted, VGG_MEAN) 63 | 64 | def next_batch(self): 65 | """ 66 | :return: 67 | """ 68 | print(self._batch_size) 69 | idx_start = self._batch_size * self._next_batch_loop_count 70 | idx_end = self._batch_size * self._next_batch_loop_count + self._batch_size 71 | 72 | if idx_end > len(self): 73 | idx_end = len(self) 74 | 75 | img_list = self._img_list[idx_start:idx_end] 76 | self._next_batch_loop_count += 1 77 | return img_list 78 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/demo_file/train_gt.txt: -------------------------------------------------------------------------------- 1 | /driver_23_30frame/05151649_0422.MP4/00000.jpg /laneseg_label_w16/driver_23_30frame/05151649_0422.MP4/00000.png 1 1 1 1 2 | /driver_23_30frame/05151649_0422.MP4/00030.jpg /laneseg_label_w16/driver_23_30frame/05151649_0422.MP4/00030.png 1 1 1 1 3 | /driver_23_30frame/05151649_0422.MP4/00060.jpg /laneseg_label_w16/driver_23_30frame/05151649_0422.MP4/00060.png 1 1 1 1 4 | /driver_23_30frame/05151649_0422.MP4/00090.jpg /laneseg_label_w16/driver_23_30frame/05151649_0422.MP4/00090.png 1 1 1 1 5 | /driver_23_30frame/05151649_0422.MP4/00120.jpg /laneseg_label_w16/driver_23_30frame/05151649_0422.MP4/00120.png 1 1 1 1 6 | /driver_23_30frame/05151649_0422.MP4/00150.jpg /laneseg_label_w16/driver_23_30frame/05151649_0422.MP4/00150.png 1 1 1 1 7 | /driver_23_30frame/05151649_0422.MP4/00180.jpg /laneseg_label_w16/driver_23_30frame/05151649_0422.MP4/00180.png 1 1 1 1 8 | /driver_23_30frame/05151649_0422.MP4/00210.jpg /laneseg_label_w16/driver_23_30frame/05151649_0422.MP4/00210.png 1 1 1 1 9 | /driver_23_30frame/05151649_0422.MP4/00240.jpg /laneseg_label_w16/driver_23_30frame/05151649_0422.MP4/00240.png 1 1 1 1 10 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/demo_file/val_gt.txt: -------------------------------------------------------------------------------- 1 | /driver_23_30frame/05171102_0766.MP4/00020.jpg /laneseg_label_w16/driver_23_30frame/05171102_0766.MP4/00020.png 1 1 1 0 2 | /driver_23_30frame/05171102_0766.MP4/00050.jpg /laneseg_label_w16/driver_23_30frame/05171102_0766.MP4/00050.png 1 1 1 0 3 | /driver_23_30frame/05171102_0766.MP4/00080.jpg /laneseg_label_w16/driver_23_30frame/05171102_0766.MP4/00080.png 1 1 1 0 4 | /driver_23_30frame/05171102_0766.MP4/00110.jpg /laneseg_label_w16/driver_23_30frame/05171102_0766.MP4/00110.png 1 1 1 0 5 | /driver_23_30frame/05171102_0766.MP4/00140.jpg /laneseg_label_w16/driver_23_30frame/05171102_0766.MP4/00140.png 1 1 1 0 6 | /driver_23_30frame/05171102_0766.MP4/00170.jpg /laneseg_label_w16/driver_23_30frame/05171102_0766.MP4/00170.png 1 1 1 0 7 | /driver_23_30frame/05171102_0766.MP4/00200.jpg /laneseg_label_w16/driver_23_30frame/05171102_0766.MP4/00200.png 1 1 1 0 8 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/encoder_decoder_model/.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/encoder_decoder_model/vgg_encoder.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Time : 18-1-29 下午2:04 4 | # @Author : Luo Yao 5 | # @Site : http://icode.baidu.com/repos/baidu/personal-code/Luoyao 6 | # @File : dilation_encoder.py 7 | # @IDE: PyCharm Community Edition 8 | """ 9 | 实现一个基于VGG16的特征编码类 10 | """ 11 | from collections import OrderedDict 12 | import math 13 | 14 | import tensorflow as tf 15 | 16 | from encoder_decoder_model import cnn_basenet 17 | from config import global_config 18 | 19 | CFG = global_config.cfg 20 | 21 | 22 | class VGG16Encoder(cnn_basenet.CNNBaseModel): 23 | """ 24 | 实现了一个基于vgg16的特征编码类 25 | """ 26 | 27 | def __init__(self, phase): 28 | """ 29 | 30 | :param phase: 31 | """ 32 | super(VGG16Encoder, self).__init__() 33 | self._train_phase = tf.constant('train', dtype=tf.string) 34 | self._test_phase = tf.constant('test', dtype=tf.string) 35 | self._phase = phase 36 | self._is_training = self._init_phase() 37 | 38 | def _init_phase(self): 39 | """ 40 | 41 | :return: 42 | """ 43 | return tf.equal(self._phase, self._train_phase) 44 | 45 | def _conv_stage(self, input_tensor, k_size, out_dims, name, 46 | stride=1, pad='SAME'): 47 | """ 48 | 将卷积和激活封装在一起 49 | :param input_tensor: 50 | :param k_size: 51 | :param out_dims: 52 | :param name: 53 | :param stride: 54 | :param pad: 55 | :return: 56 | """ 57 | with tf.variable_scope(name): 58 | conv = self.conv2d(inputdata=input_tensor, out_channel=out_dims, 59 | kernel_size=k_size, stride=stride, 60 | use_bias=False, padding=pad, name='conv') 61 | 62 | bn = self.layerbn(inputdata=conv, is_training=self._is_training, name='bn') 63 | 64 | relu = self.relu(inputdata=bn, name='relu') 65 | 66 | return relu 67 | 68 | def _conv_dilated_stage(self, input_tensor, k_size, out_dims, name, 69 | dilation=1, pad='SAME'): 70 | """ 71 | 将卷积和激活封装在一起 72 | :param input_tensor: 73 | :param k_size: 74 | :param out_dims: 75 | :param name: 76 | :param stride: 77 | :param pad: 78 | :return: 79 | """ 80 | with tf.variable_scope(name): 81 | conv = self.dilation_conv(input_tensor=input_tensor, out_dims=out_dims, 82 | k_size=k_size, rate=dilation, 83 | use_bias=False, padding=pad, name='conv') 84 | 85 | bn = self.layerbn(inputdata=conv, is_training=self._is_training, name='bn') 86 | 87 | relu = self.relu(inputdata=bn, name='relu') 88 | 89 | return relu 90 | 91 | def _fc_stage(self, input_tensor, out_dims, name, use_bias=False): 92 | """ 93 | 94 | :param input_tensor: 95 | :param out_dims: 96 | :param name: 97 | :param use_bias: 98 | :return: 99 | """ 100 | with tf.variable_scope(name): 101 | fc = self.fullyconnect(inputdata=input_tensor, out_dim=out_dims, use_bias=use_bias, 102 | name='fc') 103 | 104 | bn = self.layerbn(inputdata=fc, is_training=self._is_training, name='bn') 105 | 106 | relu = self.relu(inputdata=bn, name='relu') 107 | 108 | return relu 109 | 110 | def encode(self, input_tensor, name): 111 | """ 112 | 根据vgg16框架对输入的tensor进行编码 113 | :param input_tensor: 114 | :param name: 115 | :return: 输出vgg16编码特征 116 | """ 117 | ret = OrderedDict() 118 | with tf.variable_scope(name): 119 | # conv stage 1_1 120 | conv_1_1 = self._conv_stage(input_tensor=input_tensor, k_size=3, 121 | out_dims=64, name='conv1_1') 122 | 123 | # conv stage 1_2 124 | conv_1_2 = self._conv_stage(input_tensor=conv_1_1, k_size=3, 125 | out_dims=64, name='conv1_2') 126 | 127 | # pool stage 1 128 | pool1 = self.maxpooling(inputdata=conv_1_2, kernel_size=2, 129 | stride=2, name='pool1') 130 | 131 | # conv stage 2_1 132 | conv_2_1 = self._conv_stage(input_tensor=pool1, k_size=3, 133 | out_dims=128, name='conv2_1') 134 | 135 | # conv stage 2_2 136 | conv_2_2 = self._conv_stage(input_tensor=conv_2_1, k_size=3, 137 | out_dims=128, name='conv2_2') 138 | 139 | # pool stage 2 140 | pool2 = self.maxpooling(inputdata=conv_2_2, kernel_size=2, 141 | stride=2, name='pool2') 142 | 143 | # conv stage 3_1 144 | conv_3_1 = self._conv_stage(input_tensor=pool2, k_size=3, 145 | out_dims=256, name='conv3_1') 146 | 147 | # conv_stage 3_2 148 | conv_3_2 = self._conv_stage(input_tensor=conv_3_1, k_size=3, 149 | out_dims=256, name='conv3_2') 150 | 151 | # conv stage 3_3 152 | conv_3_3 = self._conv_stage(input_tensor=conv_3_2, k_size=3, 153 | out_dims=256, name='conv3_3') 154 | 155 | # pool stage 3 156 | pool3 = self.maxpooling(inputdata=conv_3_3, kernel_size=2, 157 | stride=2, name='pool3') 158 | 159 | # conv stage 4_1 160 | conv_4_1 = self._conv_stage(input_tensor=pool3, k_size=3, 161 | out_dims=512, name='conv4_1') 162 | 163 | # conv stage 4_2 164 | conv_4_2 = self._conv_stage(input_tensor=conv_4_1, k_size=3, 165 | out_dims=512, name='conv4_2') 166 | 167 | # conv stage 4_3 168 | conv_4_3 = self._conv_stage(input_tensor=conv_4_2, k_size=3, 169 | out_dims=512, name='conv4_3') 170 | 171 | ### add dilated convolution ### 172 | 173 | # conv stage 5_1 174 | conv_5_1 = self._conv_dilated_stage(input_tensor=conv_4_3, k_size=3, 175 | out_dims=512, dilation=2, name='conv5_1') 176 | 177 | # conv stage 5_2 178 | conv_5_2 = self._conv_dilated_stage(input_tensor=conv_5_1, k_size=3, 179 | out_dims=512, dilation=2, name='conv5_2') 180 | 181 | # conv stage 5_3 182 | conv_5_3 = self._conv_dilated_stage(input_tensor=conv_5_2, k_size=3, 183 | out_dims=512, dilation=2, name='conv5_3') 184 | 185 | # added part of SCNN # 186 | 187 | # conv stage 5_4 188 | conv_5_4 = self._conv_dilated_stage(input_tensor=conv_5_3, k_size=3, 189 | out_dims=1024, dilation=4, name='conv5_4') 190 | 191 | # conv stage 5_5 192 | conv_5_5 = self._conv_stage(input_tensor=conv_5_4, k_size=1, 193 | out_dims=128, name='conv5_5') # 8 x 36 x 100 x 128 194 | 195 | # add message passing # 196 | 197 | # top to down # 198 | 199 | feature_list_old = [] 200 | feature_list_new = [] 201 | for cnt in range(conv_5_5.get_shape().as_list()[1]): 202 | feature_list_old.append(tf.expand_dims(conv_5_5[:, cnt, :, :], axis=1)) 203 | feature_list_new.append(tf.expand_dims(conv_5_5[:, 0, :, :], axis=1)) 204 | 205 | w1 = tf.get_variable('W1', [1, 9, 128, 128], 206 | initializer=tf.random_normal_initializer(0, math.sqrt(2.0 / (9 * 128 * 128 * 5)))) 207 | with tf.variable_scope("convs_6_1"): 208 | conv_6_1 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_old[0], w1, [1, 1, 1, 1], 'SAME')), 209 | feature_list_old[1]) 210 | feature_list_new.append(conv_6_1) 211 | 212 | for cnt in range(2, conv_5_5.get_shape().as_list()[1]): 213 | with tf.variable_scope("convs_6_1", reuse=True): 214 | conv_6_1 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_new[cnt - 1], w1, [1, 1, 1, 1], 'SAME')), 215 | feature_list_old[cnt]) 216 | feature_list_new.append(conv_6_1) 217 | 218 | # down to top # 219 | feature_list_old = feature_list_new 220 | feature_list_new = [] 221 | length = int(CFG.TRAIN.IMG_HEIGHT / 8) - 1 222 | feature_list_new.append(feature_list_old[length]) 223 | 224 | w2 = tf.get_variable('W2', [1, 9, 128, 128], 225 | initializer=tf.random_normal_initializer(0, math.sqrt(2.0 / (9 * 128 * 128 * 5)))) 226 | with tf.variable_scope("convs_6_2"): 227 | conv_6_2 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_old[length], w2, [1, 1, 1, 1], 'SAME')), 228 | feature_list_old[length - 1]) 229 | feature_list_new.append(conv_6_2) 230 | 231 | for cnt in range(2, conv_5_5.get_shape().as_list()[1]): 232 | with tf.variable_scope("convs_6_2", reuse=True): 233 | conv_6_2 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_new[cnt - 1], w2, [1, 1, 1, 1], 'SAME')), 234 | feature_list_old[length - cnt]) 235 | feature_list_new.append(conv_6_2) 236 | 237 | feature_list_new.reverse() 238 | 239 | processed_feature = tf.stack(feature_list_new, axis=1) 240 | processed_feature = tf.squeeze(processed_feature, axis=2) 241 | 242 | # left to right # 243 | 244 | feature_list_old = [] 245 | feature_list_new = [] 246 | for cnt in range(processed_feature.get_shape().as_list()[2]): 247 | feature_list_old.append(tf.expand_dims(processed_feature[:, :, cnt, :], axis=2)) 248 | feature_list_new.append(tf.expand_dims(processed_feature[:, :, 0, :], axis=2)) 249 | 250 | w3 = tf.get_variable('W3', [9, 1, 128, 128], 251 | initializer=tf.random_normal_initializer(0, math.sqrt(2.0 / (9 * 128 * 128 * 5)))) 252 | with tf.variable_scope("convs_6_3"): 253 | conv_6_3 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_old[0], w3, [1, 1, 1, 1], 'SAME')), 254 | feature_list_old[1]) 255 | feature_list_new.append(conv_6_3) 256 | 257 | for cnt in range(2, processed_feature.get_shape().as_list()[2]): 258 | with tf.variable_scope("convs_6_3", reuse=True): 259 | conv_6_3 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_new[cnt - 1], w3, [1, 1, 1, 1], 'SAME')), 260 | feature_list_old[cnt]) 261 | feature_list_new.append(conv_6_3) 262 | 263 | # right to left # 264 | 265 | feature_list_old = feature_list_new 266 | feature_list_new = [] 267 | length = int(CFG.TRAIN.IMG_WIDTH / 8) - 1 268 | feature_list_new.append(feature_list_old[length]) 269 | 270 | w4 = tf.get_variable('W4', [9, 1, 128, 128], 271 | initializer=tf.random_normal_initializer(0, math.sqrt(2.0 / (9 * 128 * 128 * 5)))) 272 | with tf.variable_scope("convs_6_4"): 273 | conv_6_4 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_old[length], w4, [1, 1, 1, 1], 'SAME')), 274 | feature_list_old[length - 1]) 275 | feature_list_new.append(conv_6_4) 276 | 277 | for cnt in range(2, processed_feature.get_shape().as_list()[2]): 278 | with tf.variable_scope("convs_6_4", reuse=True): 279 | conv_6_4 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_new[cnt - 1], w4, [1, 1, 1, 1], 'SAME')), 280 | feature_list_old[length - cnt]) 281 | feature_list_new.append(conv_6_4) 282 | 283 | feature_list_new.reverse() 284 | processed_feature = tf.stack(feature_list_new, axis=2) 285 | processed_feature = tf.squeeze(processed_feature, axis=3) 286 | 287 | ####################### 288 | 289 | dropout_output = self.dropout(processed_feature, 0.9, is_training=self._is_training, 290 | name='dropout') # 0.9 denotes the probability of being kept 291 | 292 | conv_output = self.conv2d(inputdata=dropout_output, out_channel=5, 293 | kernel_size=1, use_bias=True, name='conv_6') 294 | 295 | ret['prob_output'] = tf.image.resize_images(conv_output, [CFG.TRAIN.IMG_HEIGHT, CFG.TRAIN.IMG_WIDTH]) 296 | 297 | ### add lane existence prediction branch ### 298 | 299 | # spatial softmax # 300 | features = conv_output # N x H x W x C 301 | softmax = tf.nn.softmax(features) 302 | 303 | avg_pool = self.avgpooling(softmax, kernel_size=2, stride=2) 304 | _, H, W, C = avg_pool.get_shape().as_list() 305 | reshape_output = tf.reshape(avg_pool, [-1, H * W * C]) 306 | fc_output = self.fullyconnect(reshape_output, 128) 307 | relu_output = self.relu(inputdata=fc_output, name='relu6') 308 | fc_output = self.fullyconnect(relu_output, 4) 309 | existence_output = fc_output 310 | 311 | ret['existence_output'] = existence_output 312 | 313 | return ret 314 | 315 | 316 | if __name__ == '__main__': 317 | a = tf.placeholder(dtype=tf.float32, shape=[CFG.TRAIN.BATCH_SIZE, CFG.TRAIN.IMG_HEIGHT, CFG.TRAIN.IMG_WIDTH, 3], 318 | name='input') 319 | encoder = VGG16Encoder(phase=tf.constant('train', dtype=tf.string)) 320 | ret = encoder.encode(a, name='encode') 321 | print(ret) 322 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/lanenet_model/.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/lanenet_model/lanenet_merge_model.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Time : 18-5-11 下午5:28 4 | # @Author : Luo Yao 5 | # @Site : http://icode.baidu.com/repos/baidu/personal-code/Luoyao 6 | # @File : lanenet_merge_model.py 7 | # @IDE: PyCharm Community Edition 8 | """ 9 | Build Lane detection model 10 | """ 11 | import tensorflow as tf 12 | 13 | from encoder_decoder_model import vgg_encoder 14 | from encoder_decoder_model import cnn_basenet 15 | from config import global_config 16 | 17 | CFG = global_config.cfg 18 | 19 | 20 | class LaneNet(cnn_basenet.CNNBaseModel): 21 | """ 22 | Lane detection model 23 | """ 24 | 25 | @staticmethod 26 | def inference(input_tensor, phase, name): 27 | """ 28 | feed forward 29 | :param name: 30 | :param input_tensor: 31 | :param phase: 32 | :return: 33 | """ 34 | with tf.variable_scope(name): 35 | with tf.variable_scope('inference'): 36 | encoder = vgg_encoder.VGG16Encoder(phase=phase) 37 | encode_ret = encoder.encode(input_tensor=input_tensor, name='encode') 38 | 39 | return encode_ret 40 | 41 | @staticmethod 42 | def test_inference(input_tensor, phase, name): 43 | inference_ret = LaneNet.inference(input_tensor, phase, name) 44 | with tf.variable_scope(name): 45 | # feed forward to obtain logits 46 | # Compute loss 47 | 48 | decode_logits = inference_ret['prob_output'] 49 | binary_seg_ret = tf.nn.softmax(logits=decode_logits) 50 | prob_list = [] 51 | kernel = tf.get_variable('kernel', [9, 9, 1, 1], initializer=tf.constant_initializer(1.0 / 81), 52 | trainable=False) 53 | 54 | with tf.variable_scope("convs_smooth"): 55 | prob_smooth = tf.nn.conv2d(tf.cast(tf.expand_dims(binary_seg_ret[:, :, :, 0], axis=3), tf.float32), 56 | kernel, [1, 1, 1, 1], 'SAME') 57 | prob_list.append(prob_smooth) 58 | 59 | for cnt in range(1, binary_seg_ret.get_shape().as_list()[3]): 60 | with tf.variable_scope("convs_smooth", reuse=True): 61 | prob_smooth = tf.nn.conv2d( 62 | tf.cast(tf.expand_dims(binary_seg_ret[:, :, :, cnt], axis=3), tf.float32), kernel, [1, 1, 1, 1], 63 | 'SAME') 64 | prob_list.append(prob_smooth) 65 | processed_prob = tf.stack(prob_list, axis=4) 66 | processed_prob = tf.squeeze(processed_prob, axis=3) 67 | binary_seg_ret = processed_prob 68 | 69 | # Predict lane existence: 70 | existence_logit = inference_ret['existence_output'] 71 | existence_output = tf.nn.sigmoid(existence_logit) 72 | 73 | return binary_seg_ret, existence_output 74 | 75 | @staticmethod 76 | def loss(inference, binary_label, existence_label, name): 77 | """ 78 | :param name: 79 | :param inference: 80 | :param existence_label: 81 | :param binary_label: 82 | :return: 83 | """ 84 | # feed forward to obtain logits 85 | 86 | with tf.variable_scope(name): 87 | 88 | inference_ret = inference 89 | 90 | # Compute the segmentation loss 91 | 92 | decode_logits = inference_ret['prob_output'] 93 | decode_logits_reshape = tf.reshape( 94 | decode_logits, 95 | shape=[decode_logits.get_shape().as_list()[0], 96 | decode_logits.get_shape().as_list()[1] * decode_logits.get_shape().as_list()[2], 97 | decode_logits.get_shape().as_list()[3]]) 98 | 99 | binary_label_reshape = tf.reshape( 100 | binary_label, 101 | shape=[binary_label.get_shape().as_list()[0], 102 | binary_label.get_shape().as_list()[1] * binary_label.get_shape().as_list()[2]]) 103 | binary_label_reshape = tf.one_hot(binary_label_reshape, depth=5) 104 | class_weights = tf.constant([[0.4, 1.0, 1.0, 1.0, 1.0]]) 105 | weights_loss = tf.reduce_sum(tf.multiply(binary_label_reshape, class_weights), 2) 106 | binary_segmentation_loss = tf.losses.softmax_cross_entropy(onehot_labels=binary_label_reshape, 107 | logits=decode_logits_reshape, 108 | weights=weights_loss) 109 | binary_segmentation_loss = tf.reduce_mean(binary_segmentation_loss) 110 | 111 | # Compute the sigmoid loss 112 | 113 | existence_logits = inference_ret['existence_output'] 114 | existence_loss = tf.nn.sigmoid_cross_entropy_with_logits(labels=existence_label, logits=existence_logits) 115 | existence_loss = tf.reduce_mean(existence_loss) 116 | 117 | # Compute the overall loss 118 | 119 | total_loss = binary_segmentation_loss + 0.1 * existence_loss 120 | ret = { 121 | 'total_loss': total_loss, 122 | 'instance_seg_logits': decode_logits, 123 | 'instance_seg_loss': binary_segmentation_loss, 124 | 'existence_logits': existence_logits, 125 | 'existence_pre_loss': existence_loss 126 | } 127 | 128 | tf.add_to_collection('total_loss', total_loss) 129 | tf.add_to_collection('instance_seg_logits', decode_logits) 130 | tf.add_to_collection('instance_seg_loss', binary_segmentation_loss) 131 | tf.add_to_collection('existence_logits', existence_logits) 132 | tf.add_to_collection('existence_pre_loss', existence_loss) 133 | 134 | return ret 135 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/prob2lines/__pycache__/getLane.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/prob2lines/__pycache__/getLane.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/prob2lines/calTotal.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Thu Jun 11 17:28:39 2020 4 | 5 | @author: Mouiad 6 | """ 7 | 8 | from sklearn.metrics import confusion_matrix 9 | import tensorflow as tf 10 | import numpy as np 11 | import matplotlib.pyplot as plt 12 | import matplotlib.image as mpimg 13 | from sklearn.metrics import precision_recall_fscore_support 14 | y_true=r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\data\CULane\driver_100_30frame\05250419_0290.MP4\00030.lines.txt" 15 | y_pred=r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\prob2lines\output\vgg_SCNN_DULR_w9\driver_100_30frame\05251517_0433.MP4\00030.lines.txt" 16 | 17 | # Read the data 18 | 19 | 20 | 21 | 22 | with open(y_true, 'r') as infile: 23 | actual = [str(i) for i in infile] 24 | s="" 25 | for i in actual: 26 | s+=i 27 | y_real=s.split() 28 | XX=[] 29 | for i in y_real: 30 | XX.append(float(i)) 31 | 32 | X=[] 33 | for i in XX: 34 | X.append(int(i)) 35 | 36 | with open(y_pred, 'r') as infile: 37 | predicted = [str(i) for i in infile] 38 | 39 | ss="" 40 | for i in predicted: 41 | ss+=i 42 | y_pred=ss.split() 43 | YY=[] 44 | for i in y_pred: 45 | YY.append(float(i)) 46 | 47 | # Make confusion matrix 48 | X=np.array(X) 49 | YY=np.array(YY) 50 | Y=np.resize(YY,(160,1)) 51 | Y=Y>500 52 | confusion = confusion_matrix(X, Y) 53 | 54 | 55 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/requirements.txt: -------------------------------------------------------------------------------- 1 | easydict==1.6 2 | matplotlib==2.0.2 3 | glog==0.3.1 4 | opencv_python==3.2.0.7 5 | numpy==1.13.1 6 | scikit_learn==0.19.1 7 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/advanced_lane.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/advanced_lane.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/cnn_basenet.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/cnn_basenet.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/getLane.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/getLane.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/global_config.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/global_config.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/global_lane.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/global_lane.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/lanenet_data_processor_test.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/lanenet_data_processor_test.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/lanenet_merge_model.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/lanenet_merge_model.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/main.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/main.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/main1.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/main1.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/perspective_regionofint_main.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/perspective_regionofint_main.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/sliding_main.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/sliding_main.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/test_lanenet.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/test_lanenet.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/thresholding_main.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/thresholding_main.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/__pycache__/vgg_encoder.cpython-37.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane-detection-model/tools/__pycache__/vgg_encoder.cpython-37.pyc -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/advanced_lane.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import cv2 3 | import matplotlib.pyplot as plt 4 | import matplotlib.image as mpimg 5 | import glob 6 | import pickle 7 | from ipywidgets import widgets 8 | from IPython.display import display 9 | from IPython.display import Image 10 | from ipywidgets import interactive, interact, fixed 11 | from moviepy.editor import VideoFileClip 12 | from IPython.display import HTML 13 | from skimage import img_as_ubyte 14 | from thresholding_main import * 15 | from perspective_regionofint_main import * 16 | from sliding_main import * 17 | from scipy import misc 18 | 19 | def draw_on_original(undist, left_fitx, right_fitx, ploty, Minv): 20 | # Create an image to draw the lines on 21 | color_warp = np.zeros_like(undist).astype(np.uint8) 22 | # Recast the x and y points into usable format for cv2.fillPoly() 23 | pts_left = np.array([np.transpose(np.vstack([left_fitx, ploty]))]) 24 | pts_right = np.array([np.flipud(np.transpose(np.vstack([right_fitx, ploty])))]) 25 | pts = np.hstack((pts_left, pts_right)) 26 | # Draw the lane with low confidence region in yollow 27 | cv2.fillPoly(color_warp, np.int_([pts]), (255, 255,0)) 28 | # confidence region in green 29 | start_width = int(undist.shape[0] * 55.5 / 100.0) 30 | pts_left = np.array([np.transpose(np.vstack([left_fitx[start_width:], ploty[start_width:]]))]) 31 | pts_right = np.array([np.flipud(np.transpose(np.vstack([right_fitx[start_width:], ploty[start_width:]])))]) 32 | pts = np.hstack((pts_left, pts_right)) 33 | # Draw the lane onto the warped blank image 34 | cv2.fillPoly(color_warp, np.int_([pts]), (0, 0, 255)) 35 | # Warp the blank back to original image space using inverse perspective matrix (Minv) 36 | newwarp = cv2.warpPerspective(color_warp, Minv, (undist.shape[1], undist.shape[0])) 37 | # Combine the result with the original image 38 | result = cv2.addWeighted(undist, 1, newwarp, 0.4, 0) 39 | return result 40 | 41 | def pipeline(img): 42 | thresh_combined, grad_th, col_th = thresholding(img) 43 | perspective, Minv = perspective_transform(thresh_combined) 44 | perspective = cv2.cvtColor(perspective, cv2.COLOR_RGB2GRAY).astype(np.uint8) 45 | slides_pers, left_fitx, right_fitx, ploty, avg_cur, dist_centre_val = for_sliding_window(perspective) 46 | mapped_lane = draw_on_original(img, left_fitx, right_fitx, ploty, Minv) # should be undist 47 | L=draw_on_original_Left(img, left_fitx, ploty, Minv) 48 | R=draw_on_original_Right(img, right_fitx, ploty, Minv) 49 | L=cv2.resize(L, (800,288)) 50 | #print(L.shape) 51 | R=cv2.resize(R, (800,288)) 52 | #print(R.shape) 53 | # font and text for drawing the offset and curvature 54 | """curvature = "Estimated lane curvature %.2fm" % (avg_cur) 55 | dist_centre = "Estimated offset from lane center %.2fm" % (dist_centre_val) 56 | font = cv2.FONT_HERSHEY_COMPLEX 57 | # using cv2 for drawing text/images in diagnostic pipeline. 58 | # else return the original mapped imaged with the curvature and offset drawn 59 | cv2.putText(mapped_lane, curvature, (30, 60), font, 1.2, (0, 255, 255), 2) 60 | cv2.putText(mapped_lane, dist_centre, (30, 120), font, 1.2, (0, 255, 255), 2)""" 61 | return mapped_lane 62 | def pipeline1(img): 63 | thresh_combined, grad_th, col_th = thresholding(img) 64 | perspective, Minv = perspective_transform(thresh_combined) 65 | perspective = cv2.cvtColor(perspective, cv2.COLOR_RGB2GRAY).astype(np.uint8) 66 | slides_pers, left_fitx, right_fitx, ploty, avg_cur, dist_centre_val = for_sliding_window(perspective) 67 | mapped_lane = draw_on_original(img, left_fitx, right_fitx, ploty, Minv) # should be undist 68 | L=draw_on_original_Left(img, left_fitx, ploty, Minv) 69 | R=draw_on_original_Right(img, right_fitx, ploty, Minv) 70 | L=cv2.resize(L, (800,288)) 71 | #print(L.shape) 72 | R=cv2.resize(R, (800,288)) 73 | #print(R.shape) 74 | # font and text for drawing the offset and curvature 75 | """curvature = "Estimated lane curvature %.2fm" % (avg_cur) 76 | dist_centre = "Estimated offset from lane center %.2fm" % (dist_centre_val) 77 | font = cv2.FONT_HERSHEY_COMPLEX 78 | # using cv2 for drawing text/images in diagnostic pipeline. 79 | # else return the original mapped imaged with the curvature and offset drawn 80 | cv2.putText(mapped_lane, curvature, (30, 60), font, 1.2, (0, 255, 255), 2) 81 | cv2.putText(mapped_lane, dist_centre, (30, 120), font, 1.2, (0, 255, 255), 2)""" 82 | return mapped_lane,L,R 83 | def draw_on_original_Left(undist, left_fitx, ploty, Minv): 84 | # Create an image to draw the lines on 85 | color_warp = np.zeros_like(undist).astype(np.uint8) 86 | # confidence region in green 87 | start_width = int(30) 88 | pts_left = np.array([np.transpose(np.vstack([left_fitx[start_width:], ploty[start_width:]]))]) 89 | # Draw the lane onto the warped blank image 90 | cv2.fillPoly(color_warp, np.int_([pts_left]), (255, 255, 255)) 91 | # Warp the blank back to original image space using inverse perspective matrix (Minv) 92 | global newwarp1 93 | newwarp1 = cv2.warpPerspective(color_warp, Minv, (undist.shape[1], undist.shape[0])) 94 | # Combine the result with the original image 95 | return newwarp1 96 | 97 | def draw_on_original_Right(undist, right_fitx, ploty, Minv): 98 | # Create an image to draw the lines on 99 | color_warp = np.zeros_like(undist).astype(np.uint8) 100 | # confidence region in green 101 | start_width = int(30) 102 | pts_right = np.array([np.flipud(np.transpose(np.vstack([right_fitx[start_width:], ploty[start_width:]])))]) 103 | # Draw the lane onto the warped blank image 104 | cv2.fillPoly(color_warp, np.int_([pts_right]), (255, 255, 255)) 105 | # Warp the blank back to original image space using inverse perspective matrix (Minv) 106 | global newwarp 107 | newwarp = cv2.warpPerspective(color_warp, Minv, (undist.shape[1], undist.shape[0])) 108 | # Combine the result with the original image 109 | return newwarp 110 | 111 | def ones_or_zeros_for_Right(): 112 | if np.mean(newwarp)>0: 113 | return True 114 | return False 115 | 116 | def ones_or_zeros_for_Left(): 117 | if np.mean(newwarp1)>0: 118 | return True 119 | return False -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/calibration_main.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import cv2 3 | import matplotlib.pyplot as plt 4 | import matplotlib.image as mpimg 5 | import glob 6 | import pickle 7 | 8 | 9 | #reads the calibrated values 10 | def get_camera_calibration(): 11 | dist_pickle = pickle.load( open( "camera_cal/camera_cal.p", "rb" ) ) 12 | mtx = dist_pickle["mtx"] 13 | dist = dist_pickle["dist"] 14 | return mtx, dist 15 | 16 | #this is the function for doijng calibration and storing the result in the pickle file 17 | def do_calibration(): 18 | #for camera calibration and store the result in a file "camera_cal/camera_cal.p" 19 | #Array to store the obj point and image points 20 | objpoints = [] 21 | imgpoints = [] 22 | objp = np.zeros((8*8,3),np.float32) 23 | objp[:,:2] = np.mgrid[0:8,0:8].T.reshape(-1,2)*.55 24 | images = glob.glob("camera_cal/Ima*") 25 | for fnames in images: 26 | img = mpimg.imread(fnames) 27 | gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) 28 | ret, corners = cv2.findChessboardCorners(gray, (8,8), None) 29 | if ret == True: 30 | imgpoints.append(corners) 31 | objpoints.append(objp) 32 | #draw lines on the cheesboard 33 | img = cv2.drawChessboardCorners(img, (8,8), corners, ret) 34 | #plt.imshow(img) 35 | ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera(objpoints, imgpoints, gray.shape[::-1],None,None) 36 | #save the calibration data in a pickle file to use later 37 | camera_cal_val = "camera_cal/camera_cal.p" 38 | output = open(camera_cal_val, 'wb') 39 | 40 | mydict2 = {'mtx': 1, 'dist': 2} 41 | mydict2['mtx'] = mtx 42 | mydict2['dist'] = dist 43 | pickle.dump(mydict2, output) 44 | output.close() -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/cnn_basenet.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | # @Time : 17-9-18 下午3:59 4 | # @Author : Luo Yao 5 | # @Site : http://github.com/TJCVRS 6 | # @File : cnn_basenet.py 7 | # @IDE: PyCharm Community Edition 8 | """ 9 | The base convolution neural networks mainly implement some useful cnn functions 10 | """ 11 | import tensorflow as tf 12 | import tensorflow.contrib.layers as tf_layer 13 | from tensorflow.contrib.layers.python.layers import initializers 14 | import numpy as np 15 | 16 | slim = tf.contrib.slim 17 | 18 | class CNNBaseModel(object): 19 | """ 20 | Base model for other specific cnn ctpn_models 21 | """ 22 | 23 | def __init__(self): 24 | pass 25 | 26 | @staticmethod 27 | def conv2d(inputdata, out_channel, kernel_size, padding='SAME', 28 | stride=1, w_init=None, b_init=None, 29 | split=1, use_bias=True, data_format='NHWC', name=None): 30 | """ 31 | Packing the tensorflow conv2d function. 32 | :param name: op name 33 | :param inputdata: A 4D tensorflow tensor which ust have known number of channels, but can have other 34 | unknown dimensions. 35 | :param out_channel: number of output channel. 36 | :param kernel_size: int so only support square kernel convolution 37 | :param padding: 'VALID' or 'SAME' 38 | :param stride: int so only support square stride 39 | :param w_init: initializer for convolution weights 40 | :param b_init: initializer for bias 41 | :param split: split channels as used in Alexnet mainly group for GPU memory save. 42 | :param use_bias: whether to use bias. 43 | :param data_format: default set to NHWC according tensorflow 44 | :return: tf.Tensor named ``output`` 45 | """ 46 | with tf.variable_scope(name): 47 | in_shape = inputdata.get_shape().as_list() 48 | channel_axis = 3 if data_format == 'NHWC' else 1 49 | in_channel = in_shape[channel_axis] 50 | assert in_channel is not None, "[Conv2D] Input cannot have unknown channel!" 51 | assert in_channel % split == 0 52 | assert out_channel % split == 0 53 | 54 | padding = padding.upper() 55 | 56 | if isinstance(kernel_size, list): 57 | filter_shape = [kernel_size[0], kernel_size[1]] + [in_channel / split, out_channel] 58 | else: 59 | filter_shape = [kernel_size, kernel_size] + [in_channel / split, out_channel] 60 | 61 | if isinstance(stride, list): 62 | strides = [1, stride[0], stride[1], 1] if data_format == 'NHWC' \ 63 | else [1, 1, stride[0], stride[1]] 64 | else: 65 | strides = [1, stride, stride, 1] if data_format == 'NHWC' \ 66 | else [1, 1, stride, stride] 67 | 68 | if w_init is None: 69 | w_init = tf.contrib.layers.variance_scaling_initializer() 70 | if b_init is None: 71 | b_init = tf.constant_initializer() 72 | 73 | w = tf.get_variable('W', filter_shape, initializer=w_init) 74 | b = None 75 | 76 | if use_bias: 77 | b = tf.get_variable('b', [out_channel], initializer=b_init) 78 | 79 | if split == 1: 80 | conv = tf.nn.conv2d(inputdata, w, strides, padding, data_format=data_format) 81 | else: 82 | inputs = tf.split(inputdata, split, channel_axis) 83 | kernels = tf.split(w, split, 3) 84 | outputs = [tf.nn.conv2d(i, k, strides, padding, data_format=data_format) 85 | for i, k in zip(inputs, kernels)] 86 | conv = tf.concat(outputs, channel_axis) 87 | 88 | ret = tf.identity(tf.nn.bias_add(conv, b, data_format=data_format) 89 | if use_bias else conv, name=name) 90 | 91 | return ret 92 | 93 | @staticmethod 94 | def relu(inputdata, name=None): 95 | """ 96 | 97 | :param name: 98 | :param inputdata: 99 | :return: 100 | """ 101 | return tf.nn.relu(features=inputdata, name=name) 102 | 103 | @staticmethod 104 | def sigmoid(inputdata, name=None): 105 | """ 106 | 107 | :param name: 108 | :param inputdata: 109 | :return: 110 | """ 111 | return tf.nn.sigmoid(x=inputdata, name=name) 112 | 113 | @staticmethod 114 | def maxpooling(inputdata, kernel_size, stride=None, padding='VALID', 115 | data_format='NHWC', name=None): 116 | """ 117 | 118 | :param name: 119 | :param inputdata: 120 | :param kernel_size: 121 | :param stride: 122 | :param padding: 123 | :param data_format: 124 | :return: 125 | """ 126 | padding = padding.upper() 127 | 128 | if stride is None: 129 | stride = kernel_size 130 | 131 | if isinstance(kernel_size, list): 132 | kernel = [1, kernel_size[0], kernel_size[1], 1] if data_format == 'NHWC' else \ 133 | [1, 1, kernel_size[0], kernel_size[1]] 134 | else: 135 | kernel = [1, kernel_size, kernel_size, 1] if data_format == 'NHWC' \ 136 | else [1, 1, kernel_size, kernel_size] 137 | 138 | if isinstance(stride, list): 139 | strides = [1, stride[0], stride[1], 1] if data_format == 'NHWC' \ 140 | else [1, 1, stride[0], stride[1]] 141 | else: 142 | strides = [1, stride, stride, 1] if data_format == 'NHWC' \ 143 | else [1, 1, stride, stride] 144 | 145 | return tf.nn.max_pool(value=inputdata, ksize=kernel, strides=strides, padding=padding, 146 | data_format=data_format, name=name) 147 | 148 | @staticmethod 149 | def avgpooling(inputdata, kernel_size, stride=None, padding='VALID', 150 | data_format='NHWC', name=None): 151 | """ 152 | 153 | :param name: 154 | :param inputdata: 155 | :param kernel_size: 156 | :param stride: 157 | :param padding: 158 | :param data_format: 159 | :return: 160 | """ 161 | if stride is None: 162 | stride = kernel_size 163 | 164 | kernel = [1, kernel_size, kernel_size, 1] if data_format == 'NHWC' \ 165 | else [1, 1, kernel_size, kernel_size] 166 | 167 | strides = [1, stride, stride, 1] if data_format == 'NHWC' else [1, 1, stride, stride] 168 | 169 | return tf.nn.avg_pool(value=inputdata, ksize=kernel, strides=strides, padding=padding, 170 | data_format=data_format, name=name) 171 | 172 | @staticmethod 173 | def globalavgpooling(inputdata, data_format='NHWC', name=None): 174 | """ 175 | 176 | :param name: 177 | :param inputdata: 178 | :param data_format: 179 | :return: 180 | """ 181 | assert inputdata.shape.ndims == 4 182 | assert data_format in ['NHWC', 'NCHW'] 183 | 184 | axis = [1, 2] if data_format == 'NHWC' else [2, 3] 185 | 186 | return tf.reduce_mean(input_tensor=inputdata, axis=axis, name=name) 187 | 188 | @staticmethod 189 | def layernorm(inputdata, epsilon=1e-5, use_bias=True, use_scale=True, 190 | data_format='NHWC', name=None): 191 | """ 192 | :param name: 193 | :param inputdata: 194 | :param epsilon: epsilon to avoid divide-by-zero. 195 | :param use_bias: whether to use the extra affine transformation or not. 196 | :param use_scale: whether to use the extra affine transformation or not. 197 | :param data_format: 198 | :return: 199 | """ 200 | shape = inputdata.get_shape().as_list() 201 | ndims = len(shape) 202 | assert ndims in [2, 4] 203 | 204 | mean, var = tf.nn.moments(inputdata, list(range(1, len(shape))), keep_dims=True) 205 | 206 | if data_format == 'NCHW': 207 | channnel = shape[1] 208 | new_shape = [1, channnel, 1, 1] 209 | else: 210 | channnel = shape[-1] 211 | new_shape = [1, 1, 1, channnel] 212 | if ndims == 2: 213 | new_shape = [1, channnel] 214 | 215 | if use_bias: 216 | beta = tf.get_variable('beta', [channnel], initializer=tf.constant_initializer()) 217 | beta = tf.reshape(beta, new_shape) 218 | else: 219 | beta = tf.zeros([1] * ndims, name='beta') 220 | if use_scale: 221 | gamma = tf.get_variable('gamma', [channnel], initializer=tf.constant_initializer(1.0)) 222 | gamma = tf.reshape(gamma, new_shape) 223 | else: 224 | gamma = tf.ones([1] * ndims, name='gamma') 225 | 226 | return tf.nn.batch_normalization(inputdata, mean, var, beta, gamma, epsilon, name=name) 227 | 228 | @staticmethod 229 | def instancenorm(inputdata, epsilon=1e-5, data_format='NHWC', use_affine=True, name=None): 230 | """ 231 | 232 | :param name: 233 | :param inputdata: 234 | :param epsilon: 235 | :param data_format: 236 | :param use_affine: 237 | :return: 238 | """ 239 | shape = inputdata.get_shape().as_list() 240 | if len(shape) != 4: 241 | raise ValueError("Input data of instancebn layer has to be 4D tensor") 242 | 243 | if data_format == 'NHWC': 244 | axis = [1, 2] 245 | ch = shape[3] 246 | new_shape = [1, 1, 1, ch] 247 | else: 248 | axis = [2, 3] 249 | ch = shape[1] 250 | new_shape = [1, ch, 1, 1] 251 | if ch is None: 252 | raise ValueError("Input of instancebn require known channel!") 253 | 254 | mean, var = tf.nn.moments(inputdata, axis, keep_dims=True) 255 | 256 | if not use_affine: 257 | return tf.divide(inputdata - mean, tf.sqrt(var + epsilon), name='output') 258 | 259 | beta = tf.get_variable('beta', [ch], initializer=tf.constant_initializer()) 260 | beta = tf.reshape(beta, new_shape) 261 | gamma = tf.get_variable('gamma', [ch], initializer=tf.constant_initializer(1.0)) 262 | gamma = tf.reshape(gamma, new_shape) 263 | return tf.nn.batch_normalization(inputdata, mean, var, beta, gamma, epsilon, name=name) 264 | 265 | @staticmethod 266 | def dropout(inputdata, keep_prob, is_training=None, noise_shape=None, name=None): 267 | """ 268 | 269 | :param name: 270 | :param inputdata: 271 | :param keep_prob: 272 | :param noise_shape: 273 | :return: 274 | """ 275 | def f1(): 276 | """ 277 | 278 | :return: 279 | """ 280 | # print('batch_normalization: train phase') 281 | return tf.nn.dropout(inputdata, keep_prob, noise_shape, name=name) 282 | 283 | def f2(): 284 | """ 285 | 286 | :return: 287 | """ 288 | # print('batch_normalization: test phase') 289 | return inputdata 290 | 291 | output = tf.cond(is_training, f1, f2) 292 | # output = tf.nn.dropout(inputdata, keep_prob, noise_shape, name=name) 293 | 294 | return output 295 | 296 | # return tf.nn.dropout(inputdata, keep_prob=keep_prob, noise_shape=noise_shape, name=name) 297 | 298 | @staticmethod 299 | def fullyconnect(inputdata, out_dim, w_init=None, b_init=None, 300 | use_bias=True, name=None): 301 | """ 302 | Fully-Connected layer, takes a N>1D tensor and returns a 2D tensor. 303 | It is an equivalent of `tf.layers.dense` except for naming conventions. 304 | 305 | :param inputdata: a tensor to be flattened except for the first dimension. 306 | :param out_dim: output dimension 307 | :param w_init: initializer for w. Defaults to `variance_scaling_initializer`. 308 | :param b_init: initializer for b. Defaults to zero 309 | :param use_bias: whether to use bias. 310 | :param name: 311 | :return: tf.Tensor: a NC tensor named ``output`` with attribute `variables`. 312 | """ 313 | shape = inputdata.get_shape().as_list()[1:] 314 | if None not in shape: 315 | inputdata = tf.reshape(inputdata, [-1, int(np.prod(shape))]) 316 | else: 317 | inputdata = tf.reshape(inputdata, tf.stack([tf.shape(inputdata)[0], -1])) 318 | 319 | if w_init is None: 320 | w_init = tf.contrib.layers.variance_scaling_initializer() 321 | if b_init is None: 322 | b_init = tf.constant_initializer() 323 | 324 | ret = tf.layers.dense(inputs=inputdata, activation=lambda x: tf.identity(x, name='output'), 325 | use_bias=use_bias, name=name, 326 | kernel_initializer=w_init, bias_initializer=b_init, 327 | trainable=True, units=out_dim) 328 | return ret 329 | 330 | @staticmethod 331 | def layerbn(inputdata, is_training, name): 332 | """ 333 | 334 | :param inputdata: 335 | :param is_training: 336 | :param name: 337 | :return: 338 | """ 339 | def f1(): 340 | """ 341 | 342 | :return: 343 | """ 344 | # print('batch_normalization: train phase') 345 | return tf_layer.batch_norm( 346 | inputdata, is_training=True, 347 | center=True, scale=True, updates_collections=None, 348 | scope=name, reuse=False) 349 | 350 | def f2(): 351 | """ 352 | 353 | :return: 354 | """ 355 | # print('batch_normalization: test phase') 356 | return tf_layer.batch_norm( 357 | inputdata, is_training=False, 358 | center=True, scale=True, updates_collections=None, 359 | scope=name, reuse=True) 360 | 361 | output = tf.cond(is_training, f1, f2) 362 | 363 | return output 364 | 365 | @staticmethod 366 | def squeeze(inputdata, axis=None, name=None): 367 | """ 368 | 369 | :param inputdata: 370 | :param axis: 371 | :param name: 372 | :return: 373 | """ 374 | return tf.squeeze(input=inputdata, axis=axis, name=name) 375 | 376 | @staticmethod 377 | def deconv2d(inputdata, out_channel, kernel_size, padding='SAME', 378 | stride=1, w_init=None, b_init=None, 379 | use_bias=True, activation=None, data_format='channels_last', 380 | trainable=True, name=None): 381 | """ 382 | Packing the tensorflow conv2d function. 383 | :param name: op name 384 | :param inputdata: A 4D tensorflow tensor which ust have known number of channels, but can have other 385 | unknown dimensions. 386 | :param out_channel: number of output channel. 387 | :param kernel_size: int so only support square kernel convolution 388 | :param padding: 'VALID' or 'SAME' 389 | :param stride: int so only support square stride 390 | :param w_init: initializer for convolution weights 391 | :param b_init: initializer for bias 392 | :param activation: whether to apply a activation func to deconv result 393 | :param use_bias: whether to use bias. 394 | :param data_format: default set to NHWC according tensorflow 395 | :return: tf.Tensor named ``output`` 396 | """ 397 | with tf.variable_scope(name): 398 | in_shape = inputdata.get_shape().as_list() 399 | channel_axis = 3 if data_format == 'channels_last' else 1 400 | in_channel = in_shape[channel_axis] 401 | assert in_channel is not None, "[Deconv2D] Input cannot have unknown channel!" 402 | 403 | padding = padding.upper() 404 | 405 | if w_init is None: 406 | w_init = tf.contrib.layers.variance_scaling_initializer() 407 | if b_init is None: 408 | b_init = tf.constant_initializer() 409 | 410 | ret = tf.layers.conv2d_transpose(inputs=inputdata, filters=out_channel, 411 | kernel_size=kernel_size, 412 | strides=stride, padding=padding, 413 | data_format=data_format, 414 | activation=activation, use_bias=use_bias, 415 | kernel_initializer=w_init, 416 | bias_initializer=b_init, trainable=trainable, 417 | name=name) 418 | return ret 419 | 420 | @staticmethod 421 | def dilation_conv(input_tensor, k_size, out_dims, rate, padding='SAME', 422 | w_init=None, b_init=None, use_bias=False, name=None): 423 | """ 424 | 425 | :param input_tensor: 426 | :param k_size: 427 | :param out_dims: 428 | :param rate: 429 | :param padding: 430 | :param w_init: 431 | :param b_init: 432 | :param use_bias: 433 | :param name: 434 | :return: 435 | """ 436 | with tf.variable_scope(name): 437 | in_shape = input_tensor.get_shape().as_list() 438 | in_channel = in_shape[3] 439 | assert in_channel is not None, "[Conv2D] Input cannot have unknown channel!" 440 | 441 | padding = padding.upper() 442 | 443 | if isinstance(k_size, list): 444 | filter_shape = [k_size[0], k_size[1]] + [in_channel, out_dims] 445 | else: 446 | filter_shape = [k_size, k_size] + [in_channel, out_dims] 447 | 448 | if w_init is None: 449 | w_init = tf.contrib.layers.variance_scaling_initializer() 450 | if b_init is None: 451 | b_init = tf.constant_initializer() 452 | 453 | w = tf.get_variable('W', filter_shape, initializer=w_init) 454 | b = None 455 | 456 | if use_bias: 457 | b = tf.get_variable('b', [out_dims], initializer=b_init) 458 | 459 | conv = tf.nn.atrous_conv2d(value=input_tensor, filters=w, rate=rate, 460 | padding=padding, name='dilation_conv') 461 | 462 | if use_bias: 463 | ret = tf.add(conv, b) 464 | else: 465 | ret = conv 466 | 467 | return ret 468 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/computer_vison_1.py: -------------------------------------------------------------------------------- 1 | 2 | from tkinter import * 3 | import imageio 4 | from tkinter import ttk 5 | from PIL import Image, ImageTk 6 | from tkinter import filedialog 7 | import glob 8 | import cv2 9 | from advanced_lane import * 10 | import os 11 | import cv2 12 | import numpy as np 13 | import test_lanenet as t 14 | from time import sleep 15 | 16 | def create_video(path): 17 | img_array = [] 18 | for filename in path: 19 | #video_name=filename[-59:-10]+".mp4" 20 | img = cv2.imread(filename) 21 | height, width, layers = img.shape 22 | size = (width,height) 23 | img_array.append(img) 24 | #video_name=video_name.replace("\\","Z") 25 | #print(video_name) 26 | out = cv2.VideoWriter(r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\Visual_output\Dl\video\video.mp4",cv2.VideoWriter_fourcc(*'DIVX'), 30, size) 27 | for i in range(len(img_array)): 28 | out.write(img_array[i]) 29 | return out.release() 30 | 31 | def cut_video(pathread,pathsave,file): 32 | vidcap=cv2.VideoCapture(pathread) 33 | success,image=vidcap.read() 34 | count=0 35 | #print(path) 36 | f=open(file,'w') 37 | while success: 38 | cv2.imwrite(pathsave+"//0000%d.jpg" % count,image) 39 | f.write(pathsave+"/0000%d.jpg" % count) 40 | f.write("\n") 41 | success,image=vidcap.read() 42 | count+=1 43 | #print("read a new frame: " , success) 44 | def fileDialog1(): 45 | filename = filedialog.askopenfilename( ) 46 | #print(filename) 47 | video_name=filename[-19:-16] 48 | print("momo"+video_name) 49 | white_output = r'C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\Visual_output\cv\video\\'+video_name+'.mp4' 50 | clip1 = VideoFileClip(filename) 51 | init_lines(590) 52 | white_clip = clip1.fl_image(pipeline) #NOTE: this function expects color images!! 53 | white_clip.write_videofile(white_output, audio=False) 54 | 55 | def fileDialog(): 56 | filename = filedialog.askopenfilename( filetype = 57 | (("jpg files","*.jpg"),("all files","*.*")) ) 58 | #print(filename) 59 | for img_name in glob.glob(filename): 60 | img = cv2.imread(img_name) 61 | init_lines(img.shape[0]) 62 | img2,L,R = pipeline1(img) 63 | image_name=img_name[-9:] 64 | img2=cv2.cvtColor(img2, cv2.COLOR_RGB2BGR) 65 | plt.figure(figsize = (10,5)) 66 | plt.imshow(img2) 67 | cv2.imwrite(r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\Visual_output\cv\image\\"+image_name+".jpg",img2) 68 | parent_path = os.path.dirname(filename)[98:] 69 | parent_path = parent_path.replace('/', '\\') 70 | directory = os.path.join(r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\experiments\predicts", 'vgg_SCNN_DULR_w9', parent_path) 71 | #print(directory) 72 | if not os.path.exists(directory): 73 | os.makedirs(directory) 74 | file_exist = open(os.path.join(directory, os.path.basename(filename)[:-3] + 'exist.txt'), 'w') 75 | for cnt_img in range(1): 76 | cv2.imwrite(os.path.join(directory, os.path.basename(filename)[:-4] + '_' + str(cnt_img + 2) + '_avg.png'),L) 77 | cv2.imwrite(os.path.join(directory, os.path.basename(filename)[:-4] + '_' + str(cnt_img + 3) + '_avg.png'),R) 78 | file_exist.write('0 ') 79 | if ones_or_zeros_for_Right()==True: 80 | file_exist.write('1 ') 81 | else: 82 | file_exist.write('0 ') 83 | if ones_or_zeros_for_Left()==True: 84 | file_exist.write('1 ') 85 | else: 86 | file_exist.write('0 ') 87 | file_exist.write('0 ') 88 | file_exist.close() 89 | 90 | 91 | def fileDialog2(): 92 | l=Button(win1,text="dl",height=2,width=15,font=(25),bd=15,bg="#ff0000",fg="black",command=DL) 93 | file=r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\data\CULane\video.txt" 94 | file2=open(r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\data\CULane\list\test.txt","w") 95 | file3=open(r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\demo_file\test_img.txt","w") 96 | filename = filedialog.askopenfilename( ) 97 | path=filename[:-4] 98 | #file="../"+path[86:] 99 | try: 100 | os.makedirs(path) 101 | #print("Directory ",path," Created") 102 | except FileExistsError: 103 | pass 104 | print("Directory ",path," already exists") 105 | cut_video(filename,path,file) 106 | with open(file,'r') as f: 107 | for line in f: 108 | if line !="\n": 109 | #global v 110 | #v=line[-20:-16] 111 | image_name="../"+line[86:] 112 | file3.write(image_name) 113 | #print(image_name) 114 | image_name2=line[97:] 115 | file2.writelines(image_name2) 116 | #print(image_name2) 117 | sleep(5) 118 | l.place(x=500,y=500) 119 | win1.pack( fill=BOTH, expand=True) 120 | 121 | 122 | def DL(): 123 | 124 | t.init() 125 | import main1 as m 126 | #sleep(20) 127 | #from main1 import * 128 | p=m.pat() 129 | path2=glob.glob(p+"\*.jpg") 130 | create_video(path2) 131 | 132 | 133 | def fileDialog3(): 134 | l=Button(win1,text="dl",height=2,width=15,font=(25),bd=15,bg="#ff0000",fg="black",command=DLL) 135 | filename = filedialog.askopenfilename( filetype = 136 | (("jpg files","*.jpg"),("all files","*.*")) ) 137 | file=open(r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\demo_file\test_img.txt","w") 138 | image_name="../"+filename[86:] 139 | image_name2=filename[97:] 140 | file2=open(r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\data\CULane\list\test.txt","w") 141 | print(image_name2) 142 | print(image_name) 143 | file.write(image_name) 144 | file2.writelines(image_name2) 145 | 146 | sleep(5) 147 | #import test_lanenet as t 148 | l.place(x=500,y=500) 149 | win1.pack( fill=BOTH, expand=True) 150 | #t.init() 151 | 152 | 153 | def DLL(): 154 | 155 | t.init() 156 | import main1 as m 157 | 158 | 159 | 160 | def clearFrame(frame): 161 | # destroy all widgets from frame 162 | for widget in frame.winfo_children(): 163 | widget.destroy() 164 | 165 | # this will clear frame and frame will be empty 166 | # if you want to hide the empty panel then 167 | frame.pack_forget() 168 | def BACK(): 169 | clearFrame(win1) 170 | root.title(" Lane line detection ") 171 | root.iconbitmap(r'C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\GUI\algorithm.ico') 172 | root.geometry("1200x700") 173 | root.configure(background="black") 174 | p3 = PhotoImage(file=r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\GUI\111111.png") 175 | global win 176 | win=Frame(root) 177 | lm=Label(win,image=p3) 178 | lm.pack( fill=BOTH, expand=True) 179 | b1=Button(lm,text="Computer Vision",height=2,width=15,compound=CENTER,font=('TimesNewRoman', 14, 'bold'),bd=10,bg="SteelBlue1",fg="black",command=switch) 180 | b2=Button(lm,text="Deep Learning",height=2,width=15,compound=CENTER,font=('TimesNewRoman', 14, 'bold'),bd=10,bg="SteelBlue1",fg="black",command=switch2) 181 | b1.place(x=200,y=300) 182 | b2.place(x=800,y=300) 183 | b3=Button(lm,text="Exit",command=root.destroy,height=2,width=15,font=('TimesNewRoman', 14, 'bold'),bd=15,bg="SteelBlue4",fg="black") 184 | b3.place(x=475,y=500) 185 | win.pack(fill=BOTH, expand=True) 186 | root.resizable(0, 0) 187 | root.mainloop() 188 | def switch(): 189 | clearFrame(win) 190 | root.title("Lane line detection Using Computer Vision") 191 | root.iconbitmap(r'C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\GUI\Balance.ico') 192 | root.geometry("1200x700") 193 | 194 | global win1 195 | win1=Frame(root) 196 | llm=Label(win1,image=p2) 197 | 198 | 199 | 200 | b4=Button(llm,text="Image",height=2,width=15,compound=CENTER,font=(25),bd=20,bg="black",fg="ivory2",command=fileDialog) 201 | b7=Button(llm,text="Video",height=2,width=15,compound=CENTER,font=(25),bd=20,bg="black",fg="ivory2",command=fileDialog1) 202 | b4.place(x=500,y=200) 203 | b7.place(x=800,y=200) 204 | b5=Button(llm,text="Exit",command=root.destroy,height=2,width=15,font=(25),bd=15,bg="#ff0000",fg="ivory2") 205 | b5.place(x=1000,y=600) 206 | b6=Button(llm,text="Back",command=BACK,height=2,width=15,font=(25),bd=15,bg="blue",fg="ivory2") 207 | b6.place(x=0,y=0) 208 | llm.pack( fill=BOTH, expand=True) 209 | win1.pack(fill=BOTH, expand=True) 210 | def switch2(): 211 | clearFrame(win) 212 | root.title(" Lane line detection Using Deep learning ") 213 | root.iconbitmap(r'C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\GUI\A cup of coffee.ico') 214 | root.geometry("1200x700") 215 | 216 | global win1 217 | win1=Frame(root) 218 | global llm 219 | llm=Label(win1,image=p1) 220 | 221 | 222 | b4=Button(llm,text="Image",height=2,width=15,compound=CENTER,font=(25),bd=10,bg="white",fg="black",command=fileDialog3) 223 | b1=Button(llm,text="Video",height=2,width=15,compound=CENTER,font=(25),bd=10,bg="white",fg="black",command=fileDialog2) 224 | b4.place(x=500,y=200) 225 | b1.place(x=700,y=200) 226 | b5=Button(llm,text="Exit",command=root.destroy,height=2,width=15,font=(25),bd=15,bg="#ff0000",fg="black") 227 | b5.place(x=1000,y=600) 228 | b6=Button(llm,text="Back",command=BACK,height=2,width=15,font=(25),bd=15,bg="blue",fg="ivory2") 229 | b6.place(x=0,y=0) 230 | llm.pack( fill=BOTH, expand=True) 231 | win1.pack(fill=BOTH, expand=True) 232 | 233 | global root 234 | root = Tk() 235 | root.title(" Lane line detection ") 236 | root.iconbitmap(r'C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\GUI\algorithm.ico') 237 | root.geometry("1200x700") 238 | root.configure(background="black") 239 | 240 | 241 | p3 = PhotoImage(file=r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\GUI\111111.png") 242 | global win 243 | win=Frame(root) 244 | lm=Label(win,image=p3) 245 | lm.pack( fill=BOTH, expand=True) 246 | 247 | 248 | p2 = PhotoImage(file=r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\GUI\1212.png") 249 | p1 = PhotoImage(file=r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\GUI\3333.png") 250 | b1=Button(lm,text="Computer Vision",height=2,width=15,compound=CENTER,font=('TimesNewRoman', 14, 'bold'),bd=10,bg="SteelBlue1",fg="black",command=switch) 251 | b2=Button(lm,text="Deep Learning",height=2,width=15,compound=CENTER,font=('TimesNewRoman', 14, 'bold'),bd=10,bg="SteelBlue1",fg="black",command=switch2) 252 | b1.place(x=200,y=300) 253 | b2.place(x=800,y=300) 254 | b3=Button(lm,text="Exit",command=root.destroy,height=2,width=15,font=('TimesNewRoman', 14, 'bold'),bd=15,bg="SteelBlue4",fg="black") 255 | b3.place(x=475,y=500) 256 | win.pack(fill=BOTH, expand=True) 257 | root.resizable(0, 0) 258 | root.mainloop() 259 | 260 | 261 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/getLane.py: -------------------------------------------------------------------------------- 1 | """ 2 | Created on Sat May 30 22:30:44 2020 3 | 4 | @author: Mouiad 5 | """ 6 | import numpy as np 7 | 8 | 9 | def getLane(score): 10 | # Calculate lane position from a probmap. 11 | thr = 0.3 12 | coordinate = np.zeros((1, 18)) 13 | for i in range(18): 14 | lineId = np.uint16(np.round(288.0 - i * 20.0 / 590.0 * 288.0)) 15 | line = score[lineId - 1, :] 16 | value, index = np.amax(line), np.where(line == np.amax(line))[0][0] 17 | if value / 255.0 > thr: 18 | coordinate[0, i] = index + 1 # TODO be sure about the '+1' 19 | if np.sum(coordinate > 0) < 2: 20 | coordinate = np.zeros((1, 18)) 21 | return coordinate 22 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/global_config.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Time : 18-1-31 上午11:21 4 | # @Author : Luo Yao 5 | # @Site : http://icode.baidu.com/repos/baidu/personal-code/Luoyao 6 | # @File : global_config.py 7 | # @IDE: PyCharm Community Edition 8 | """ 9 | 设置全局变量 10 | """ 11 | from easydict import EasyDict as edict 12 | 13 | __C = edict() 14 | # Consumers can get config by: from config import cfg 15 | 16 | cfg = __C 17 | 18 | # Train options 19 | __C.TRAIN = edict() 20 | 21 | # Set the shadownet training epochs 22 | __C.TRAIN.EPOCHS = 90100 # 200010 23 | # Set the display step 24 | __C.TRAIN.DISPLAY_STEP = 1 25 | # Set the test display step during training process 26 | __C.TRAIN.TEST_DISPLAY_STEP = 1000 27 | # Set the momentum parameter of the optimizer 28 | __C.TRAIN.MOMENTUM = 0.9 29 | # Set the initial learning rate 30 | __C.TRAIN.LEARNING_RATE = 0.01 # 0.0005 31 | # Set the GPU resource used during training process 32 | __C.TRAIN.GPU_MEMORY_FRACTION = 0.85 33 | # Set the GPU allow growth parameter during tensorflow training process 34 | __C.TRAIN.TF_ALLOW_GROWTH = True 35 | # Set the shadownet training batch size 36 | __C.TRAIN.BATCH_SIZE = 8 # 4 37 | # Set the shadownet validation batch size 38 | __C.TRAIN.VAL_BATCH_SIZE = 8 # 4 39 | # Set the learning rate decay steps 40 | __C.TRAIN.LR_DECAY_STEPS = 210000 41 | # Set the learning rate decay rate 42 | __C.TRAIN.LR_DECAY_RATE = 0.1 43 | # Set the class numbers 44 | __C.TRAIN.CLASSES_NUMS = 2 45 | # Set the image height 46 | __C.TRAIN.IMG_HEIGHT = 288 # 256 47 | # Set the image width 48 | __C.TRAIN.IMG_WIDTH = 800 # 512 49 | # Set GPU number 50 | __C.TRAIN.GPU_NUM = 4 # 8 51 | # Set CPU thread number 52 | __C.TRAIN.CPU_NUM = 4 # 53 | 54 | # Test options 55 | __C.TEST = edict() 56 | 57 | # Set the GPU resource used during testing process 58 | __C.TEST.GPU_MEMORY_FRACTION = 0.8 59 | # Set the GPU allow growth parameter during tensorflow testing process 60 | __C.TEST.TF_ALLOW_GROWTH = True 61 | # Set the test batch size 62 | __C.TEST.BATCH_SIZE = 8 63 | # Set the test CPU thread number 64 | __C.TEST.CPU_NUM = 8 65 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/global_lane.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | # Define a class to receive the characteristics of each line detection 4 | class Line(): 5 | def __init__(self): 6 | # was the line detected in the last iteration? 7 | self.detected = False 8 | # x values of the last n fits of the line 9 | self.recent_xfitted = [] 10 | #average x values of the fitted line over the last n iterations 11 | self.bestx = np.zeros(590) # TODO This supposes that the image height is always 720 BAD but it was fixed with the init function 12 | #polynomial coefficients averaged over the last n iterations 13 | self.best_fit = np.zeros(3) 14 | #polynomial coefficients for the most recent fit 15 | self.current_fit = [np.array([False])] 16 | #radius of curvature of the line in some units 17 | self.radius_of_curvature = np.zeros(1) 18 | #distance in meters of vehicle center from the line 19 | self.line_base_pos = np.zeros(1) 20 | #difference in fit coefficients between last and new fits 21 | self.diffs = np.array([0,0,0], dtype='float') 22 | #x values for detected line pixels 23 | self.allx = None 24 | #y values for detected line pixels 25 | self.ally = None 26 | #smoothen the n frames 27 | self.smoothen_nframes = 10 28 | #first frame 29 | self.first_frame = True 30 | 31 | 32 | left_lane = Line() 33 | right_lane = Line() 34 | 35 | # TODO this function should be called once for stream of images(video) and every time for separate images 36 | def init_lines(width): 37 | global left_lane 38 | left_lane.bestx = np.zeros(width) 39 | # was the line detected in the last iteration? 40 | left_lane.detected = False 41 | # x values of the last n fits of the line 42 | left_lane.recent_xfitted = [] 43 | # polynomial coefficients averaged over the last n iterations 44 | left_lane.best_fit = np.zeros(3) 45 | # polynomial coefficients for the most recent fit 46 | left_lane.current_fit = [np.array([False])] 47 | # radius of curvature of the line in some units 48 | left_lane.radius_of_curvature = np.zeros(1) 49 | # distance in meters of vehicle center from the line 50 | left_lane.line_base_pos = np.zeros(1) 51 | # difference in fit coefficients between last and new fits 52 | left_lane.diffs = np.array([0, 0, 0], dtype='float') 53 | # x values for detected line pixels 54 | left_lane.allx = None 55 | # y values for detected line pixels 56 | left_lane.ally = None 57 | # smoothen the n frames 58 | left_lane.smoothen_nframes = 10 59 | # first frame 60 | left_lane.first_frame = True 61 | 62 | global right_lane 63 | right_lane.bestx = np.zeros(width) 64 | # was the line detected in the last iteration? 65 | right_lane.detected = False 66 | # x values of the last n fits of the line 67 | right_lane.recent_xfitted = [] 68 | # polynomial coefficients averaged over the last n iterations 69 | right_lane.best_fit = np.zeros(3) 70 | # polynomial coefficients for the most recent fit 71 | right_lane.current_fit = [np.array([False])] 72 | # radius of curvature of the line in some units 73 | right_lane.radius_of_curvature = np.zeros(1) 74 | # distance in meters of vehicle center from the line 75 | right_lane.line_base_pos = np.zeros(1) 76 | # difference in fit coefficients between last and new fits 77 | right_lane.diffs = np.array([0, 0, 0], dtype='float') 78 | # x values for detected line pixels 79 | right_lane.allx = None 80 | # y values for detected line pixels 81 | right_lane.ally = None 82 | # smoothen the n frames 83 | right_lane.smoothen_nframes = 10 84 | # first frame 85 | right_lane.first_frame = True 86 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/lanenet_data_processor_test.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Time : 18-5-11 下午4:58 4 | # @Author : Luo Yao 5 | # @Site : http://icode.baidu.com/repos/baidu/personal-code/Luoyao 6 | # @File : lanenet_data_processor.py 7 | # @IDE: PyCharm Community Edition 8 | """ 9 | 实现LaneNet的数据解析类 10 | """ 11 | import tensorflow as tf 12 | 13 | import global_config 14 | 15 | CFG = global_config.cfg 16 | VGG_MEAN = [123.68, 116.779, 103.939] 17 | 18 | 19 | class DataSet(object): 20 | """ 21 | 实现数据集类 22 | """ 23 | 24 | def __init__(self, dataset_info_file, batch_size): 25 | """ 26 | 27 | :param dataset_info_file: 28 | """ 29 | self._dataset_info_file = dataset_info_file 30 | self._batch_size = batch_size 31 | self._img_list = self._init_dataset() 32 | self._next_batch_loop_count = 0 33 | 34 | def __len__(self): 35 | return self._len 36 | 37 | def _init_dataset(self): 38 | """ 39 | :return: 40 | """ 41 | img_list = [] 42 | 43 | if not tf.gfile.Exists(self._dataset_info_file): 44 | raise ValueError('Failed to find file: ' + self._dataset_info_file) 45 | 46 | with open(self._dataset_info_file, 'r') as file: 47 | for _info in file: 48 | info_tmp = _info.strip(' ').split() 49 | img_list.append(info_tmp[0][:]) # changed this for windows path 50 | 51 | self._len = len(img_list) 52 | 53 | return img_list 54 | 55 | @staticmethod 56 | def process_img(img_path): 57 | img_raw = tf.read_file(img_path) 58 | img_decoded = tf.image.decode_jpeg(img_raw, channels=3) 59 | img_resized = tf.image.resize_images(img_decoded, [CFG.TRAIN.IMG_HEIGHT, CFG.TRAIN.IMG_WIDTH], 60 | method=tf.image.ResizeMethod.BICUBIC) 61 | img_casted = tf.cast(img_resized, tf.float32) 62 | return tf.subtract(img_casted, VGG_MEAN) 63 | 64 | def next_batch(self): 65 | """ 66 | :return: 67 | """ 68 | print(self._batch_size) 69 | idx_start = self._batch_size * self._next_batch_loop_count 70 | idx_end = self._batch_size * self._next_batch_loop_count + self._batch_size 71 | 72 | if idx_end > len(self): 73 | idx_end = len(self) 74 | 75 | img_list = self._img_list[idx_start:idx_end] 76 | self._next_batch_loop_count += 1 77 | return img_list 78 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/lanenet_merge_model.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Time : 18-5-11 下午5:28 4 | # @Author : Luo Yao 5 | # @Site : http://icode.baidu.com/repos/baidu/personal-code/Luoyao 6 | # @File : lanenet_merge_model.py 7 | # @IDE: PyCharm Community Edition 8 | """ 9 | Build Lane detection model 10 | """ 11 | import tensorflow as tf 12 | 13 | import vgg_encoder 14 | import cnn_basenet 15 | import global_config 16 | 17 | CFG = global_config.cfg 18 | 19 | 20 | class LaneNet(cnn_basenet.CNNBaseModel): 21 | """ 22 | Lane detection model 23 | """ 24 | 25 | @staticmethod 26 | def inference(input_tensor, phase, name): 27 | """ 28 | feed forward 29 | :param name: 30 | :param input_tensor: 31 | :param phase: 32 | :return: 33 | """ 34 | with tf.variable_scope(name): 35 | with tf.variable_scope('inference'): 36 | encoder = vgg_encoder.VGG16Encoder(phase=phase) 37 | encode_ret = encoder.encode(input_tensor=input_tensor, name='encode') 38 | 39 | return encode_ret 40 | 41 | @staticmethod 42 | def test_inference(input_tensor, phase, name): 43 | inference_ret = LaneNet.inference(input_tensor, phase, name) 44 | with tf.variable_scope(name): 45 | # feed forward to obtain logits 46 | # Compute loss 47 | 48 | decode_logits = inference_ret['prob_output'] 49 | binary_seg_ret = tf.nn.softmax(logits=decode_logits) 50 | prob_list = [] 51 | kernel = tf.get_variable('kernel', [9, 9, 1, 1], initializer=tf.constant_initializer(1.0 / 81), 52 | trainable=False) 53 | 54 | with tf.variable_scope("convs_smooth"): 55 | prob_smooth = tf.nn.conv2d(tf.cast(tf.expand_dims(binary_seg_ret[:, :, :, 0], axis=3), tf.float32), 56 | kernel, [1, 1, 1, 1], 'SAME') 57 | prob_list.append(prob_smooth) 58 | 59 | for cnt in range(1, binary_seg_ret.get_shape().as_list()[3]): 60 | with tf.variable_scope("convs_smooth", reuse=True): 61 | prob_smooth = tf.nn.conv2d( 62 | tf.cast(tf.expand_dims(binary_seg_ret[:, :, :, cnt], axis=3), tf.float32), kernel, [1, 1, 1, 1], 63 | 'SAME') 64 | prob_list.append(prob_smooth) 65 | processed_prob = tf.stack(prob_list, axis=4) 66 | processed_prob = tf.squeeze(processed_prob, axis=3) 67 | binary_seg_ret = processed_prob 68 | 69 | # Predict lane existence: 70 | existence_logit = inference_ret['existence_output'] 71 | existence_output = tf.nn.sigmoid(existence_logit) 72 | 73 | return binary_seg_ret, existence_output 74 | 75 | @staticmethod 76 | def loss(inference, binary_label, existence_label, name): 77 | """ 78 | :param name: 79 | :param inference: 80 | :param existence_label: 81 | :param binary_label: 82 | :return: 83 | """ 84 | # feed forward to obtain logits 85 | 86 | with tf.variable_scope(name): 87 | 88 | inference_ret = inference 89 | 90 | # Compute the segmentation loss 91 | 92 | decode_logits = inference_ret['prob_output'] 93 | decode_logits_reshape = tf.reshape( 94 | decode_logits, 95 | shape=[decode_logits.get_shape().as_list()[0], 96 | decode_logits.get_shape().as_list()[1] * decode_logits.get_shape().as_list()[2], 97 | decode_logits.get_shape().as_list()[3]]) 98 | 99 | binary_label_reshape = tf.reshape( 100 | binary_label, 101 | shape=[binary_label.get_shape().as_list()[0], 102 | binary_label.get_shape().as_list()[1] * binary_label.get_shape().as_list()[2]]) 103 | binary_label_reshape = tf.one_hot(binary_label_reshape, depth=5) 104 | class_weights = tf.constant([[0.4, 1.0, 1.0, 1.0, 1.0]]) 105 | weights_loss = tf.reduce_sum(tf.multiply(binary_label_reshape, class_weights), 2) 106 | binary_segmentation_loss = tf.losses.softmax_cross_entropy(onehot_labels=binary_label_reshape, 107 | logits=decode_logits_reshape, 108 | weights=weights_loss) 109 | binary_segmentation_loss = tf.reduce_mean(binary_segmentation_loss) 110 | 111 | # Compute the sigmoid loss 112 | 113 | existence_logits = inference_ret['existence_output'] 114 | existence_loss = tf.nn.sigmoid_cross_entropy_with_logits(labels=existence_label, logits=existence_logits) 115 | existence_loss = tf.reduce_mean(existence_loss) 116 | 117 | # Compute the overall loss 118 | 119 | total_loss = binary_segmentation_loss + 0.1 * existence_loss 120 | ret = { 121 | 'total_loss': total_loss, 122 | 'instance_seg_logits': decode_logits, 123 | 'instance_seg_loss': binary_segmentation_loss, 124 | 'existence_logits': existence_logits, 125 | 'existence_pre_loss': existence_loss 126 | } 127 | 128 | tf.add_to_collection('total_loss', total_loss) 129 | tf.add_to_collection('instance_seg_logits', decode_logits) 130 | tf.add_to_collection('instance_seg_loss', binary_segmentation_loss) 131 | tf.add_to_collection('existence_logits', existence_logits) 132 | tf.add_to_collection('existence_pre_loss', existence_loss) 133 | 134 | return ret 135 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/main.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Sat May 30 22:31:10 2020 4 | 5 | @author: Mouiad 6 | """ 7 | 8 | import numpy as np 9 | import matplotlib.pyplot as plt 10 | import matplotlib.image as mpimg 11 | import os 12 | import re 13 | import getLane 14 | 15 | # Code to generate lane coordinates from probablity maps. 16 | 17 | # Experiment name 18 | exp = "vgg_SCNN_DULR_w9" 19 | # Data root 20 | data = "../data/CULane" 21 | # Directory where prob imgs generated by CNN are saved. 22 | probRoot = "../experiments/predicts/" + exp 23 | # Directory to save fitted lanes. 24 | output = "../prob2lines/output/" + exp 25 | 26 | testList = data + "/list/test.txt" # TODO edited the text file 27 | show = True # set to true to visualize 28 | 29 | 30 | with open(testList) as f: 31 | image_list = f.readlines() 32 | 33 | 34 | image_list = [x.strip() for x in image_list] 35 | #print(image_list) 36 | num = len(image_list) 37 | pts = 18 38 | 39 | for i in range(num): 40 | if np.mod(i + 1, 100) == 0: 41 | print(('Processing the %d th image...\\n'), i) 42 | imname = image_list[i] 43 | existPath = probRoot + imname[:-3] + "exist.txt" 44 | 45 | with open(existPath) as f: 46 | exist = f.readlines() 47 | exist = [y for x in exist for y in x if y == '1' or y == '0'] 48 | # TODO better to show with opencv 49 | coordinates = np.zeros((4, pts)) 50 | for j in range(4): 51 | if exist[j] == '1': 52 | scorePath = probRoot + imname[:-4] + "_" + str(j + 1) + "_avg.png" 53 | scoreMap = mpimg.imread(scorePath) * 255.0 54 | coordinate = getLane.getLane(scoreMap) 55 | coordinates[j, :] = coordinate 56 | if show: 57 | img = mpimg.imread(data + imname) 58 | probMaps = np.uint8(np.zeros((288, 800, 3))) # TODO this needs to change for the right size 59 | plt.imshow(img) 60 | for k in range(4): 61 | color = ['g', 'b', 'r', 'y'] 62 | if exist[k] == '1': 63 | for m in range(pts): # The devil is in the details m and k and -1 64 | if coordinates[k, m] > 0: # plotting! 65 | plt.plot(np.uint16(np.round(coordinates[k, m] * 1640.0 / 800.0)) - 1, np.uint16(np.round(590 - m * 20)) - 1, "." + color[k], "markersize", 30) 66 | probPath = probRoot + imname[:-4] + "_" + str(k + 1) + "_avg.png" 67 | probMap = mpimg.imread(probPath) * 255 # The mod 68 | probMaps[:, :, np.mod(k + 1, 3)] = probMaps[:, :, np.mod(k + 1, 3)] + probMap 69 | #plt.show() 70 | #plt.imshow(probMaps) 71 | plt.show() 72 | else: 73 | save_name = output + imname[:- 3] + "lines.txt" 74 | position = [m.start() for m in re.finditer('/', save_name)] 75 | prefix = '' 76 | if position: 77 | prefix = save_name[:position[-1]] 78 | if not os.path.exists(prefix) and prefix != ' ': 79 | os.makedirs(prefix) 80 | fp = open(save_name, "w") 81 | for k in range(4): 82 | if exist[k] == '1' and np.sum(coordinates[k, :] > 0) > 1: 83 | for m in range(pts): 84 | if coordinates[k, m] > 0: 85 | fp.write(str(np.uint16(np.round(coordinates[k, m] * 1640.0 / 800.0)) - 1) + ' ' + str(np.uint16(np.round(590 - m * 20)) - 1) + ' ') 86 | #print(np.uint16(np.round(coordinates[k, m] * 1640.0 / 800.0)) - 1, np.uint16(np.round(590 - m * 20)) - 1) 87 | fp.write('\n') 88 | fp.close() 89 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/main1.py: -------------------------------------------------------------------------------- 1 | # -*- coding: utf-8 -*- 2 | """ 3 | Created on Sat May 30 22:31:10 2020 4 | 5 | @author: Mouiad 6 | """ 7 | 8 | import numpy as np 9 | import matplotlib.pyplot as plt 10 | import matplotlib.image as mpimg 11 | import os 12 | import re 13 | import getLane 14 | import cv2 15 | # Code to generate lane coordinates from probablity maps. 16 | def save(original,path22): 17 | mask = np.zeros_like(original).astype(np.unicode) 18 | f = open(path22, "r") 19 | for line in f : 20 | x = [] 21 | y = [] 22 | points = line.split(" ") 23 | for i in range(0,(len(points)-1),2): 24 | x.append(points[i]) 25 | y.append(points[i+1]) 26 | 27 | for i in range(0 , len(x)-1): 28 | #print(x[i] , y[i]) 29 | cv2.line(original, (int(float(x[i])), int(y[i])),(int(float(x[i+1])), int(y[i+1])) , (0,255,0), 8) 30 | #print("--------------------------------------------------------------------------") 31 | 32 | return original 33 | #cv2.waitKey(0) 34 | # Experiment name 35 | exp = "vgg_SCNN_DULR_w9" 36 | # Data root 37 | data = "../data/CULane" 38 | # Directory where prob imgs generated by CNN are saved. 39 | probRoot = "../experiments/predicts/" + exp 40 | # Directory to save fitted lanes. 41 | output = "../prob2lines/output/" + exp 42 | #print(output) 43 | testList = data + "/list/test.txt" # TODO edited the text file 44 | show = False # set to true to visualize 45 | #fil=open(r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\data\CULane\video_out.txt",'w') 46 | with open(testList) as f: 47 | image_list = f.readlines() 48 | 49 | count=0 50 | with open(testList,'r') as f: 51 | for line in f: 52 | count+=1 53 | #print("Total number of line is: "+str(count)) 54 | image_list = [x.strip() for x in image_list] 55 | imname=image_list[0] 56 | img = mpimg.imread(data + imname) 57 | #print(img) 58 | im=imname[-20:-14] 59 | global path11 60 | path11=r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\Visual_output\Dl\video\\"+im 61 | try: 62 | os.makedirs(path11) 63 | #print("Directory ",path," Created") 64 | except FileExistsError: 65 | pass 66 | #print("Directory ",path," already exists") 67 | num = len(image_list) 68 | pts = 18 69 | for i in range(num): 70 | if np.mod(i + 1, 100) == 0: 71 | print(('Processing the %d th image...\\n'), i) 72 | imname = image_list[i] 73 | existPath = probRoot + imname[:-3] + "exist.txt" 74 | 75 | with open(existPath) as f: 76 | 77 | exist = f.readlines() 78 | exist = [y for x in exist for y in x if y == '1' or y == '0'] 79 | # TODO better to show with opencv 80 | coordinates = np.zeros((4, pts)) 81 | for j in range(4): 82 | if exist[j] == '1': 83 | scorePath = probRoot + imname[:-4] + "_" + str(j + 1) + "_avg.png" 84 | scoreMap = mpimg.imread(scorePath) * 255.0 85 | coordinate = getLane.getLane(scoreMap) 86 | coordinates[j, :] = coordinate 87 | if show: 88 | img = mpimg.imread(data + imname) 89 | image_name=imname[-8:] 90 | #im=imname[-20:-14] 91 | 92 | #print(im) 93 | #print(image_name) 94 | probMaps = np.uint8(np.zeros((288, 800, 3))) # TODO this needs to change for the right size 95 | 96 | for k in range(4): 97 | color = ['g', 'b', 'r', 'y'] 98 | if exist[k] == '1': 99 | for m in range(pts): # The devil is in the details m and k and -1 100 | if coordinates[k, m] > 0: # plotting! 101 | plt.plot(np.uint16(np.round(coordinates[k, m] * 1640.0 / 800.0)) - 1, np.uint16(np.round(590 - m * 20)) - 1, "." + color[k], "markersize", 30) 102 | probPath = probRoot + imname[:-4] + "_" + str(k + 1) + "_avg.png" 103 | name=probRoot + imname[:-4] + "_" + str(k + 1) 104 | probMap = mpimg.imread(probPath) * 255 # The mod 105 | probMaps[:, :, np.mod(k + 1, 3)] = probMaps[:, :, np.mod(k + 1, 3)] + probMap 106 | #plt.figure(figsize = (10,5)) 107 | if j==3 and count==1: 108 | 109 | plt.imshow(img) 110 | plt.savefig(r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\Visual_output\Dl\image\\"+image_name,bbox_inches='tight',pad_inches=-0.4) 111 | elif j==3 and count>1: 112 | plt.imshow(img) 113 | plt.savefig(path11+"/"+image_name,bbox_inches='tight',pad_inches=-0.4) 114 | #fil.write(r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\Visual_output\Dl\video\\"+image_name) 115 | #fil.write("\n") 116 | plt.close() 117 | 118 | else: 119 | img = mpimg.imread(data + imname) 120 | image_name=imname[-8:] 121 | save_name = output + imname[:- 3] + "lines.txt" 122 | position = [m.start() for m in re.finditer('/', save_name)] 123 | prefix = '' 124 | if position: 125 | prefix = save_name[:position[-1]] 126 | if not os.path.exists(prefix) and prefix != ' ': 127 | os.makedirs(prefix) 128 | fp = open(save_name, "w") 129 | for k in range(4): 130 | if exist[k] == '1' and np.sum(coordinates[k, :] > 0) > 1: 131 | for m in range(pts): 132 | if coordinates[k, m] > 0: 133 | fp.write(str(np.uint16(np.round(coordinates[k, m] * 1640.0 / 800.0)) - 1) + ' ' + str(np.uint16(np.round(590 - m * 20)) - 1) + ' ') 134 | #print(np.uint16(np.round(coordinates[k, m] * 1640.0 / 800.0)) - 1, np.uint16(np.round(590 - m * 20)) - 1) 135 | fp.write('\n') 136 | fp.close() 137 | if j==3 and count==1: 138 | ime=save(img,save_name) 139 | plt.imsave(r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\Visual_output\Dl\image\\"+image_name,ime) 140 | cv2.imshow("image" , ime) 141 | elif j==3 and count>1: 142 | ime=save(img,save_name) 143 | #cv2.imwrite(path11+"/"+image_name,ime) 144 | plt.imsave(path11+"/"+image_name,ime) 145 | def pat(): 146 | return path11 -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/perspective_regionofint_main.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import cv2 3 | import matplotlib.pyplot as plt 4 | import matplotlib.image as mpimg 5 | 6 | #perspective transform on undistorted images 7 | def perspective_transform(img): 8 | imshape = img.shape 9 | #vertices = np.array([[(.5281*imshape[1], .6586*imshape[0]), (.5619*imshape[1],.8213*imshape[0]), 10 | #(.376*imshape[1],.8213*imshape[0]),(.4251*imshape[1], .6586*imshape[0])]], dtype=np.float32) 11 | vertices = np.array([[(.30731 * imshape[1], .77627 * imshape[0]), (.63841 * imshape[1], .75762 * imshape[0]), 12 | (.42743 * imshape[1], .55932 * imshape[0]), (.49695 * imshape[1], .55593 * imshape[0])]], dtype=np.float32) 13 | src = np.float32(vertices) 14 | #dst = np.array([[(.5281*imshape[1], .6586*imshape[0]), (.5281*imshape[1],.8213*imshape[0]), 15 | #(.376*imshape[1],.8213*imshape[0]),(.376*imshape[1], .6586*imshape[0])]], dtype=np.float32) 16 | dst = np.array([[(.2737*imshape[1], .8711*imshape[0]), (.6841*imshape[1],.8254*imshape[0]), 17 | (.2737*imshape[1],.03379*imshape[0]),(.7*imshape[1], .033*imshape[0])]], dtype=np.float32) 18 | M = cv2.getPerspectiveTransform(src, dst) 19 | Minv = cv2.getPerspectiveTransform(dst, src) 20 | img_size = (imshape[1], imshape[0]) 21 | perspective_img = cv2.warpPerspective(img, M, img_size, flags = cv2.INTER_LINEAR) 22 | return perspective_img, Minv 23 | 24 | #region of interest 25 | def region_of_interest(img, vertices): 26 | #defining a blank mask to start with 27 | mask = np.zeros_like(img, dtype=np.uint8) 28 | if len(img.shape) > 2: 29 | channel_count = img.shape[2] # depending on out image 30 | ignore_mask_color = (255,) * channel_count 31 | else: 32 | ignore_mask_color = 255 33 | cv2.fillPoly(mask, vertices, ignore_mask_color) 34 | mask=cv2.imread(r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\Settings\mask.png") 35 | masked_image = cv2.bitwise_and(img, mask) 36 | return masked_image 37 | 38 | 39 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/sliding_main.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import cv2 3 | import matplotlib.pyplot as plt 4 | import matplotlib.image as mpimg 5 | from global_lane import * 6 | 7 | def for_sliding_window(binary_warped): 8 | # Assuming you have created a warped binary image called "binary_warped" 9 | # Take a histogram of the bottom half of the image 10 | histogram = np.sum(binary_warped[binary_warped.shape[0]//2:,:], axis=0) 11 | # Create an output image to draw on and visualize the result 12 | out_img = np.dstack((binary_warped, binary_warped, binary_warped))*255 13 | # Find the peak of the left and right halves of the histogram 14 | # These will be the starting point for the left and right lines 15 | midpoint = np.int(histogram.shape[0]/2) 16 | leftx_base = np.argmax(histogram[:midpoint]) 17 | rightx_base = np.argmax(histogram[midpoint:]) + midpoint 18 | 19 | # Choose the number of sliding windows 20 | nwindows = 15 21 | # Set height of windows 22 | window_height = np.int(binary_warped.shape[0]/nwindows) 23 | # Identify the x and y positions of all nonzero pixels in the image 24 | nonzero = binary_warped.nonzero() 25 | nonzeroy = np.array(nonzero[0]) 26 | nonzerox = np.array(nonzero[1]) 27 | # Current positions to be updated for each window 28 | leftx_current = leftx_base 29 | rightx_current = rightx_base 30 | # Set the width of the windows +/- margin 31 | margin = 75 32 | # Set minimum number of pixels found to recenter window 33 | minpix = 100 34 | # Create empty lists to receive left and right lane pixel indices 35 | left_lane_inds = [] 36 | right_lane_inds = [] 37 | 38 | # Step through the windows one by one 39 | for window in range(nwindows): 40 | # Identify window boundaries in x and y (and right and left) 41 | win_y_low = binary_warped.shape[0] - (window+1)*window_height 42 | win_y_high = binary_warped.shape[0] - window*window_height 43 | win_xleft_low = leftx_current - margin 44 | win_xleft_high = leftx_current + margin 45 | win_xright_low = rightx_current - margin 46 | win_xright_high = rightx_current + margin 47 | # Draw the windows on the visualization image 48 | #cv2.rectangle(out_img,(win_xleft_low,win_y_low),(win_xleft_high,win_y_high),(0,255,0), 2) 49 | #cv2.rectangle(out_img,(win_xright_low,win_y_low),(win_xright_high,win_y_high),(0,255,0), 2) 50 | # Identify the nonzero pixels in x and y within the window 51 | good_left_inds = ((nonzeroy >= win_y_low) & (nonzeroy < win_y_high) & (nonzerox >= win_xleft_low) & (nonzerox < win_xleft_high)).nonzero()[0] 52 | good_right_inds = ((nonzeroy >= win_y_low) & (nonzeroy < win_y_high) & (nonzerox >= win_xright_low) & (nonzerox < win_xright_high)).nonzero()[0] 53 | # Append these indices to the lists 54 | left_lane_inds.append(good_left_inds) 55 | right_lane_inds.append(good_right_inds) 56 | # If you found > minpix pixels, recenter next window on their mean position 57 | if len(good_left_inds) > minpix: 58 | leftx_current = np.int(np.mean(nonzerox[good_left_inds])) 59 | if len(good_right_inds) > minpix: 60 | rightx_current = np.int(np.mean(nonzerox[good_right_inds])) 61 | 62 | # Concatenate the arrays of indices 63 | left_lane_inds = np.concatenate(left_lane_inds) 64 | right_lane_inds = np.concatenate(right_lane_inds) 65 | 66 | # Extract left and right line pixel positions 67 | leftx = nonzerox[left_lane_inds] 68 | lefty = nonzeroy[left_lane_inds] 69 | rightx = nonzerox[right_lane_inds] 70 | righty = nonzeroy[right_lane_inds] 71 | #check for any lanes that are not detected in this current frame then use the history 72 | if (leftx.size < 5): 73 | left_lane.detected = False 74 | #print ("Left lane deteceted - False") 75 | else: 76 | left_lane.detected = True 77 | #print ("Left lane detected - true") 78 | 79 | if (rightx.size < 5): 80 | right_lane.detected = False 81 | #print ("Right lane detected False") 82 | else: 83 | right_lane.detected = True 84 | #print ("Right lane detected True") 85 | 86 | #print (left_lane.detected, right_lane.detected) 87 | #if lane is detected then try to fit the poly 88 | if left_lane.detected == True & right_lane.detected == True: 89 | # Fit a second order polynomial to each 90 | left_fit = np.polyfit(lefty, leftx, 2) 91 | right_fit = np.polyfit(righty, rightx, 2) 92 | left_lane.best_fit = np.vstack([left_lane.best_fit,left_fit]) 93 | left_lane.best_fit[0] = left_fit 94 | right_lane.best_fit = np.vstack([right_lane.best_fit,right_fit]) 95 | right_lane.best_fit[0] = right_fit 96 | left_lane.best_fit = np.average(left_lane.best_fit[-left_lane.smoothen_nframes:], axis = 0) 97 | right_lane.best_fit = np.average(right_lane.best_fit[-right_lane.smoothen_nframes:], axis = 0) 98 | #print ("saved best fit") 99 | else: 100 | #use the history avg values 101 | left_fit = left_lane.best_fit 102 | right_fit = right_lane.best_fit 103 | #print ("used previous best fit") 104 | #calculate the actual points in x and y is from 0 to 589 105 | ploty = np.linspace(0, out_img.shape[0]-1, out_img.shape[0]) 106 | left_fitx = left_fit[0]*ploty**2 + left_fit[1]*ploty + left_fit[2] 107 | right_fitx = right_fit[0]*ploty**2 + right_fit[1]*ploty + right_fit[2] 108 | 109 | #print (left_lane.first_frame) 110 | #Take the average here of the last n frames 111 | if left_lane.first_frame == True: 112 | left_lane.first_frame = False 113 | left_lane.bestx = np.vstack([left_lane.bestx,left_fitx]) 114 | left_lane.bestx[0] = left_fitx 115 | #print ("Inside first frame") 116 | 117 | # TODO looks bad Debug it 118 | if ((left_fitx[0] > right_fitx[0]) | (abs(left_fitx[0] - right_fitx[0])<350) | (abs(left_fitx[0] - right_fitx[0])>700) | 119 | (left_fitx[0] > 1640 )): 120 | left_lane.bestx = np.vstack([left_lane.bestx,left_lane.bestx]) 121 | #print ("Skip lane left") 122 | else: 123 | left_lane.bestx = np.vstack([left_lane.bestx,left_fitx]) 124 | #print ("read lane left") 125 | 126 | left_lane.bestx = np.average(left_lane.bestx[-left_lane.smoothen_nframes:], axis = 0) 127 | 128 | if right_lane.first_frame == True: 129 | right_lane.first_frame = False 130 | right_lane.bestx = np.vstack([right_lane.bestx,right_fitx]) 131 | right_lane.bestx[0] = right_fitx 132 | #print ("Inside first frame") 133 | 134 | # TODO same as above 135 | if ((left_fitx[0] > right_fitx[0]) | (abs(left_fitx[0] - right_fitx[0])<350) | (abs(left_fitx[0] - right_fitx[0])>700) | 136 | (right_fitx[0] > 1640)): 137 | right_lane.bestx = np.vstack([right_lane.bestx,right_lane.bestx]) 138 | #print ("Skip lane right") 139 | else: 140 | right_lane.bestx = np.vstack([right_lane.bestx,right_fitx]) 141 | #print ("read lane right") 142 | 143 | right_lane.bestx = np.average(right_lane.bestx[-right_lane.smoothen_nframes:], axis = 0) 144 | 145 | #print (left_lane.bestx[0]) 146 | #print (left_fitx[0]) 147 | #print (right_lane.bestx[0]) 148 | #print (right_fitx[0]) 149 | 150 | 151 | window_img = np.zeros_like(out_img) 152 | margin = 15 153 | 154 | out_img[nonzeroy[left_lane_inds], nonzerox[left_lane_inds]] = [255, 0, 255] 155 | out_img[nonzeroy[right_lane_inds], nonzerox[right_lane_inds]] = [255, 0, 255] 156 | # Generate a polygon to illustrate the SMOOTHENED FIT 157 | # And recast the x and y points into usable format for cv2.fillPoly() 158 | left_line_window1 = np.array([np.transpose(np.vstack([left_lane.bestx-margin, ploty]))]) 159 | left_line_window2 = np.array([np.flipud(np.transpose(np.vstack([left_lane.bestx+margin, ploty])))]) 160 | left_line_pts = np.hstack((left_line_window1, left_line_window2)) 161 | right_line_window1 = np.array([np.transpose(np.vstack([right_lane.bestx-margin, ploty]))]) 162 | right_line_window2 = np.array([np.flipud(np.transpose(np.vstack([right_lane.bestx+margin, ploty])))]) 163 | right_line_pts = np.hstack((right_line_window1, right_line_window2)) 164 | 165 | # Draw the lane onto the warped blank image 166 | cv2.fillPoly(window_img, np.int_([left_line_pts]), (0,255, 0)) 167 | cv2.fillPoly(window_img, np.int_([right_line_pts]), (0,255, 0)) 168 | result = cv2.addWeighted(out_img, 1, window_img, 1, 0) 169 | 170 | # Generate a polygon to illustrate the CURRENT FRAME FIT 171 | # And recast the x and y points into usable format for cv2.fillPoly() 172 | left_line_window1 = np.array([np.transpose(np.vstack([left_fitx-margin, ploty]))]) 173 | left_line_window2 = np.array([np.flipud(np.transpose(np.vstack([left_fitx+margin, ploty])))]) 174 | left_line_pts = np.hstack((left_line_window1, left_line_window2)) 175 | right_line_window1 = np.array([np.transpose(np.vstack([right_fitx-margin, ploty]))]) 176 | right_line_window2 = np.array([np.flipud(np.transpose(np.vstack([right_fitx+margin, ploty])))]) 177 | right_line_pts = np.hstack((right_line_window1, right_line_window2)) 178 | 179 | # Draw the lane onto the warped blank image 180 | cv2.fillPoly(window_img, np.int_([left_line_pts]), (0,0, 255)) 181 | cv2.fillPoly(window_img, np.int_([right_line_pts]), (0,0, 255)) 182 | result = cv2.addWeighted(out_img, 1, window_img, 1, 0) 183 | 184 | # TODO FIX 185 | # TODO the 30 and 3.7 while not image dependant are road dependant see Measuring Curvature II for more details 186 | # TODO it looks like perspective transform might have an effect 187 | #if the lane was detectded then calculate the curvatire or use the history 188 | if (leftx.size > 2 | rightx.size > 2) : 189 | y_eval = np.max(ploty) 190 | # Define conversions in x and y from pixels space to meters 191 | # ym_per_pix = 30/720 # meters per pixel in y dimension 192 | ym_per_pix = 16 / binary_warped.shape[0] 193 | xm_per_pix = 3.7/590 # meters per pixel in x dimension was 700 194 | 195 | # Fit new polynomials to x,y in world space 196 | left_fit_cr = np.polyfit(lefty*ym_per_pix, leftx*xm_per_pix, 2) 197 | right_fit_cr = np.polyfit(righty*ym_per_pix, rightx*xm_per_pix, 2) 198 | # Calculate the new radii of curvature 199 | left_curverad = ((1 + (2*left_fit_cr[0]*y_eval*ym_per_pix + left_fit_cr[1])**2)**1.5) / np.absolute(2*left_fit_cr[0]) 200 | right_curverad = ((1 + (2*right_fit_cr[0]*y_eval*ym_per_pix + right_fit_cr[1])**2)**1.5) / np.absolute(2*right_fit_cr[0]) 201 | # Now our radius of curvature is in meters and dist from centre 202 | lane_centre = (left_fitx[-1] + right_fitx[-1])/2.0 203 | camera_centre = result.shape[1]/2.0 204 | # TODO that pesky 700 again 205 | dist_centre_val = (lane_centre - camera_centre)*3.7/400 # was 700 206 | avg_cur = (right_curverad+left_curverad)/2.0 207 | 208 | left_lane.line_base_pos = np.vstack([left_lane.line_base_pos,dist_centre_val]) 209 | left_lane.line_base_pos[0] = dist_centre_val 210 | left_lane.line_base_pos = np.average(left_lane.line_base_pos[-left_lane.smoothen_nframes:], axis = 0) 211 | 212 | left_lane.radius_of_curvature = np.vstack([left_lane.radius_of_curvature,avg_cur]) 213 | left_lane.radius_of_curvature[0] = avg_cur 214 | left_lane.radius_of_curvature = np.average(left_lane.radius_of_curvature[-left_lane.smoothen_nframes:], axis = 0) 215 | 216 | #print (avg_cur, dist_centre_val) 217 | # else use the history curvature 218 | else: 219 | dist_centre_val = left_lane.line_base_pos 220 | avg_cur = left_lane.radius_of_curvature 221 | 222 | #reset the lane detected to false for the next frame 223 | left_lane.detected == False 224 | right_lane.detected == False 225 | return result, left_lane.bestx, right_lane.bestx, ploty, left_lane.radius_of_curvature, left_lane.line_base_pos -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/test_lanenet.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Time : 18-5-23 上午11:33 4 | # @Author : Luo Yao 5 | # @Site : http://icode.baidu.com/repos/baidu/personal-code/Luoyao 6 | # @File : test_lanenet.py 7 | # @IDE: PyCharm Community Edition 8 | """ 9 | 测试LaneNet模型 10 | """ 11 | 12 | ''' 13 | I have edited the test_img.txt and the CULane dataset test.txt files to change the image path 14 | also edited main.m script and removed the first ../ 15 | ''' 16 | import os 17 | import os.path as ops 18 | import argparse 19 | import math 20 | import tensorflow as tf 21 | import glog as log 22 | import cv2 23 | 24 | 25 | import lanenet_merge_model 26 | import global_config 27 | import lanenet_data_processor_test 28 | 29 | 30 | CFG = global_config.cfg 31 | VGG_MEAN = [103.939, 116.779, 123.68] 32 | 33 | 34 | def init_args(): 35 | """ 36 | 37 | :return: 38 | """ 39 | parser = argparse.ArgumentParser() 40 | parser.add_argument('--image_path', type=str, help='The image path or the src image save dir') 41 | parser.add_argument('--weights_path', type=str, help='The model weights path') 42 | parser.add_argument('--is_batch', type=str, help='If test a batch of images', default='false') 43 | parser.add_argument('--batch_size', type=int, help='The batch size of the test images', default=8) 44 | parser.add_argument('--save_dir', type=str, help='Test result image save dir', default=None) 45 | parser.add_argument('--use_gpu', type=int, help='If use gpu set 1 or 0 instead', default=1) 46 | return parser.parse_args() 47 | 48 | 49 | 50 | def test_lanenet(image_path, weights_path, use_gpu, image_list, batch_size, save_dir): 51 | 52 | """ 53 | :param image_path: 54 | :param weights_path: 55 | :param use_gpu: 56 | :return: 57 | """ 58 | test_dataset = lanenet_data_processor_test.DataSet(image_path, batch_size) 59 | input_tensor = tf.placeholder(dtype=tf.string, shape=[None], name='input_tensor') 60 | imgs = tf.map_fn(test_dataset.process_img, input_tensor, dtype=tf.float32) 61 | phase_tensor = tf.constant('test', tf.string) 62 | 63 | net = lanenet_merge_model.LaneNet() 64 | binary_seg_ret, instance_seg_ret = net.test_inference(imgs, phase_tensor, 'lanenet_loss') 65 | initial_var = tf.global_variables() 66 | final_var = initial_var[:-1] 67 | saver = tf.train.Saver(final_var) 68 | # Set sess configuration 69 | if use_gpu: 70 | sess_config = tf.ConfigProto(device_count={'GPU': 1}) 71 | else: 72 | sess_config = tf.ConfigProto(device_count={'GPU': 0}) 73 | sess_config.gpu_options.per_process_gpu_memory_fraction = CFG.TEST.GPU_MEMORY_FRACTION 74 | sess_config.gpu_options.allow_growth = CFG.TRAIN.TF_ALLOW_GROWTH 75 | sess_config.gpu_options.allocator_type = 'BFC' 76 | sess = tf.Session(config=sess_config) 77 | with sess.as_default(): 78 | sess.run(tf.global_variables_initializer()) 79 | saver.restore(sess=sess, save_path=weights_path) 80 | for i in range(math.ceil(len(image_list) / batch_size)): 81 | print(i) 82 | paths = test_dataset.next_batch() 83 | instance_seg_image, existence_output = sess.run([binary_seg_ret, instance_seg_ret], 84 | feed_dict={input_tensor: paths}) 85 | for cnt, image_name in enumerate(paths): 86 | print(image_name) 87 | # TODO converted for windows 88 | parent_path = os.path.dirname(image_name)[15:] 89 | parent_path = parent_path.replace('/', '\\') 90 | directory = os.path.join(save_dir, 'vgg_SCNN_DULR_w9', parent_path) 91 | #print(directory) 92 | # directory = os.path.join(save_dir, 'vgg_SCNN_DULR_w9') 93 | if not os.path.exists(directory): 94 | os.makedirs(directory) 95 | file_exist = open(os.path.join(directory, os.path.basename(image_name)[:-3] + 'exist.txt'), 'w') 96 | for cnt_img in range(4): 97 | cv2.imwrite(os.path.join(directory, os.path.basename(image_name)[:-4] + '_' + str(cnt_img + 1) + '_avg.png'), 98 | (instance_seg_image[cnt, :, :, cnt_img + 1] * 255).astype(int)) 99 | if existence_output[cnt, cnt_img] > 0.5: 100 | file_exist.write('1 ') 101 | else: 102 | file_exist.write('0 ') 103 | file_exist.close() 104 | sess.close() 105 | return 106 | 107 | def init(): 108 | #if __name__ == '__main__': 109 | # init args 110 | args = init_args() 111 | 112 | if args.save_dir is not None and not ops.exists(args.save_dir): 113 | log.error('{:s} not exist and has been made'.format(args.save_dir)) 114 | os.makedirs(args.save_dir) 115 | 116 | save_dir = os.path.join(args.image_path, 'predicts') 117 | if args.save_dir is not None: 118 | save_dir = args.save_dir 119 | 120 | img_name = [] 121 | with open(str(args.image_path), 'r') as g: 122 | for line in g.readlines(): 123 | img_name.append(line.strip()) 124 | 125 | test_lanenet(args.image_path, args.weights_path, args.use_gpu, img_name, args.batch_size, save_dir) 126 | #cv2.waitKey() -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/thresholding_main.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import cv2 3 | import matplotlib.pyplot as plt 4 | import matplotlib.image as mpimg 5 | import glob 6 | import pickle 7 | from perspective_regionofint_main import * 8 | from skimage import img_as_ubyte 9 | import json 10 | #grad threshold sobel x/y 11 | # Define a function that takes an image, gradient orientation, kernel 12 | # and threshold min / max values. 13 | def abs_sobel_thresh(img, orient='x', sobel_kernel = 3, thresh = (0,255)): 14 | if orient == 'x': 15 | abs_sobel = np.absolute(cv2.Sobel(img, cv2.CV_64F, 1, 0, ksize = sobel_kernel)) 16 | if orient == 'y': 17 | abs_sobel = np.absolute(cv2.Sobel(img, cv2.CV_64F, 0, 1, ksize = sobel_kernel)) 18 | scaled_sobel = np.uint8(255*abs_sobel/np.max(abs_sobel)) 19 | binary_output = np.zeros_like(scaled_sobel) 20 | binary_output[(scaled_sobel >= thresh[0]) & (scaled_sobel <= thresh[1])] = 1 21 | return binary_output 22 | 23 | # Define a function to return the magnitude of the gradient 24 | # for a given sobel kernel size and threshold values 25 | def mag_thresh(img, sobel_kernel=3, mag_thresh=(0, 255)): 26 | sobelx = cv2.Sobel(img, cv2.CV_64F, 1, 0, ksize=sobel_kernel) 27 | sobely = cv2.Sobel(img, cv2.CV_64F, 0, 1, ksize=sobel_kernel) 28 | gradmag = np.sqrt(sobelx**2 + sobely**2) 29 | scale_factor = np.max(gradmag)/255 30 | gradmag = (gradmag/scale_factor).astype(np.uint8) 31 | binary_output = np.zeros_like(gradmag) 32 | binary_output[(gradmag >= mag_thresh[0]) & (gradmag <= mag_thresh[1])] = 1 33 | return binary_output 34 | 35 | 36 | # Define a function to threshold an image for a given range and Sobel kernel for directionsobel 37 | # TODO these are changed in the other file 38 | def dir_threshold(img, sobel_kernel=3, thresh=(0, np.pi/2)): 39 | sobelx = cv2.Sobel(img, cv2.CV_64F, 1, 0, ksize=sobel_kernel) 40 | sobely = cv2.Sobel(img, cv2.CV_64F, 0, 1, ksize=sobel_kernel) 41 | absgraddir = np.arctan2(np.absolute(sobely), np.absolute(sobelx)) 42 | binary_output = np.zeros_like(absgraddir) 43 | binary_output[(absgraddir >= thresh[0]) & (absgraddir <= thresh[1])] = 1 44 | return binary_output.astype(np.uint8) 45 | 46 | #used for the white lanes histogram equalisation thresholding 47 | def adp_thresh_grayscale(gray, thr = 250): 48 | img = cv2.equalizeHist(gray) 49 | ret, thrs = cv2.threshold(img, thresh=thr, maxval=255, type=cv2.THRESH_BINARY) 50 | return thrs 51 | 52 | #Color thresholding, takes saturation and value images in single channel and corresponding threshold values 53 | # TODO these are also changed 54 | def color_thr(s_img, l_img, s_threshold = (0,255), l_threshold = (0,255)): 55 | s_binary = np.zeros_like(s_img).astype(np.uint8) 56 | s_binary[(s_img > s_threshold[0]) & (s_img <= s_threshold[1])] = 1 57 | l_binary = np.zeros_like(l_img).astype(np.uint8) 58 | l_binary[(l_img > l_threshold[0]) & (l_img <= l_threshold[1])] = 1 59 | col = ((s_binary == 1) | (l_binary == 1)) 60 | return col 61 | 62 | with open(r'C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\Settings\config_2.JSON') as config_file: 63 | data = json.load(config_file) 64 | grad_thx_mins=data['grad_thx_min'] 65 | grad_thx_maxs=data['grad_thx_max'] 66 | grad_thy_mins=data['grad_thy_min'] 67 | grad_thy_maxs=data['grad_thy_max'] 68 | mag_th_mins=data['mag_th_min'] 69 | mag_th_maxs=data['mag_th_max'] 70 | dir_th_mins=data['dir_th_min'] 71 | dir_th_maxs=data['dir_th_max'] 72 | s_threshold_mins=data['s_threshold_min'] 73 | s_threshold_maxs=data['s_threshold_max'] 74 | l_threshold_mins=data['l_threshold_min'] 75 | l_threshold_maxs=data['l_threshold_max'] 76 | k_sizes=data['k_size'] 77 | adp_thrs=data['adp_thr'] 78 | #the main thresholding operaion is performed here 79 | def thresholding(img, grad_thx_min =grad_thx_mins, grad_thx_max =grad_thx_maxs, 80 | grad_thy_min =grad_thy_mins, grad_thy_max = grad_thy_maxs, mag_th_min = mag_th_mins, 81 | mag_th_max = mag_th_maxs, dir_th_min =dir_th_mins, dir_th_max = dir_th_maxs, 82 | s_threshold_min = s_threshold_mins, s_threshold_max = s_threshold_maxs, 83 | l_threshold_min = l_threshold_mins, l_threshold_max = l_threshold_maxs, k_size = k_sizes, 84 | adp_thr = adp_thrs): 85 | imshape = img.shape 86 | hls = cv2.cvtColor(img, cv2.COLOR_RGB2HLS).astype(np.float) 87 | s_channel = hls[:,:,2].astype(np.uint8) 88 | l_channel = hls[:,:,1].astype(np.uint8) 89 | ksize = k_size 90 | gradx = abs_sobel_thresh(l_channel, orient='x', sobel_kernel=ksize, thresh=(grad_thx_min,grad_thx_max)) 91 | grady = abs_sobel_thresh(l_channel, orient='y', sobel_kernel=ksize, thresh=(grad_thy_min, grad_thy_max)) 92 | mag_binary = mag_thresh(l_channel, sobel_kernel=ksize, mag_thresh=(mag_th_min, mag_th_max)) 93 | dir_binary = dir_threshold(l_channel, sobel_kernel=ksize, thresh=(dir_th_min, dir_th_max)) 94 | combined = np.zeros_like(gradx) 95 | combined[((gradx == 1) & (grady == 1)) | ((mag_binary == 1) & (dir_binary == 1))] = 1 96 | s_binary = color_thr(s_channel, l_channel, s_threshold=(s_threshold_min,s_threshold_max), 97 | l_threshold= (l_threshold_min,l_threshold_max)).astype(np.uint8) 98 | gray = cv2.cvtColor(img, cv2.COLOR_RGB2GRAY).astype(np.uint8) 99 | adp = adp_thresh_grayscale(gray, adp_thr)/255 100 | color_binary = np.zeros_like(gradx) 101 | color_binary[(combined == 1) | (s_binary == 1) | (adp == 1)] = 1 102 | color_binary = np.dstack(( color_binary,color_binary,color_binary)).astype(np.float32) 103 | vertices = np.array([[(.4*imshape[1], .4*imshape[0]), (.6*imshape[1],.4*imshape[0]), 104 | (.8*imshape[1],imshape[0]),(.23*imshape[1], imshape[0])]], dtype=np.int32) 105 | color_binary = region_of_interest(color_binary.astype(np.uint8), vertices) 106 | return color_binary.astype(np.float32), combined, s_binary 107 | 108 | #img=cv2.imread(r"C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\data\CULane\driver_100_30frame\05250510_0307.MP4\00030.jpg") 109 | #img2=cv2.imread(r"C:\Users\Mouiad\Desktop\SACES_Second_test\train_images\um_000000.png") 110 | #x,y,z=thresholding(img) 111 | #cv2.imshow("r",x) 112 | 113 | 114 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/train_lanenet.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Time : 18-5-18 下午7:31 4 | # @Author : Luo Yao 5 | # @Site : http://icode.baidu.com/repos/baidu/personal-code/Luoyao 6 | # @File : train_lanenet.py 7 | # @IDE: PyCharm Community Edition 8 | """ 9 | 训练lanenet模型 10 | """ 11 | import argparse 12 | import os 13 | import os.path as ops 14 | import time 15 | 16 | import glog as log 17 | import numpy as np 18 | import tensorflow as tf 19 | 20 | import sys 21 | 22 | from config import global_config 23 | from lanenet_model import lanenet_merge_model 24 | from data_provider import lanenet_data_processor 25 | 26 | CFG = global_config.cfg 27 | 28 | 29 | def init_args(): 30 | """ 31 | 32 | :return: 33 | """ 34 | parser = argparse.ArgumentParser() 35 | 36 | parser.add_argument('--dataset_dir', type=str, help='The training dataset dir path') 37 | parser.add_argument('--net', type=str, help='Which base net work to use', default='vgg') 38 | parser.add_argument('--weights_path', type=str, help='The pretrained weights path') 39 | 40 | return parser.parse_args() 41 | 42 | 43 | def average_gradients(tower_grads): 44 | """Calculate the average gradient for each shared variable across all towers. 45 | Note that this function provides a synchronization point across all towers. 46 | Args: 47 | tower_grads: List of lists of (gradient, variable) tuples. The outer list 48 | is over individual gradients. The inner list is over the gradient 49 | calculation for each tower. 50 | Returns: 51 | List of pairs of (gradient, variable) where the gradient has been averaged 52 | across all towers. 53 | """ 54 | average_grads = [] 55 | for grad_and_vars in zip(*tower_grads): 56 | # Note that each grad_and_vars looks like the following: 57 | # ((grad0_gpu0, var0_gpu0), ... , (grad0_gpuN, var0_gpuN)) 58 | grads = [] 59 | for g, _ in grad_and_vars: 60 | # Add 0 dimension to the gradients to represent the tower. 61 | expanded_g = tf.expand_dims(g, 0) 62 | 63 | # Append on a 'tower' dimension which we will average over below. 64 | grads.append(expanded_g) 65 | 66 | # Average over the 'tower' dimension. 67 | grad = tf.concat(grads, 0) 68 | grad = tf.reduce_mean(grad, 0) 69 | 70 | # Keep in mind that the Variables are redundant because they are shared 71 | # across towers. So .. we will just return the first tower's pointer to 72 | # the Variable. 73 | v = grad_and_vars[0][1] 74 | grad_and_var = (grad, v) 75 | average_grads.append(grad_and_var) 76 | return average_grads 77 | 78 | 79 | def forward(batch_queue, net, phase, scope, optimizer=None): 80 | img_batch, label_instance_batch, label_existence_batch = batch_queue.dequeue() 81 | inference = net.inference(img_batch, phase, 'lanenet_loss') 82 | _ = net.loss(inference, label_instance_batch, label_existence_batch, 'lanenet_loss') 83 | total_loss = tf.add_n(tf.get_collection('total_loss', scope)) 84 | instance_loss = tf.add_n(tf.get_collection('instance_seg_loss', scope)) 85 | existence_loss = tf.add_n(tf.get_collection('existence_pre_loss', scope)) 86 | 87 | out_logits = tf.add_n(tf.get_collection('instance_seg_logits', scope)) 88 | # calculate the accuracy 89 | out_logits = tf.nn.softmax(logits=out_logits) 90 | out_logits_out = tf.argmax(out_logits, axis=-1) 91 | 92 | pred_0 = tf.count_nonzero(tf.multiply(tf.cast(tf.equal(label_instance_batch, 0), tf.int32), 93 | tf.cast(tf.equal(out_logits_out, 0), tf.int32)), 94 | dtype=tf.int32) 95 | pred_1 = tf.count_nonzero(tf.multiply(tf.cast(tf.equal(label_instance_batch, 1), tf.int32), 96 | tf.cast(tf.equal(out_logits_out, 1), tf.int32)), 97 | dtype=tf.int32) 98 | pred_2 = tf.count_nonzero(tf.multiply(tf.cast(tf.equal(label_instance_batch, 2), tf.int32), 99 | tf.cast(tf.equal(out_logits_out, 2), tf.int32)), 100 | dtype=tf.int32) 101 | pred_3 = tf.count_nonzero(tf.multiply(tf.cast(tf.equal(label_instance_batch, 3), tf.int32), 102 | tf.cast(tf.equal(out_logits_out, 3), tf.int32)), 103 | dtype=tf.int32) 104 | pred_4 = tf.count_nonzero(tf.multiply(tf.cast(tf.equal(label_instance_batch, 4), tf.int32), 105 | tf.cast(tf.equal(out_logits_out, 4), tf.int32)), 106 | dtype=tf.int32) 107 | gt_all = tf.count_nonzero(tf.cast(tf.greater(label_instance_batch, 0), tf.int32), dtype=tf.int32) 108 | gt_back = tf.count_nonzero(tf.cast(tf.equal(label_instance_batch, 0), tf.int32), dtype=tf.int32) 109 | 110 | pred_all = tf.add(tf.add(tf.add(pred_1, pred_2), pred_3), pred_4) 111 | 112 | accuracy = tf.divide(tf.cast(pred_all, tf.float32), tf.cast(gt_all, tf.float32)) 113 | accuracy_back = tf.divide(tf.cast(pred_0, tf.float32), tf.cast(gt_back, tf.float32)) 114 | 115 | # Compute mIoU of Lanes 116 | overlap_1 = pred_1 117 | union_1 = tf.add(tf.count_nonzero(tf.cast(tf.equal(label_instance_batch, 1), 118 | tf.int32), dtype=tf.int32), 119 | tf.count_nonzero(tf.cast(tf.equal(out_logits_out, 1), 120 | tf.int32), dtype=tf.int32)) 121 | union_1 = tf.subtract(union_1, overlap_1) 122 | IoU_1 = tf.divide(tf.cast(overlap_1, tf.float32), tf.cast(union_1, tf.float32)) 123 | 124 | overlap_2 = pred_2 125 | union_2 = tf.add(tf.count_nonzero(tf.cast(tf.equal(label_instance_batch, 2), 126 | tf.int32), dtype=tf.int32), 127 | tf.count_nonzero(tf.cast(tf.equal(out_logits_out, 2), 128 | tf.int32), dtype=tf.int32)) 129 | union_2 = tf.subtract(union_2, overlap_2) 130 | IoU_2 = tf.divide(tf.cast(overlap_2, tf.float32), tf.cast(union_2, tf.float32)) 131 | 132 | overlap_3 = pred_3 133 | union_3 = tf.add(tf.count_nonzero(tf.cast(tf.equal(label_instance_batch, 3), 134 | tf.int32), dtype=tf.int32), 135 | tf.count_nonzero(tf.cast(tf.equal(out_logits_out, 3), 136 | tf.int32), dtype=tf.int32)) 137 | union_3 = tf.subtract(union_3, overlap_3) 138 | IoU_3 = tf.divide(tf.cast(overlap_3, tf.float32), tf.cast(union_3, tf.float32)) 139 | 140 | overlap_4 = pred_4 141 | union_4 = tf.add(tf.count_nonzero(tf.cast(tf.equal(label_instance_batch, 4), 142 | tf.int64), dtype=tf.int32), 143 | tf.count_nonzero(tf.cast(tf.equal(out_logits_out, 4), 144 | tf.int64), dtype=tf.int32)) 145 | union_4 = tf.subtract(union_4, overlap_4) 146 | IoU_4 = tf.divide(tf.cast(overlap_4, tf.float32), tf.cast(union_4, tf.float32)) 147 | 148 | IoU = tf.reduce_mean(tf.stack([IoU_1, IoU_2, IoU_3, IoU_4])) 149 | 150 | tf.get_variable_scope().reuse_variables() 151 | 152 | if optimizer is not None: 153 | grads = optimizer.compute_gradients(total_loss) 154 | else: 155 | grads = None 156 | return total_loss, instance_loss, existence_loss, accuracy, accuracy_back, IoU, out_logits_out, grads 157 | 158 | 159 | def train_net(dataset_dir, weights_path=None, net_flag='vgg'): 160 | train_dataset_file = ops.join(dataset_dir, 'train_gt.txt') 161 | val_dataset_file = ops.join(dataset_dir, 'val_gt.txt') 162 | 163 | assert ops.exists(train_dataset_file) 164 | 165 | phase = tf.placeholder(dtype=tf.string, shape=None, name='net_phase') 166 | 167 | train_dataset = lanenet_data_processor.DataSet(train_dataset_file) 168 | val_dataset = lanenet_data_processor.DataSet(val_dataset_file) 169 | 170 | net = lanenet_merge_model.LaneNet() 171 | 172 | tower_grads = [] 173 | 174 | global_step = tf.Variable(0, trainable=False) 175 | 176 | learning_rate = tf.train.polynomial_decay(CFG.TRAIN.LEARNING_RATE, global_step, 177 | CFG.TRAIN.EPOCHS, power=0.9) 178 | 179 | optimizer = tf.train.MomentumOptimizer(learning_rate=learning_rate, momentum=0.9) 180 | img, label_instance, label_existence = train_dataset.next_batch(CFG.TRAIN.BATCH_SIZE) 181 | batch_queue = tf.contrib.slim.prefetch_queue.prefetch_queue( 182 | [img, label_instance, label_existence], capacity=2 * CFG.TRAIN.GPU_NUM, num_threads=CFG.TRAIN.CPU_NUM) 183 | 184 | val_img, val_label_instance, val_label_existence = val_dataset.next_batch(CFG.TRAIN.VAL_BATCH_SIZE) 185 | val_batch_queue = tf.contrib.slim.prefetch_queue.prefetch_queue( 186 | [val_img, val_label_instance, val_label_existence], capacity=2 * CFG.TRAIN.GPU_NUM, 187 | num_threads=CFG.TRAIN.CPU_NUM) 188 | with tf.variable_scope(tf.get_variable_scope()): 189 | for i in range(CFG.TRAIN.GPU_NUM): 190 | with tf.device('/gpu:%d' % i): 191 | with tf.name_scope('tower_%d' % i) as scope: 192 | total_loss, instance_loss, existence_loss, accuracy, accuracy_back, _, out_logits_out, \ 193 | grad = forward(batch_queue, net, phase, scope, optimizer) 194 | tower_grads.append(grad) 195 | with tf.name_scope('test_%d' % i) as scope: 196 | val_op_total_loss, val_op_instance_loss, val_op_existence_loss, val_op_accuracy, \ 197 | val_op_accuracy_back, val_op_IoU, _, _ = forward(val_batch_queue, net, phase, scope) 198 | 199 | grads = average_gradients(tower_grads) 200 | 201 | train_op = optimizer.apply_gradients(grads, global_step=global_step) 202 | 203 | train_cost_time_mean = [] 204 | train_instance_loss_mean = [] 205 | train_existence_loss_mean = [] 206 | train_accuracy_mean = [] 207 | train_accuracy_back_mean = [] 208 | 209 | saver = tf.train.Saver() 210 | model_save_dir = 'model/culane_lanenet/culane_scnn' 211 | if not ops.exists(model_save_dir): 212 | os.makedirs(model_save_dir) 213 | train_start_time = time.strftime('%Y-%m-%d-%H-%M-%S', time.localtime(time.time())) 214 | model_name = 'culane_lanenet_{:s}_{:s}.ckpt'.format(net_flag, str(train_start_time)) 215 | model_save_path = ops.join(model_save_dir, model_name) 216 | 217 | sess_config = tf.ConfigProto(device_count={'GPU': CFG.TRAIN.GPU_NUM}, allow_soft_placement=True) 218 | sess_config.gpu_options.per_process_gpu_memory_fraction = CFG.TRAIN.GPU_MEMORY_FRACTION 219 | sess_config.gpu_options.allow_growth = CFG.TRAIN.TF_ALLOW_GROWTH 220 | sess_config.gpu_options.allocator_type = 'BFC' 221 | 222 | with tf.Session(config=sess_config) as sess: 223 | with sess.as_default(): 224 | 225 | if weights_path is None: 226 | log.info('Training from scratch') 227 | init = tf.global_variables_initializer() 228 | sess.run(init) 229 | else: 230 | log.info('Restore model from last model checkpoint {:s}'.format(weights_path)) 231 | saver.restore(sess=sess, save_path=weights_path) 232 | 233 | # 加载预训练参数 234 | if net_flag == 'vgg' and weights_path is None: 235 | pretrained_weights = np.load( 236 | './data/vgg16.npy', 237 | encoding='latin1').item() 238 | 239 | for vv in tf.trainable_variables(): 240 | weights = vv.name.split('/') 241 | if len(weights) >= 3 and weights[-3] in pretrained_weights: 242 | try: 243 | weights_key = weights[-3] 244 | weights = pretrained_weights[weights_key][0] 245 | _op = tf.assign(vv, weights) 246 | sess.run(_op) 247 | except Exception as e: 248 | continue 249 | tf.train.start_queue_runners(sess=sess) 250 | for epoch in range(CFG.TRAIN.EPOCHS): 251 | t_start = time.time() 252 | 253 | _, c, train_accuracy, train_accuracy_back, train_instance_loss, train_existence_loss, _ = \ 254 | sess.run([train_op, total_loss, accuracy, accuracy_back, instance_loss, existence_loss, out_logits_out], 255 | feed_dict={phase: 'train'}) 256 | 257 | cost_time = time.time() - t_start 258 | train_cost_time_mean.append(cost_time) 259 | train_instance_loss_mean.append(train_instance_loss) 260 | train_existence_loss_mean.append(train_existence_loss) 261 | train_accuracy_mean.append(train_accuracy) 262 | train_accuracy_back_mean.append(train_accuracy_back) 263 | 264 | if epoch % CFG.TRAIN.DISPLAY_STEP == 0: 265 | print( 266 | 'Epoch: {:d} loss_ins= {:6f} ({:6f}) loss_ext= {:6f} ({:6f}) accuracy= {:6f} ({:6f}) ' 267 | 'accuracy_back= {:6f} ({:6f}) mean_time= {:5f}s '.format(epoch + 1, train_instance_loss, 268 | np.mean(train_instance_loss_mean), 269 | train_existence_loss, 270 | np.mean(train_existence_loss_mean), 271 | train_accuracy, 272 | np.mean(train_accuracy_mean), 273 | train_accuracy_back, 274 | np.mean(train_accuracy_back_mean), 275 | np.mean(train_cost_time_mean))) 276 | 277 | if epoch % 500 == 0: 278 | train_cost_time_mean.clear() 279 | train_instance_loss_mean.clear() 280 | train_existence_loss_mean.clear() 281 | train_accuracy_mean.clear() 282 | train_accuracy_back_mean.clear() 283 | 284 | if epoch % 1000 == 0: 285 | saver.save(sess=sess, save_path=model_save_path, global_step=epoch) 286 | 287 | if epoch % 10000 != 0 or epoch == 0: 288 | continue 289 | 290 | val_cost_time_mean = [] 291 | val_instance_loss_mean = [] 292 | val_existence_loss_mean = [] 293 | val_accuracy_mean = [] 294 | val_accuracy_back_mean = [] 295 | val_IoU_mean = [] 296 | 297 | for epoch_val in range(int(len(val_dataset) / CFG.TRAIN.VAL_BATCH_SIZE / CFG.TRAIN.GPU_NUM)): 298 | t_start_val = time.time() 299 | c_val, val_accuracy, val_accuracy_back, val_IoU, val_instance_loss, val_existence_loss = \ 300 | sess.run( 301 | [val_op_total_loss, val_op_accuracy, val_op_accuracy_back, 302 | val_op_IoU, val_op_instance_loss, val_op_existence_loss], 303 | feed_dict={phase: 'test'}) 304 | 305 | cost_time_val = time.time() - t_start_val 306 | val_cost_time_mean.append(cost_time_val) 307 | val_instance_loss_mean.append(val_instance_loss) 308 | val_existence_loss_mean.append(val_existence_loss) 309 | val_accuracy_mean.append(val_accuracy) 310 | val_accuracy_back_mean.append(val_accuracy_back) 311 | val_IoU_mean.append(val_IoU) 312 | 313 | if epoch_val % 1 == 0: 314 | print('Epoch_Val: {:d} loss_ins= {:6f} ({:6f}) ' 315 | 'loss_ext= {:6f} ({:6f}) accuracy= {:6f} ({:6f}) accuracy_back= {:6f} ({:6f}) ' 316 | 'mIoU= {:6f} ({:6f}) mean_time= {:5f}s '. 317 | format(epoch_val + 1, val_instance_loss, np.mean(val_instance_loss_mean), val_existence_loss, 318 | np.mean(val_existence_loss_mean), val_accuracy, np.mean(val_accuracy_mean), 319 | val_accuracy_back, np.mean(val_accuracy_back_mean), val_IoU, np.mean(val_IoU_mean), 320 | np.mean(val_cost_time_mean))) 321 | 322 | val_cost_time_mean.clear() 323 | val_instance_loss_mean.clear() 324 | val_existence_loss_mean.clear() 325 | val_accuracy_mean.clear() 326 | val_accuracy_back_mean.clear() 327 | val_IoU_mean.clear() 328 | 329 | 330 | if __name__ == '__main__': 331 | # init args 332 | args = init_args() 333 | 334 | # train lanenet 335 | train_net(args.dataset_dir, args.weights_path, net_flag=args.net) 336 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane-detection-model/tools/vgg_encoder.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # @Time : 18-1-29 下午2:04 4 | # @Author : Luo Yao 5 | # @Site : http://icode.baidu.com/repos/baidu/personal-code/Luoyao 6 | # @File : dilation_encoder.py 7 | # @IDE: PyCharm Community Edition 8 | """ 9 | 实现一个基于VGG16的特征编码类 10 | """ 11 | from collections import OrderedDict 12 | import math 13 | 14 | import tensorflow as tf 15 | 16 | import cnn_basenet 17 | import global_config 18 | 19 | CFG = global_config.cfg 20 | 21 | 22 | class VGG16Encoder(cnn_basenet.CNNBaseModel): 23 | """ 24 | 实现了一个基于vgg16的特征编码类 25 | """ 26 | 27 | def __init__(self, phase): 28 | """ 29 | 30 | :param phase: 31 | """ 32 | super(VGG16Encoder, self).__init__() 33 | self._train_phase = tf.constant('train', dtype=tf.string) 34 | self._test_phase = tf.constant('test', dtype=tf.string) 35 | self._phase = phase 36 | self._is_training = self._init_phase() 37 | 38 | def _init_phase(self): 39 | """ 40 | 41 | :return: 42 | """ 43 | return tf.equal(self._phase, self._train_phase) 44 | 45 | def _conv_stage(self, input_tensor, k_size, out_dims, name, 46 | stride=1, pad='SAME'): 47 | """ 48 | 将卷积和激活封装在一起 49 | :param input_tensor: 50 | :param k_size: 51 | :param out_dims: 52 | :param name: 53 | :param stride: 54 | :param pad: 55 | :return: 56 | """ 57 | with tf.variable_scope(name): 58 | conv = self.conv2d(inputdata=input_tensor, out_channel=out_dims, 59 | kernel_size=k_size, stride=stride, 60 | use_bias=False, padding=pad, name='conv') 61 | 62 | bn = self.layerbn(inputdata=conv, is_training=self._is_training, name='bn') 63 | 64 | relu = self.relu(inputdata=bn, name='relu') 65 | 66 | return relu 67 | 68 | def _conv_dilated_stage(self, input_tensor, k_size, out_dims, name, 69 | dilation=1, pad='SAME'): 70 | """ 71 | 将卷积和激活封装在一起 72 | :param input_tensor: 73 | :param k_size: 74 | :param out_dims: 75 | :param name: 76 | :param stride: 77 | :param pad: 78 | :return: 79 | """ 80 | with tf.variable_scope(name): 81 | conv = self.dilation_conv(input_tensor=input_tensor, out_dims=out_dims, 82 | k_size=k_size, rate=dilation, 83 | use_bias=False, padding=pad, name='conv') 84 | 85 | bn = self.layerbn(inputdata=conv, is_training=self._is_training, name='bn') 86 | 87 | relu = self.relu(inputdata=bn, name='relu') 88 | 89 | return relu 90 | 91 | def _fc_stage(self, input_tensor, out_dims, name, use_bias=False): 92 | """ 93 | 94 | :param input_tensor: 95 | :param out_dims: 96 | :param name: 97 | :param use_bias: 98 | :return: 99 | """ 100 | with tf.variable_scope(name): 101 | fc = self.fullyconnect(inputdata=input_tensor, out_dim=out_dims, use_bias=use_bias, 102 | name='fc') 103 | 104 | bn = self.layerbn(inputdata=fc, is_training=self._is_training, name='bn') 105 | 106 | relu = self.relu(inputdata=bn, name='relu') 107 | 108 | return relu 109 | 110 | def encode(self, input_tensor, name): 111 | """ 112 | 根据vgg16框架对输入的tensor进行编码 113 | :param input_tensor: 114 | :param name: 115 | :return: 输出vgg16编码特征 116 | """ 117 | ret = OrderedDict() 118 | with tf.variable_scope(name): 119 | # conv stage 1_1 120 | conv_1_1 = self._conv_stage(input_tensor=input_tensor, k_size=3, 121 | out_dims=64, name='conv1_1') 122 | 123 | # conv stage 1_2 124 | conv_1_2 = self._conv_stage(input_tensor=conv_1_1, k_size=3, 125 | out_dims=64, name='conv1_2') 126 | 127 | # pool stage 1 128 | pool1 = self.maxpooling(inputdata=conv_1_2, kernel_size=2, 129 | stride=2, name='pool1') 130 | 131 | # conv stage 2_1 132 | conv_2_1 = self._conv_stage(input_tensor=pool1, k_size=3, 133 | out_dims=128, name='conv2_1') 134 | 135 | # conv stage 2_2 136 | conv_2_2 = self._conv_stage(input_tensor=conv_2_1, k_size=3, 137 | out_dims=128, name='conv2_2') 138 | 139 | # pool stage 2 140 | pool2 = self.maxpooling(inputdata=conv_2_2, kernel_size=2, 141 | stride=2, name='pool2') 142 | 143 | # conv stage 3_1 144 | conv_3_1 = self._conv_stage(input_tensor=pool2, k_size=3, 145 | out_dims=256, name='conv3_1') 146 | 147 | # conv_stage 3_2 148 | conv_3_2 = self._conv_stage(input_tensor=conv_3_1, k_size=3, 149 | out_dims=256, name='conv3_2') 150 | 151 | # conv stage 3_3 152 | conv_3_3 = self._conv_stage(input_tensor=conv_3_2, k_size=3, 153 | out_dims=256, name='conv3_3') 154 | 155 | # pool stage 3 156 | pool3 = self.maxpooling(inputdata=conv_3_3, kernel_size=2, 157 | stride=2, name='pool3') 158 | 159 | # conv stage 4_1 160 | conv_4_1 = self._conv_stage(input_tensor=pool3, k_size=3, 161 | out_dims=512, name='conv4_1') 162 | 163 | # conv stage 4_2 164 | conv_4_2 = self._conv_stage(input_tensor=conv_4_1, k_size=3, 165 | out_dims=512, name='conv4_2') 166 | 167 | # conv stage 4_3 168 | conv_4_3 = self._conv_stage(input_tensor=conv_4_2, k_size=3, 169 | out_dims=512, name='conv4_3') 170 | 171 | ### add dilated convolution ### 172 | 173 | # conv stage 5_1 174 | conv_5_1 = self._conv_dilated_stage(input_tensor=conv_4_3, k_size=3, 175 | out_dims=512, dilation=2, name='conv5_1') 176 | 177 | # conv stage 5_2 178 | conv_5_2 = self._conv_dilated_stage(input_tensor=conv_5_1, k_size=3, 179 | out_dims=512, dilation=2, name='conv5_2') 180 | 181 | # conv stage 5_3 182 | conv_5_3 = self._conv_dilated_stage(input_tensor=conv_5_2, k_size=3, 183 | out_dims=512, dilation=2, name='conv5_3') 184 | 185 | # added part of SCNN # 186 | 187 | # conv stage 5_4 188 | conv_5_4 = self._conv_dilated_stage(input_tensor=conv_5_3, k_size=3, 189 | out_dims=1024, dilation=4, name='conv5_4') 190 | 191 | # conv stage 5_5 192 | conv_5_5 = self._conv_stage(input_tensor=conv_5_4, k_size=1, 193 | out_dims=128, name='conv5_5') # 8 x 36 x 100 x 128 194 | 195 | # add message passing # 196 | 197 | # top to down # 198 | 199 | feature_list_old = [] 200 | feature_list_new = [] 201 | for cnt in range(conv_5_5.get_shape().as_list()[1]): 202 | feature_list_old.append(tf.expand_dims(conv_5_5[:, cnt, :, :], axis=1)) 203 | feature_list_new.append(tf.expand_dims(conv_5_5[:, 0, :, :], axis=1)) 204 | 205 | w1 = tf.get_variable('W1', [1, 9, 128, 128], 206 | initializer=tf.random_normal_initializer(0, math.sqrt(2.0 / (9 * 128 * 128 * 5)))) 207 | with tf.variable_scope("convs_6_1"): 208 | conv_6_1 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_old[0], w1, [1, 1, 1, 1], 'SAME')), 209 | feature_list_old[1]) 210 | feature_list_new.append(conv_6_1) 211 | 212 | for cnt in range(2, conv_5_5.get_shape().as_list()[1]): 213 | with tf.variable_scope("convs_6_1", reuse=True): 214 | conv_6_1 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_new[cnt - 1], w1, [1, 1, 1, 1], 'SAME')), 215 | feature_list_old[cnt]) 216 | feature_list_new.append(conv_6_1) 217 | 218 | # down to top # 219 | feature_list_old = feature_list_new 220 | feature_list_new = [] 221 | length = int(CFG.TRAIN.IMG_HEIGHT / 8) - 1 222 | feature_list_new.append(feature_list_old[length]) 223 | 224 | w2 = tf.get_variable('W2', [1, 9, 128, 128], 225 | initializer=tf.random_normal_initializer(0, math.sqrt(2.0 / (9 * 128 * 128 * 5)))) 226 | with tf.variable_scope("convs_6_2"): 227 | conv_6_2 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_old[length], w2, [1, 1, 1, 1], 'SAME')), 228 | feature_list_old[length - 1]) 229 | feature_list_new.append(conv_6_2) 230 | 231 | for cnt in range(2, conv_5_5.get_shape().as_list()[1]): 232 | with tf.variable_scope("convs_6_2", reuse=True): 233 | conv_6_2 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_new[cnt - 1], w2, [1, 1, 1, 1], 'SAME')), 234 | feature_list_old[length - cnt]) 235 | feature_list_new.append(conv_6_2) 236 | 237 | feature_list_new.reverse() 238 | 239 | processed_feature = tf.stack(feature_list_new, axis=1) 240 | processed_feature = tf.squeeze(processed_feature, axis=2) 241 | 242 | # left to right # 243 | 244 | feature_list_old = [] 245 | feature_list_new = [] 246 | for cnt in range(processed_feature.get_shape().as_list()[2]): 247 | feature_list_old.append(tf.expand_dims(processed_feature[:, :, cnt, :], axis=2)) 248 | feature_list_new.append(tf.expand_dims(processed_feature[:, :, 0, :], axis=2)) 249 | 250 | w3 = tf.get_variable('W3', [9, 1, 128, 128], 251 | initializer=tf.random_normal_initializer(0, math.sqrt(2.0 / (9 * 128 * 128 * 5)))) 252 | with tf.variable_scope("convs_6_3"): 253 | conv_6_3 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_old[0], w3, [1, 1, 1, 1], 'SAME')), 254 | feature_list_old[1]) 255 | feature_list_new.append(conv_6_3) 256 | 257 | for cnt in range(2, processed_feature.get_shape().as_list()[2]): 258 | with tf.variable_scope("convs_6_3", reuse=True): 259 | conv_6_3 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_new[cnt - 1], w3, [1, 1, 1, 1], 'SAME')), 260 | feature_list_old[cnt]) 261 | feature_list_new.append(conv_6_3) 262 | 263 | # right to left # 264 | 265 | feature_list_old = feature_list_new 266 | feature_list_new = [] 267 | length = int(CFG.TRAIN.IMG_WIDTH / 8) - 1 268 | feature_list_new.append(feature_list_old[length]) 269 | 270 | w4 = tf.get_variable('W4', [9, 1, 128, 128], 271 | initializer=tf.random_normal_initializer(0, math.sqrt(2.0 / (9 * 128 * 128 * 5)))) 272 | with tf.variable_scope("convs_6_4"): 273 | conv_6_4 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_old[length], w4, [1, 1, 1, 1], 'SAME')), 274 | feature_list_old[length - 1]) 275 | feature_list_new.append(conv_6_4) 276 | 277 | for cnt in range(2, processed_feature.get_shape().as_list()[2]): 278 | with tf.variable_scope("convs_6_4", reuse=True): 279 | conv_6_4 = tf.add(tf.nn.relu(tf.nn.conv2d(feature_list_new[cnt - 1], w4, [1, 1, 1, 1], 'SAME')), 280 | feature_list_old[length - cnt]) 281 | feature_list_new.append(conv_6_4) 282 | 283 | feature_list_new.reverse() 284 | processed_feature = tf.stack(feature_list_new, axis=2) 285 | processed_feature = tf.squeeze(processed_feature, axis=3) 286 | 287 | ####################### 288 | 289 | dropout_output = self.dropout(processed_feature, 0.9, is_training=self._is_training, 290 | name='dropout') # 0.9 denotes the probability of being kept 291 | 292 | conv_output = self.conv2d(inputdata=dropout_output, out_channel=5, 293 | kernel_size=1, use_bias=True, name='conv_6') 294 | 295 | ret['prob_output'] = tf.image.resize_images(conv_output, [CFG.TRAIN.IMG_HEIGHT, CFG.TRAIN.IMG_WIDTH]) 296 | 297 | ### add lane existence prediction branch ### 298 | 299 | # spatial softmax # 300 | features = conv_output # N x H x W x C 301 | softmax = tf.nn.softmax(features) 302 | 303 | avg_pool = self.avgpooling(softmax, kernel_size=2, stride=2) 304 | _, H, W, C = avg_pool.get_shape().as_list() 305 | reshape_output = tf.reshape(avg_pool, [-1, H * W * C]) 306 | fc_output = self.fullyconnect(reshape_output, 128) 307 | relu_output = self.relu(inputdata=fc_output, name='relu6') 308 | fc_output = self.fullyconnect(relu_output, 4) 309 | existence_output = fc_output 310 | 311 | ret['existence_output'] = existence_output 312 | 313 | return ret 314 | 315 | 316 | if __name__ == '__main__': 317 | a = tf.placeholder(dtype=tf.float32, shape=[CFG.TRAIN.BATCH_SIZE, CFG.TRAIN.IMG_HEIGHT, CFG.TRAIN.IMG_WIDTH, 3], 318 | name='input') 319 | encoder = VGG16Encoder(phase=tf.constant('train', dtype=tf.string)) 320 | ret = encoder.encode(a, name='encode') 321 | print(ret) 322 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/.gitignore: -------------------------------------------------------------------------------- 1 | build/ 2 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/Makefile: -------------------------------------------------------------------------------- 1 | PROJECT_NAME:= evaluate 2 | 3 | # config ---------------------------------- 4 | 5 | INCLUDE_DIRS := include 6 | LIBRARY_DIRS := lib 7 | 8 | COMMON_FLAGS := -DCPU_ONLY 9 | CXXFLAGS := -std=c++11 -fopenmp 10 | LDFLAGS := -fopenmp -Wl,-rpath,./lib 11 | 12 | BUILD_DIR := build 13 | 14 | # make rules ------------------------------- 15 | CXX ?= g++ 16 | BUILD_DIR ?= ./build 17 | 18 | LIBRARIES += opencv_core opencv_highgui opencv_imgproc # opencv_imgcodecs 19 | 20 | CXXFLAGS += $(COMMON_FLAGS) $(foreach includedir,$(INCLUDE_DIRS),-I$(includedir)) 21 | LDFLAGS += $(COMMON_FLAGS) $(foreach includedir,$(LIBRARY_DIRS),-L$(includedir)) $(foreach library,$(LIBRARIES),-l$(library)) 22 | SRC_DIRS += $(shell find * -type d -exec bash -c "find {} -maxdepth 1 \( -name '*.cpp' -o -name '*.proto' \) | grep -q ." \; -print) 23 | CXX_SRCS += $(shell find src/ -name "*.cpp") 24 | CXX_TARGETS:=$(patsubst %.cpp, $(BUILD_DIR)/%.o, $(CXX_SRCS)) 25 | ALL_BUILD_DIRS := $(sort $(BUILD_DIR) $(addprefix $(BUILD_DIR)/, $(SRC_DIRS))) 26 | 27 | .PHONY: all 28 | all: $(PROJECT_NAME) 29 | 30 | .PHONY: $(ALL_BUILD_DIRS) 31 | $(ALL_BUILD_DIRS): 32 | @mkdir -p $@ 33 | 34 | $(BUILD_DIR)/%.o: %.cpp | $(ALL_BUILD_DIRS) 35 | @echo "CXX" $< 36 | @$(CXX) $(CXXFLAGS) -c -o $@ $< 37 | 38 | $(PROJECT_NAME): $(CXX_TARGETS) 39 | @echo "CXX/LD" $@ 40 | @$(CXX) -o $@ $^ $(LDFLAGS) 41 | 42 | .PHONY: clean 43 | clean: 44 | @rm -rf $(CXX_TARGETS) 45 | @rm -rf $(PROJECT_NAME) 46 | @rm -rf $(BUILD_DIR) 47 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/Run.sh: -------------------------------------------------------------------------------- 1 | root=../../ 2 | data_dir=${root}data/CULane/ 3 | exp=vgg_SCNN_DULR_w9 4 | detect_dir=${root}tools/prob2lines/output/${exp}/ 5 | w_lane=30; 6 | iou=0.5; # Set iou to 0.3 or 0.5 7 | im_w=1640 8 | im_h=590 9 | frame=1 10 | list0=${data_dir}list/test_split/test0_normal.txt 11 | list1=${data_dir}list/test_split/test1_crowd.txt 12 | list2=${data_dir}list/test_split/test2_hlight.txt 13 | list3=${data_dir}list/test_split/test3_shadow.txt 14 | list4=${data_dir}list/test_split/test4_noline.txt 15 | list5=${data_dir}list/test_split/test5_arrow.txt 16 | list6=${data_dir}list/test_split/test6_curve.txt 17 | list7=${data_dir}list/test_split/test7_cross.txt 18 | list8=${data_dir}list/test_split/test8_night.txt 19 | out0=./output/out0_normal.txt 20 | out1=./output/out1_crowd.txt 21 | out2=./output/out2_hlight.txt 22 | out3=./output/out3_shadow.txt 23 | out4=./output/out4_noline.txt 24 | out5=./output/out5_arrow.txt 25 | out6=./output/out6_curve.txt 26 | out7=./output/out7_cross.txt 27 | out8=./output/out8_night.txt 28 | ./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list0 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out0 29 | ./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list1 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out1 30 | ./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list2 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out2 31 | ./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list3 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out3 32 | ./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list4 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out4 33 | ./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list5 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out5 34 | ./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list6 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out6 35 | ./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list7 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out7 36 | ./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list8 -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out8 37 | cat ./output/out*.txt>./output/${exp}_iou${iou}_split.txt 38 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/calTotal.m: -------------------------------------------------------------------------------- 1 | %% Calculate overall Fmeasure from each scenarios 2 | clc; clear; close all; 3 | 4 | allFile = 'output/vgg_SCNN_DULR_w9_iou0.5.txt'; 5 | 6 | all = textread(allFile,'%s'); 7 | TP = 0; 8 | FP = 0; 9 | FN = 0; 10 | 11 | for i=1:9 12 | tpline = (i-1)*14+4; 13 | tp = str2double(all(tpline)); 14 | fp = str2double(all(tpline+2)); 15 | fn = str2double(all(tpline+4)); 16 | TP = TP + tp; 17 | FP = FP + fp; 18 | FN = FN + fn; 19 | end 20 | 21 | P = TP/(TP + FP) 22 | R = TP/(TP + FN) 23 | F = 2*P*R/(P + R)*100 24 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/evl/evaluate.exe: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Mouiad-JRA/Lane-Line-Detection-using-Image-Processing-vs-Deep-Learning/f713474a3a9c005ab8926f6decacaa8225df5984/SCNN-Tensorflow/lane_evaluation/evl/evaluate.exe -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/evl/results.txt: -------------------------------------------------------------------------------- 1 | file: results.txt 2 | tp: 4 fp: 0 fn: 0 3 | precision: 1 4 | recall: 1 5 | Fmeasure: 1 6 | 7 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/evl/run.bat: -------------------------------------------------------------------------------- 1 | evaluate -a C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\data\CULane -d C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\prob2lines\output\vgg_SCNN_DULR_w9 -i C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\data\CULane -l C:\Users\Mouiad\Desktop\Codes-for-Lane-Detection\SCNN-Tensorflow\lane-detection-model\data\CULane\list\test.txt -w 30 -t 0.5 -c 1242 -r 375 -s -f 1 -o results.txt 2 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/include/counter.hpp: -------------------------------------------------------------------------------- 1 | #ifndef COUNTER_HPP 2 | #define COUNTER_HPP 3 | 4 | #include "lane_compare.hpp" 5 | #include "hungarianGraph.hpp" 6 | #include 7 | #include 8 | #include 9 | #include 10 | #include 11 | 12 | using namespace std; 13 | using namespace cv; 14 | 15 | // before coming to use functions of this class, the lanes should resize to im_width and im_height using resize_lane() in lane_compare.hpp 16 | class Counter 17 | { 18 | public: 19 | Counter(int _im_width, int _im_height, double _iou_threshold=0.4, int _lane_width=10):tp(0),fp(0),fn(0){ 20 | im_width = _im_width; 21 | im_height = _im_height; 22 | sim_threshold = _iou_threshold; 23 | lane_compare = new LaneCompare(_im_width, _im_height, _lane_width, LaneCompare::IOU); 24 | }; 25 | double get_precision(void); 26 | double get_recall(void); 27 | long getTP(void); 28 | long getFP(void); 29 | long getFN(void); 30 | void setTP(long); 31 | void setFP(long); 32 | void setFN(long); 33 | // direct add tp, fp, tn and fn 34 | // first match with hungarian 35 | tuple, long, long, long, long> count_im_pair(const vector > &anno_lanes, const vector > &detect_lanes); 36 | void makeMatch(const vector > &similarity, vector &match1, vector &match2); 37 | 38 | private: 39 | double sim_threshold; 40 | int im_width; 41 | int im_height; 42 | long tp; 43 | long fp; 44 | long fn; 45 | LaneCompare *lane_compare; 46 | }; 47 | #endif 48 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/include/hungarianGraph.hpp: -------------------------------------------------------------------------------- 1 | #ifndef HUNGARIAN_GRAPH_HPP 2 | #define HUNGARIAN_GRAPH_HPP 3 | #include 4 | using namespace std; 5 | 6 | struct pipartiteGraph { 7 | vector > mat; 8 | vector leftUsed, rightUsed; 9 | vector leftWeight, rightWeight; 10 | vectorrightMatch, leftMatch; 11 | int leftNum, rightNum; 12 | bool matchDfs(int u) { 13 | leftUsed[u] = true; 14 | for (int v = 0; v < rightNum; v++) { 15 | if (!rightUsed[v] && fabs(leftWeight[u] + rightWeight[v] - mat[u][v]) < 1e-2) { 16 | rightUsed[v] = true; 17 | if (rightMatch[v] == -1 || matchDfs(rightMatch[v])) { 18 | rightMatch[v] = u; 19 | leftMatch[u] = v; 20 | return true; 21 | } 22 | } 23 | } 24 | return false; 25 | } 26 | void resize(int leftNum, int rightNum) { 27 | this->leftNum = leftNum; 28 | this->rightNum = rightNum; 29 | leftMatch.resize(leftNum); 30 | rightMatch.resize(rightNum); 31 | leftUsed.resize(leftNum); 32 | rightUsed.resize(rightNum); 33 | leftWeight.resize(leftNum); 34 | rightWeight.resize(rightNum); 35 | mat.resize(leftNum); 36 | for (int i = 0; i < leftNum; i++) mat[i].resize(rightNum); 37 | } 38 | void match() { 39 | for (int i = 0; i < leftNum; i++) leftMatch[i] = -1; 40 | for (int i = 0; i < rightNum; i++) rightMatch[i] = -1; 41 | for (int i = 0; i < rightNum; i++) rightWeight[i] = 0; 42 | for (int i = 0; i < leftNum; i++) { 43 | leftWeight[i] = -1e5; 44 | for (int j = 0; j < rightNum; j++) { 45 | if (leftWeight[i] < mat[i][j]) leftWeight[i] = mat[i][j]; 46 | } 47 | } 48 | 49 | for (int u = 0; u < leftNum; u++) { 50 | while (1) { 51 | for (int i = 0; i < leftNum; i++) leftUsed[i] = false; 52 | for (int i = 0; i < rightNum; i++) rightUsed[i] = false; 53 | if (matchDfs(u)) break; 54 | double d = 1e10; 55 | for (int i = 0; i < leftNum; i++) { 56 | if (leftUsed[i] ) { 57 | for (int j = 0; j < rightNum; j++) { 58 | if (!rightUsed[j]) d = min(d, leftWeight[i] + rightWeight[j] - mat[i][j]); 59 | } 60 | } 61 | } 62 | if (d == 1e10) return ; 63 | for (int i = 0; i < leftNum; i++) if (leftUsed[i]) leftWeight[i] -= d; 64 | for (int i = 0; i < rightNum; i++) if (rightUsed[i]) rightWeight[i] += d; 65 | } 66 | } 67 | } 68 | }; 69 | 70 | 71 | #endif // HUNGARIAN_GRAPH_HPP 72 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/include/lane_compare.hpp: -------------------------------------------------------------------------------- 1 | #ifndef LANE_COMPARE_HPP 2 | #define LANE_COMPARE_HPP 3 | 4 | #include "spline.hpp" 5 | #include 6 | #include 7 | #include 8 | #include 9 | 10 | using namespace std; 11 | using namespace cv; 12 | 13 | class LaneCompare{ 14 | public: 15 | enum CompareMode{ 16 | IOU, 17 | Caltech 18 | }; 19 | 20 | LaneCompare(int _im_width, int _im_height, int _lane_width = 10, CompareMode _compare_mode = IOU){ 21 | im_width = _im_width; 22 | im_height = _im_height; 23 | compare_mode = _compare_mode; 24 | lane_width = _lane_width; 25 | } 26 | 27 | double get_lane_similarity(const vector &lane1, const vector &lane2); 28 | void resize_lane(vector &curr_lane, int curr_width, int curr_height); 29 | private: 30 | CompareMode compare_mode; 31 | int im_width; 32 | int im_height; 33 | int lane_width; 34 | Spline splineSolver; 35 | }; 36 | 37 | #endif 38 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/include/spline.hpp: -------------------------------------------------------------------------------- 1 | #ifndef SPLINE_HPP 2 | #define SPLINE_HPP 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | using namespace cv; 9 | using namespace std; 10 | 11 | struct Func { 12 | double a_x; 13 | double b_x; 14 | double c_x; 15 | double d_x; 16 | double a_y; 17 | double b_y; 18 | double c_y; 19 | double d_y; 20 | double h; 21 | }; 22 | class Spline { 23 | public: 24 | vector splineInterpTimes(const vector &tmp_line, int times); 25 | vector splineInterpStep(vector tmp_line, double step); 26 | vector cal_fun(const vector &point_v); 27 | }; 28 | #endif 29 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/run2.sh: -------------------------------------------------------------------------------- 1 | root=../../ 2 | data_dir=${root}data/CULane/ 3 | exp=vgg_SCNN_DULR_w9 4 | detect_dir=${root}tools/prob2lines/output/${exp}/ 5 | w_lane=30; 6 | iou=0.5; # Set iou to 0.3 or 0.5 7 | im_w=1640 8 | im_h=590 9 | frame=1 10 | list=${data_dir}list/test.txt 11 | out=./output/${exp}_iou${iou}.txt 12 | ./evaluate -a $data_dir -d $detect_dir -i $data_dir -l $list -w $w_lane -t $iou -c $im_w -r $im_h -f $frame -o $out 13 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/src/counter.cpp: -------------------------------------------------------------------------------- 1 | /************************************************************************* 2 | > File Name: counter.cpp 3 | > Author: Xingang Pan, Jun Li 4 | > Mail: px117@ie.cuhk.edu.hk 5 | > Created Time: Thu Jul 14 20:23:08 2016 6 | ************************************************************************/ 7 | 8 | #include "counter.hpp" 9 | 10 | double Counter::get_precision(void) 11 | { 12 | cerr<<"tp: "<, long, long, long, long> Counter::count_im_pair(const vector > &anno_lanes, const vector > &detect_lanes) 62 | { 63 | vector anno_match(anno_lanes.size(), -1); 64 | vector detect_match; 65 | if(anno_lanes.empty()) 66 | { 67 | return make_tuple(anno_match, 0, detect_lanes.size(), 0, 0); 68 | } 69 | 70 | if(detect_lanes.empty()) 71 | { 72 | return make_tuple(anno_match, 0, 0, 0, anno_lanes.size()); 73 | } 74 | // hungarian match first 75 | 76 | // first calc similarity matrix 77 | vector > similarity(anno_lanes.size(), vector(detect_lanes.size(), 0)); 78 | for(int i=0; i &curr_anno_lane = anno_lanes[i]; 81 | for(int j=0; j &curr_detect_lane = detect_lanes[j]; 84 | similarity[i][j] = lane_compare->get_lane_similarity(curr_anno_lane, curr_detect_lane); 85 | } 86 | } 87 | 88 | 89 | 90 | makeMatch(similarity, anno_match, detect_match); 91 | 92 | 93 | int curr_tp = 0; 94 | // count and add 95 | for(int i=0; i=0 && similarity[i][anno_match[i]] > sim_threshold) 98 | { 99 | curr_tp++; 100 | } 101 | else 102 | { 103 | anno_match[i] = -1; 104 | } 105 | } 106 | int curr_fn = anno_lanes.size() - curr_tp; 107 | int curr_fp = detect_lanes.size() - curr_tp; 108 | return make_tuple(anno_match, curr_tp, curr_fp, 0, curr_fn); 109 | } 110 | 111 | 112 | void Counter::makeMatch(const vector > &similarity, vector &match1, vector &match2) { 113 | int m = similarity.size(); 114 | int n = similarity[0].size(); 115 | pipartiteGraph gra; 116 | bool have_exchange = false; 117 | if (m > n) { 118 | have_exchange = true; 119 | swap(m, n); 120 | } 121 | gra.resize(m, n); 122 | for (int i = 0; i < gra.leftNum; i++) { 123 | for (int j = 0; j < gra.rightNum; j++) { 124 | if(have_exchange) 125 | gra.mat[i][j] = similarity[j][i]; 126 | else 127 | gra.mat[i][j] = similarity[i][j]; 128 | } 129 | } 130 | gra.match(); 131 | match1 = gra.leftMatch; 132 | match2 = gra.rightMatch; 133 | if (have_exchange) swap(match1, match2); 134 | } 135 | -------------------------------------------------------------------------------- /SCNN-Tensorflow/lane_evaluation/src/evaluate.cpp: -------------------------------------------------------------------------------- 1 | /************************************************************************* 2 | > File Name: evaluate.cpp 3 | > Author: Xingang Pan, Jun Li 4 | > Mail: px117@ie.cuhk.edu.hk 5 | > Created Time: 2016年07月14日星期四 18时28分45秒 6 | ************************************************************************/ 7 | 8 | #include "counter.hpp" 9 | #include "spline.hpp" 10 | #include 11 | #include 12 | #include 13 | #include 14 | #include 15 | #include 16 | #include 17 | #include 18 | using namespace std; 19 | using namespace cv; 20 | 21 | void help(void) 22 | { 23 | cout<<"./evaluate [OPTIONS]"< > &lanes); 39 | void visualize(string &full_im_name, vector > &anno_lanes, vector > &detect_lanes, vector anno_match, int width_lane); 40 | 41 | int main(int argc, char **argv) 42 | { 43 | // process params 44 | string anno_dir = "/data/driving/eval_data/anno_label/"; 45 | string detect_dir = "/data/driving/eval_data/predict_label/"; 46 | string im_dir = "/data/driving/eval_data/img/"; 47 | string list_im_file = "/data/driving/eval_data/img/all.txt"; 48 | string output_file = "./output.txt"; 49 | int width_lane = 10; 50 | double iou_threshold = 0.4; 51 | int im_width = 1920; 52 | int im_height = 1080; 53 | int oc; 54 | bool show = false; 55 | int frame = 1; 56 | while((oc = getopt(argc, argv, "ha:d:i:l:w:t:c:r:sf:o:")) != -1) 57 | { 58 | switch(oc) 59 | { 60 | case 'h': 61 | help(); 62 | return 0; 63 | case 'a': 64 | anno_dir = optarg; 65 | break; 66 | case 'd': 67 | detect_dir = optarg; 68 | break; 69 | case 'i': 70 | im_dir = optarg; 71 | break; 72 | case 'l': 73 | list_im_file = optarg; 74 | break; 75 | case 'w': 76 | width_lane = atoi(optarg); 77 | break; 78 | case 't': 79 | iou_threshold = atof(optarg); 80 | break; 81 | case 'c': 82 | im_width = atoi(optarg); 83 | break; 84 | case 'r': 85 | im_height = atoi(optarg); 86 | break; 87 | case 's': 88 | show = true; 89 | break; 90 | case 'f': 91 | frame = atoi(optarg); 92 | break; 93 | case 'o': 94 | output_file = optarg; 95 | break; 96 | } 97 | } 98 | 99 | 100 | cout<<"------------Configuration---------"< anno_match; 132 | string sub_im_name; 133 | // pre-load filelist 134 | vector filelists; 135 | while (getline(ifs_im_list, sub_im_name)) { 136 | filelists.push_back(sub_im_name); 137 | } 138 | ifs_im_list.close(); 139 | 140 | vector, long, long, long, long>> tuple_lists; 141 | tuple_lists.resize(filelists.size()); 142 | 143 | #pragma omp parallel for 144 | for (size_t i = 0; i < filelists.size(); i++) 145 | { 146 | auto sub_im_name = filelists[i]; 147 | string full_im_name = im_dir + sub_im_name; 148 | string sub_txt_name = sub_im_name.substr(0, sub_im_name.find_last_of(".")) + ".lines.txt"; 149 | string anno_file_name = anno_dir + sub_txt_name; 150 | string detect_file_name = detect_dir + sub_txt_name; 151 | vector > anno_lanes; 152 | vector > detect_lanes; 153 | read_lane_file(anno_file_name, anno_lanes); 154 | read_lane_file(detect_file_name, detect_lanes); 155 | //cerr<(tuple_lists[i]); 160 | visualize(full_im_name, anno_lanes, detect_lanes, anno_match, width_lane); 161 | waitKey(0); 162 | } 163 | } 164 | 165 | long tp = 0, fp = 0, tn = 0, fn = 0; 166 | for (auto result: tuple_lists) { 167 | tp += get<1>(result); 168 | fp += get<2>(result); 169 | // tn = get<3>(result); 170 | fn += get<4>(result); 171 | } 172 | counter.setTP(tp); 173 | counter.setFP(fp); 174 | counter.setFN(fn); 175 | 176 | double precision = counter.get_precision(); 177 | double recall = counter.get_recall(); 178 | double F = 2 * precision * recall / (precision + recall); 179 | cerr<<"finished process file"< > &lanes) 197 | { 198 | lanes.clear(); 199 | ifstream ifs_lane(file_name, ios::in); 200 | if(ifs_lane.fail()) 201 | { 202 | return; 203 | } 204 | 205 | string str_line; 206 | while(getline(ifs_lane, str_line)) 207 | { 208 | vector curr_lane; 209 | stringstream ss; 210 | ss<>x>>y) 213 | { 214 | curr_lane.push_back(Point2f(x, y)); 215 | } 216 | lanes.push_back(curr_lane); 217 | } 218 | 219 | ifs_lane.close(); 220 | } 221 | 222 | void visualize(string &full_im_name, vector > &anno_lanes, vector > &detect_lanes, vector anno_match, int width_lane) 223 | { 224 | Mat img = imread(full_im_name, 1); 225 | Mat img2 = imread(full_im_name, 1); 226 | vector curr_lane; 227 | vector p_interp; 228 | Spline splineSolver; 229 | Scalar color_B = Scalar(255, 0, 0); 230 | Scalar color_G = Scalar(0, 255, 0); 231 | Scalar color_R = Scalar(0, 0, 255); 232 | Scalar color_P = Scalar(255, 0, 255); 233 | Scalar color; 234 | for (int i=0; i= 0) 246 | { 247 | color = color_G; 248 | } 249 | else 250 | { 251 | color = color_G; 252 | } 253 | for (int n=0; n File Name: lane_compare.cpp 3 | > Author: Xingang Pan, Jun Li 4 | > Mail: px117@ie.cuhk.edu.hk 5 | > Created Time: Fri Jul 15 10:26:32 2016 6 | ************************************************************************/ 7 | 8 | #include "lane_compare.hpp" 9 | 10 | double LaneCompare::get_lane_similarity(const vector &lane1, const vector &lane2) 11 | { 12 | if(lane1.size()<2 || lane2.size()<2) 13 | { 14 | cerr<<"lane size must be greater or equal to 2"< p_interp1; 21 | vector p_interp2; 22 | if(lane1.size() == 2) 23 | { 24 | p_interp1 = lane1; 25 | } 26 | else 27 | { 28 | p_interp1 = splineSolver.splineInterpTimes(lane1, 50); 29 | } 30 | 31 | if(lane2.size() == 2) 32 | { 33 | p_interp2 = lane2; 34 | } 35 | else 36 | { 37 | p_interp2 = splineSolver.splineInterpTimes(lane2, 50); 38 | } 39 | 40 | Scalar color_white = Scalar(1); 41 | for(int n=0; n &curr_lane, int curr_width, int curr_height) 61 | { 62 | if(curr_width == im_width && curr_height == im_height) 63 | { 64 | return; 65 | } 66 | double x_scale = im_width/(double)curr_width; 67 | double y_scale = im_height/(double)curr_height; 68 | for(int n=0; n 2 | #include 3 | #include "spline.hpp" 4 | using namespace std; 5 | using namespace cv; 6 | 7 | vector Spline::splineInterpTimes(const vector& tmp_line, int times) { 8 | vector res; 9 | 10 | if(tmp_line.size() == 2) { 11 | double x1 = tmp_line[0].x; 12 | double y1 = tmp_line[0].y; 13 | double x2 = tmp_line[1].x; 14 | double y2 = tmp_line[1].y; 15 | 16 | for (int k = 0; k <= times; k++) { 17 | double xi = x1 + double((x2 - x1) * k) / times; 18 | double yi = y1 + double((y2 - y1) * k) / times; 19 | res.push_back(Point2f(xi, yi)); 20 | } 21 | } 22 | 23 | else if(tmp_line.size() > 2) 24 | { 25 | vector tmp_func; 26 | tmp_func = this->cal_fun(tmp_line); 27 | if (tmp_func.empty()) { 28 | cout << "in splineInterpTimes: cal_fun failed" << endl; 29 | return res; 30 | } 31 | for(int j = 0; j < tmp_func.size(); j++) 32 | { 33 | double delta = tmp_func[j].h / times; 34 | for(int k = 0; k < times; k++) 35 | { 36 | double t1 = delta*k; 37 | double x1 = tmp_func[j].a_x + tmp_func[j].b_x*t1 + tmp_func[j].c_x*pow(t1,2) + tmp_func[j].d_x*pow(t1,3); 38 | double y1 = tmp_func[j].a_y + tmp_func[j].b_y*t1 + tmp_func[j].c_y*pow(t1,2) + tmp_func[j].d_y*pow(t1,3); 39 | res.push_back(Point2f(x1, y1)); 40 | } 41 | } 42 | res.push_back(tmp_line[tmp_line.size() - 1]); 43 | } 44 | else { 45 | cerr << "in splineInterpTimes: not enough points" << endl; 46 | } 47 | return res; 48 | } 49 | vector Spline::splineInterpStep(vector tmp_line, double step) { 50 | vector res; 51 | /* 52 | if (tmp_line.size() == 2) { 53 | double x1 = tmp_line[0].x; 54 | double y1 = tmp_line[0].y; 55 | double x2 = tmp_line[1].x; 56 | double y2 = tmp_line[1].y; 57 | 58 | for (double yi = std::min(y1, y2); yi < std::max(y1, y2); yi += step) { 59 | double xi; 60 | if (yi == y1) xi = x1; 61 | else xi = (x2 - x1) / (y2 - y1) * (yi - y1) + x1; 62 | res.push_back(Point2f(xi, yi)); 63 | } 64 | }*/ 65 | if (tmp_line.size() == 2) { 66 | double x1 = tmp_line[0].x; 67 | double y1 = tmp_line[0].y; 68 | double x2 = tmp_line[1].x; 69 | double y2 = tmp_line[1].y; 70 | tmp_line[1].x = (x1 + x2) / 2; 71 | tmp_line[1].y = (y1 + y2) / 2; 72 | tmp_line.push_back(Point2f(x2, y2)); 73 | } 74 | if (tmp_line.size() > 2) { 75 | vector tmp_func; 76 | tmp_func = this->cal_fun(tmp_line); 77 | double ystart = tmp_line[0].y; 78 | double yend = tmp_line[tmp_line.size() - 1].y; 79 | bool down; 80 | if (ystart < yend) down = 1; 81 | else down = 0; 82 | if (tmp_func.empty()) { 83 | cerr << "in splineInterpStep: cal_fun failed" << endl; 84 | } 85 | 86 | for(int j = 0; j < tmp_func.size(); j++) 87 | { 88 | for(double t1 = 0; t1 < tmp_func[j].h; t1 += step) 89 | { 90 | double x1 = tmp_func[j].a_x + tmp_func[j].b_x*t1 + tmp_func[j].c_x*pow(t1,2) + tmp_func[j].d_x*pow(t1,3); 91 | double y1 = tmp_func[j].a_y + tmp_func[j].b_y*t1 + tmp_func[j].c_y*pow(t1,2) + tmp_func[j].d_y*pow(t1,3); 92 | res.push_back(Point2f(x1, y1)); 93 | } 94 | } 95 | res.push_back(tmp_line[tmp_line.size() - 1]); 96 | } 97 | else { 98 | cerr << "in splineInterpStep: not enough points" << endl; 99 | } 100 | return res; 101 | } 102 | 103 | vector Spline::cal_fun(const vector &point_v) 104 | { 105 | vector func_v; 106 | int n = point_v.size(); 107 | if(n<=2) { 108 | cout << "in cal_fun: point number less than 3" << endl; 109 | return func_v; 110 | } 111 | 112 | func_v.resize(point_v.size()-1); 113 | 114 | vector Mx(n); 115 | vector My(n); 116 | vector A(n-2); 117 | vector B(n-2); 118 | vector C(n-2); 119 | vector Dx(n-2); 120 | vector Dy(n-2); 121 | vector h(n-1); 122 | //vector func_v(n-1); 123 | 124 | for(int i = 0; i < n-1; i++) 125 | { 126 | h[i] = sqrt(pow(point_v[i+1].x - point_v[i].x, 2) + pow(point_v[i+1].y - point_v[i].y, 2)); 127 | } 128 | 129 | for(int i = 0; i < n-2; i++) 130 | { 131 | A[i] = h[i]; 132 | B[i] = 2*(h[i]+h[i+1]); 133 | C[i] = h[i+1]; 134 | 135 | Dx[i] = 6*( (point_v[i+2].x - point_v[i+1].x)/h[i+1] - (point_v[i+1].x - point_v[i].x)/h[i] ); 136 | Dy[i] = 6*( (point_v[i+2].y - point_v[i+1].y)/h[i+1] - (point_v[i+1].y - point_v[i].y)/h[i] ); 137 | } 138 | 139 | //TDMA 140 | C[0] = C[0] / B[0]; 141 | Dx[0] = Dx[0] / B[0]; 142 | Dy[0] = Dy[0] / B[0]; 143 | for(int i = 1; i < n-2; i++) 144 | { 145 | double tmp = B[i] - A[i]*C[i-1]; 146 | C[i] = C[i] / tmp; 147 | Dx[i] = (Dx[i] - A[i]*Dx[i-1]) / tmp; 148 | Dy[i] = (Dy[i] - A[i]*Dy[i-1]) / tmp; 149 | } 150 | Mx[n-2] = Dx[n-3]; 151 | My[n-2] = Dy[n-3]; 152 | for(int i = n-4; i >= 0; i--) 153 | { 154 | Mx[i+1] = Dx[i] - C[i]*Mx[i+2]; 155 | My[i+1] = Dy[i] - C[i]*My[i+2]; 156 | } 157 | 158 | Mx[0] = 0; 159 | Mx[n-1] = 0; 160 | My[0] = 0; 161 | My[n-1] = 0; 162 | 163 | for(int i = 0; i < n-1; i++) 164 | { 165 | func_v[i].a_x = point_v[i].x; 166 | func_v[i].b_x = (point_v[i+1].x - point_v[i].x)/h[i] - (2*h[i]*Mx[i] + h[i]*Mx[i+1]) / 6; 167 | func_v[i].c_x = Mx[i]/2; 168 | func_v[i].d_x = (Mx[i+1] - Mx[i]) / (6*h[i]); 169 | 170 | func_v[i].a_y = point_v[i].y; 171 | func_v[i].b_y = (point_v[i+1].y - point_v[i].y)/h[i] - (2*h[i]*My[i] + h[i]*My[i+1]) / 6; 172 | func_v[i].c_y = My[i]/2; 173 | func_v[i].d_y = (My[i+1] - My[i]) / (6*h[i]); 174 | 175 | func_v[i].h = h[i]; 176 | } 177 | return func_v; 178 | } 179 | --------------------------------------------------------------------------------