├── Images ├── 1.jpg ├── 10.jpg ├── 11.jpg ├── 12.jpg ├── 13.png ├── 14.JPG ├── 15.jpg ├── 16.jpg ├── 2.jpg ├── 3.jpg ├── 4.JPG ├── 5.jpg ├── 6.jpg ├── 7.jpg ├── 8.jpg └── 9.jpg ├── LicPlateOCRGenData ├── classifications.xml ├── frmMain.vb ├── images.xml └── lic_plate_training_chars.png ├── LicensePlateRecognition ├── DetectChars.vb ├── DetectPlates.vb ├── PossibleChar.vb ├── PossiblePlate.vb ├── Preprocess.vb └── frmMain.vb ├── presentation ├── Steps With Images.docx ├── Steps With Images.pdf ├── form_design.png ├── steps.png └── steps.txt └── readme.txt /Images/1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/1.jpg -------------------------------------------------------------------------------- /Images/10.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/10.jpg -------------------------------------------------------------------------------- /Images/11.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/11.jpg -------------------------------------------------------------------------------- /Images/12.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/12.jpg -------------------------------------------------------------------------------- /Images/13.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/13.png -------------------------------------------------------------------------------- /Images/14.JPG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/14.JPG -------------------------------------------------------------------------------- /Images/15.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/15.jpg -------------------------------------------------------------------------------- /Images/16.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/16.jpg -------------------------------------------------------------------------------- /Images/2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/2.jpg -------------------------------------------------------------------------------- /Images/3.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/3.jpg -------------------------------------------------------------------------------- /Images/4.JPG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/4.JPG -------------------------------------------------------------------------------- /Images/5.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/5.jpg -------------------------------------------------------------------------------- /Images/6.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/6.jpg -------------------------------------------------------------------------------- /Images/7.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/7.jpg -------------------------------------------------------------------------------- /Images/8.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/8.jpg -------------------------------------------------------------------------------- /Images/9.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/Images/9.jpg -------------------------------------------------------------------------------- /LicPlateOCRGenData/classifications.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | AAC0QgAAqEIAAKRCAACgQgAAmkIAAIxCAACKQgAAiEIAAIRCAACCQgAAskIAALBCAACuQgAArEIAAKpCAACmQgAAokIAAJ5CAACcQgAAmEIAAJZCAACUQgAAkkIAAJBCAACOQgAAhkIAAGRCAABgQgAAXEIAAFhCAABUQgAATEIAAEhCAABAQgAAUEIAAERCAACkQgAAoEIAAJpCAACIQgAAhEIAAIJCAAC0QgAAskIAALBCAACuQgAArEIAAKpCAACoQgAApkIAAKJCAACeQgAAnEIAAJhCAACWQgAAlEIAAJJCAACQQgAAjkIAAIxCAACKQgAAhkIAAFBCAABEQgAAZEIAAGBCAABcQgAAWEIAAFRCAABMQgAASEIAAEBCAAC0QgAAskIAALBCAACuQgAArEIAAKpCAACoQgAApkIAAKRCAACiQgAAoEIAAJ5CAACcQgAAmkIAAJhCAACWQgAAlEIAAJJCAACQQgAAjkIAAIxCAACKQgAAiEIAAIZCAACEQgAAgkIAAERCAABkQgAAYEIAAFxCAABYQgAAVEIAAFBCAABMQgAASEIAAEBCAAC0QgAAskIAALBCAACuQgAArEIAAKpCAACoQgAApEIAAKBCAACcQgAAmkIAAJhCAACWQgAAlEIAAJJCAACQQgAAjEIAAIpCAACIQgAAhEIAAIJCAACmQgAAokIAAJ5CAACOQgAAhkIAAFxCAABMQgAASEIAAERCAABkQgAAYEIAAFhCAABUQgAAUEIAAEBCAACkQgAAoEIAAJpCAACIQgAAhEIAAIJCAAC0QgAAskIAALBCAACuQgAArEIAAKpCAACoQgAApkIAAKJCAACeQgAAnEIAAJhCAACWQgAAlEIAAJJCAACQQgAAjkIAAIxCAACKQgAAhkIAAFxCAABUQgAAUEIAAGRCAABgQgAAWEIAAExCAABIQgAAREIAAEBC 4 | -------------------------------------------------------------------------------- /LicPlateOCRGenData/frmMain.vb: -------------------------------------------------------------------------------- 1 | 'LicPlateOCRGenData.vb 2 | ' 3 | 'Emgu CV 2.4.10 4 | ' 5 | 'add the following components to your form: 6 | 'tableLayoutPanel (TableLayoutPanel) 7 | 'btnOpenTrainingImage (Button) 8 | 'lblChosenFile (Label) 9 | 'txtInfo (TextBox) 10 | 'ofdOpenFile (OpenFileDialog) 11 | 12 | Option Explicit On 'require explicit declaration of variables, this is NOT Python !! 13 | Option Strict On 'restrict implicit data type conversions to only widening conversions 14 | 15 | Imports Emgu.CV ' 16 | Imports Emgu.CV.CvEnum 'Emgu Cv imports 17 | Imports Emgu.CV.Structure ' 18 | Imports Emgu.CV.UI ' 19 | Imports Emgu.CV.ML ' 20 | 21 | Imports System.Xml ' 22 | Imports System.Xml.Serialization 'these imports are for writing Matrix objects to file, see end of program 23 | Imports System.IO ' 24 | 25 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 26 | Public Class frmMain 27 | 28 | ' module level variables '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 29 | Const MIN_CONTOUR_AREA As Integer = 100 30 | 31 | Const RESIZED_IMAGE_WIDTH As Integer = 20 32 | Const RESIZED_IMAGE_HEIGHT As Integer = 30 33 | 34 | Dim intNumberOfTrainingSamples As Integer 35 | 36 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 37 | Private Sub btnOpenTrainingImage_Click( sender As Object, e As EventArgs) Handles btnOpenTrainingImage.Click 38 | Dim drChosenFile As DialogResult 39 | 40 | drChosenFile = ofdOpenFile.ShowDialog() 'open file dialog 41 | 42 | If (drChosenFile <> Windows.Forms.DialogResult.OK Or ofdOpenFile.FileName = "") Then 'if user chose Cancel or filename is blank . . . 43 | lblChosenFile.Text = "file not chosen" 'show error message on label 44 | Return 'and exit function 45 | End If 46 | 47 | Dim imgTrainingNumbers As Image(Of Bgr, Byte) 'this is the main input image 48 | 49 | Try 50 | imgTrainingNumbers = New Image(Of Bgr, Byte)(ofdOpenFile.FileName) 'open image 51 | Catch ex As Exception 'if error occurred 52 | lblChosenFile.Text = "unable to open image, error: " + ex.Message 'show error message on label 53 | Return 'and exit function 54 | End Try 55 | 56 | If imgTrainingNumbers Is Nothing Then 'if image could not be opened 57 | lblChosenFile.Text = "unable to open image" 'show error message on label 58 | Return 'and exit function 59 | End If 60 | 61 | lblChosenFile.Text = ofdOpenFile.FileName 'update label with file name 62 | 63 | Dim imgGrayscale As Image(Of Gray, Byte) ' 64 | Dim imgBlurred As Image(Of Gray, Byte) 'declare various images 65 | Dim imgThresh As Image(Of Gray, Byte) ' 66 | Dim imgThreshCopy As Image(Of Gray, Byte) ' 67 | Dim imgContours As Image(Of Gray, Byte) ' 68 | 69 | Dim contours As Contour(Of Point) 70 | Dim listOfContours As List(Of Contour(Of Point)) = New List(Of Contour(Of Point)) 71 | 72 | 'possible chars we are interested in are digits 0 through 9, put these in list intValidChars 73 | Dim intValidChars As New List(Of Integer)(New Integer() { Asc("0"), Asc("1"), Asc("2"), Asc("3"), Asc("4"), Asc("5"), Asc("6"), Asc("7"), Asc("8"), Asc("9"), _ 74 | Asc("A"), Asc("B"), Asc("C"), Asc("D"), Asc("E"), Asc("F"), Asc("G"), Asc("H"), Asc("I"), Asc("J"), _ 75 | Asc("K"), Asc("L"), Asc("M"), Asc("N"), Asc("O"), Asc("P"), Asc("Q"), Asc("R"), Asc("S"), Asc("T"), _ 76 | Asc("U"), Asc("V"), Asc("W"), Asc("X"), Asc("Y"), Asc("Z") } ) 77 | 78 | imgGrayscale = imgTrainingNumbers.Convert(Of Gray, Byte)() 'convert to grayscale 79 | 80 | imgBlurred = imgGrayscale.SmoothGaussian(5) 'blur 81 | 82 | imgBlurred = imgGrayscale.SmoothGaussian(5) 'blur 83 | 84 | 'filter image from grayscale to black and white 85 | imgThresh = imgBlurred.ThresholdAdaptive(New Gray(255), ADAPTIVE_THRESHOLD_TYPE.CV_ADAPTIVE_THRESH_GAUSSIAN_C, THRESH.CV_THRESH_BINARY_INV, 11, New Gray(2)) 86 | 87 | CvInvoke.cvShowImage("imgThresh", imgThresh) 'show threshold image for reference 88 | 89 | imgThreshCopy = imgThresh.Clone() 'make a copy of the thresh image, this in necessary b/c findContours modifies the image 90 | 91 | 'get external countours only 92 | contours = imgThreshCopy.FindContours(CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE, RETR_TYPE.CV_RETR_EXTERNAL) 93 | 94 | 'next we count the contours 95 | intNumberOfTrainingSamples = 0 'init number of contours (i.e. training samples) to zero 96 | 97 | While (Not contours Is Nothing) 98 | intNumberOfTrainingSamples = intNumberOfTrainingSamples + 1 99 | contours = contours.HNext 100 | End While 101 | 102 | contours = imgThreshCopy.FindContours(CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE, RETR_TYPE.CV_RETR_EXTERNAL) 'get contours again to go back to beginning 103 | 104 | imgContours = New Image(Of Gray, Byte)(imgThresh.Size()) 'instantiate contours image 105 | 106 | 'draw contours onto contours image 107 | CvInvoke.cvDrawContours(imgContours, contours, New MCvScalar(255), New MCvScalar(255), 100, 1, LINE_TYPE.CV_AA, New Point(0, 0)) 108 | 109 | CvInvoke.cvShowImage("imgContours", imgContours) 'show contours image for reference 110 | 111 | 'this is our classifications data structure 112 | Dim mtxClassifications As Matrix(Of Single) = New Matrix(Of Single)(intNumberOfTrainingSamples, 1) 113 | 114 | 'this is our training images data structure, note we will have to perform some conversions to write to this later 115 | Dim mtxTrainingImages As Matrix(Of Single) = New Matrix(Of Single)(intNumberOfTrainingSamples, RESIZED_IMAGE_WIDTH * RESIZED_IMAGE_HEIGHT) 116 | 117 | 'this keeps track of which row we are on in both classifications and training images, 118 | Dim intTrainingDataRowToAdd As Integer = 0 'note that each sample will correspond to one row in 119 | 'both the classifications XML file and the training images XML file 120 | 121 | While(Not contours Is Nothing) 'for each contour 122 | Dim contour As Contour(Of Point) = contours.ApproxPoly(contours.Perimeter * 0.0001) 'get the current contour, note that the lower the multiplier, the higher the precision 123 | If (contour.Area >= MIN_CONTOUR_AREA) Then 'if contour is big enough to consider 124 | Dim rect As Rectangle = contour.BoundingRectangle() 'get the bounding rect 125 | imgTrainingNumbers.Draw(rect, New Bgr(Color.Red), 2) 'draw red rectangle around each contour as we ask user for input 126 | Dim imgROI As Image(Of Gray, Byte) = imgThresh.Copy(rect) 'get ROI image of current char 127 | 128 | 'resize image, this is necessary for recognition and storage 129 | Dim imgROIResized As Image(Of Gray, Byte) = imgROI.Resize(RESIZED_IMAGE_WIDTH, RESIZED_IMAGE_HEIGHT, INTER.CV_INTER_LINEAR) 130 | 131 | CvInvoke.cvShowImage("imgROI", imgROI) 'show ROI image for reference 132 | CvInvoke.cvShowImage("imgROIResized", imgROIResized) 'show resized ROI image for reference 133 | CvInvoke.cvShowImage("imgTrainingNumbers", imgTrainingNumbers) 'show training numbers image, this will now have red rectangles drawn on it 134 | 135 | Dim intChar As Integer = CvInvoke.cvWaitKey(0) 'get key press 136 | 137 | If (intChar = 27) Then 'if esc key was pressed 138 | Return 'exit the function 139 | ElseIf (intValidChars.Contains(intChar)) Then 'else if the char is in the list of chars we are looking for . . . 140 | 141 | mtxClassifications(intTrainingDataRowToAdd, 0) = Convert.ToSingle(intChar) 'write classification char to classifications Matrix 142 | 143 | 'now add the training image (some conversion is necessary first) . . . 144 | 'note that we have to covert the images to Matrix(Of Single) type, this is necessary to pass into the KNearest object call to train 145 | Dim mtxTemp As Matrix(Of Single) = New Matrix(Of Single)(imgROIResized.Size()) 'declare a Matrix of the same dimensions as the Image we are adding to the data structure of training images 146 | Dim mtxTempReshaped As Matrix(Of Single) = New Matrix(Of Single)(1, RESIZED_IMAGE_WIDTH * RESIZED_IMAGE_HEIGHT) 'declare a flattened (only 1 row) matrix of the same total size 147 | 148 | CvInvoke.cvConvert(imgROIResized, mtxTemp) 'convert Image to a Matrix of Singles with the same dimensions 149 | 150 | For intRow As Integer = 0 To RESIZED_IMAGE_HEIGHT - 1 'flatten Matrix into one row by RESIZED_IMAGE_WIDTH * RESIZED_IMAGE_HEIGHT number of columns 151 | For intCol As Integer = 0 To RESIZED_IMAGE_WIDTH - 1 152 | mtxTempReshaped(0, (intRow * RESIZED_IMAGE_WIDTH) + intCol) = mtxTemp(intRow, intCol) 153 | Next 154 | Next 155 | 156 | For intCol As Integer = 0 To (RESIZED_IMAGE_WIDTH * RESIZED_IMAGE_HEIGHT) - 1 'write flattened Matrix into one row of training images Matrix 157 | mtxTrainingImages(intTrainingDataRowToAdd, intCol) = mtxTempReshaped(0, intCol) 158 | Next 159 | 160 | intTrainingDataRowToAdd = intTrainingDataRowToAdd + 1 'increment which row, i.e. sample we are on 161 | End If 162 | End If 163 | contours = contours.HNext 'move on to next contour 164 | End While 165 | 166 | txtInfo.Text = txtInfo.Text + "training complete !!" + vbCrLf + vbCrLf 167 | 168 | 'save classifications to file ''''''''''''''''''''''''''''''''''''''''''''''''''''' 169 | 170 | Dim xmlSerializer As XmlSerializer = New XmlSerializer(mtxClassifications.GetType) 171 | Dim streamWriter As StreamWriter 172 | 173 | Try 174 | streamWriter = new StreamWriter("classifications.xml") 'attempt to open classifications file 175 | Catch ex As Exception 'if error is encountered, show error and return 176 | txtInfo.Text = vbCrLf + txtInfo.Text + "unable to open 'classifications.xml', error:" + vbCrLf 177 | txtInfo.Text = txtInfo.Text + ex.Message + vbCrLf + vbCrLf 178 | Return 179 | End Try 180 | 181 | xmlSerializer.Serialize(streamWriter, mtxClassifications) 182 | streamWriter.Close() 183 | 184 | 'save training images to file ''''''''''''''''''''''''''''''''''''''''''''''''''''' 185 | 186 | xmlSerializer = New XmlSerializer(mtxTrainingImages.GetType) 187 | 188 | Try 189 | streamWriter = new StreamWriter("images.xml") 'attempt to open images file 190 | Catch ex As Exception 'if error is encountered, show error and return 191 | txtInfo.Text = vbCrLf + txtInfo.Text + "unable to open 'images.xml', error:" + vbCrLf 192 | txtInfo.Text = txtInfo.Text + ex.Message + vbCrLf + vbCrLf 193 | Return 194 | End Try 195 | 196 | xmlSerializer.Serialize(streamWriter, mtxTrainingImages) 197 | streamWriter.Close() 198 | 199 | txtInfo.Text = vbCrLf + txtInfo.Text + "file writing done" + vbCrLf 200 | 201 | End Sub 202 | 203 | End Class 204 | -------------------------------------------------------------------------------- /LicPlateOCRGenData/lic_plate_training_chars.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/LicPlateOCRGenData/lic_plate_training_chars.png -------------------------------------------------------------------------------- /LicensePlateRecognition/DetectChars.vb: -------------------------------------------------------------------------------- 1 | 'DetectChars.vb 2 | ' 3 | 'Emgu CV 2.4.10 4 | 5 | Option Explicit On 'require explicit declaration of variables, this is NOT Python !! 6 | Option Strict On 'restrict implicit data type conversions to only widening conversions 7 | 8 | Imports Emgu.CV ' 9 | Imports Emgu.CV.CvEnum 'Emgu Cv imports 10 | Imports Emgu.CV.Structure ' 11 | Imports Emgu.CV.UI ' 12 | Imports Emgu.CV.ML ' 13 | 14 | Imports System.Xml 15 | Imports System.Xml.Serialization 'these imports are for reading Matrix objects from file 16 | Imports System.IO 17 | 18 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 19 | Module DetectChars 20 | 21 | ' module level variables '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 22 | 'constants for checkIfPossibleChar, this checks one possible char only (does not compare to another char) 23 | Const MIN_PIXEL_WIDTH As Long = 2 24 | Const MIN_PIXEL_HEIGHT As Long = 8 25 | 26 | Const MIN_ASPECT_RATIO As Double = 0.25 27 | Const MAX_ASPECT_RATIO As Double = 1.0 28 | 29 | Const MIN_PIXEL_AREA As Long = 20 30 | 31 | 'constants for comparing two chars 32 | Const MIN_DIAG_SIZE_MULTIPLE_AWAY As Double = 0.3 33 | Const MAX_DIAG_SIZE_MULTIPLE_AWAY As Double = 5.0 34 | 35 | Const MAX_CHANGE_IN_AREA As Double = 0.5 36 | 37 | Const MAX_CHANGE_IN_WIDTH As Double = 0.8 38 | Const MAX_CHANGE_IN_HEIGHT As Double = 0.2 39 | 40 | Const MAX_ANGLE_BETWEEN_CHARS As Double = 12.0 41 | 42 | 'other constants 43 | Const MIN_NUMBER_OF_MATCHING_CHARS As Integer = 3 44 | 45 | Const RESIZED_CHAR_IMAGE_WIDTH As Integer = 20 46 | Const RESIZED_CHAR_IMAGE_HEIGHT As Integer = 30 47 | 48 | 'variables 49 | Dim kNearest As KNearest 50 | 51 | Const MIN_CONTOUR_AREA As Integer = 100 52 | 53 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 54 | Function detectCharsInPlates(listOfPossiblePlates As List(Of PossiblePlate)) As List(Of PossiblePlate) 55 | Dim intPlateCounter As Integer = 0 'this is only for showing steps 56 | Dim random As New Random() 'this is only for showing steps 57 | 58 | If (listOfPossiblePlates Is Nothing) Then 'if list of possible plates is null, 59 | Return listOfPossiblePlates 'return 60 | ElseIf (listOfPossiblePlates.Count = 0) Then 'if list of possible plates has zero plates 61 | Return listOfPossiblePlates 'return 62 | End If 63 | 'at this point we can be sure list of possible plates has at least one plate 64 | 65 | For Each possiblePlate As PossiblePlate In listOfPossiblePlates 66 | Preprocess.preprocess(possiblePlate.imgPlate, possiblePlate.imgGrayscale, possiblePlate.imgThresh) 67 | 68 | If (frmMain.cbShowSteps.Checked = True) Then 69 | CvInvoke.cvShowImage("5a", possiblePlate.imgPlate) 70 | CvInvoke.cvShowImage("5b", possiblePlate.imgGrayscale) 71 | CvInvoke.cvShowImage("5c", possiblePlate.imgThresh) 72 | End If 73 | 74 | possiblePlate.imgThresh = possiblePlate.imgThresh.Resize(1.6, INTER.CV_INTER_LINEAR) 'increase size of plate image for easier viewing and char detection 75 | 76 | 'threshold image to only black or white (eliminate grayscale) 77 | CvInvoke.cvThreshold(possiblePlate.imgThresh, possiblePlate.imgThresh, 0, 255, THRESH.CV_THRESH_BINARY Or THRESH.CV_THRESH_OTSU) 78 | 79 | If (frmMain.cbShowSteps.Checked = True) Then 80 | CvInvoke.cvShowImage("5d", possiblePlate.imgThresh) 81 | End If 82 | 83 | Dim listOfPossibleCharsInPlate As List(Of PossibleChar) = findPossibleCharsInPlate(possiblePlate.imgGrayscale, possiblePlate.imgThresh) 84 | 85 | If (frmMain.cbShowSteps.Checked = True) Then 86 | Dim imgContours As Image(Of Gray, Byte) = New Image(Of Gray, Byte)(possiblePlate.imgThresh.Size()) 87 | 88 | For Each possibleChar As PossibleChar In listOfPossibleCharsInPlate 89 | CvInvoke.cvDrawContours(imgContours, possibleChar.contour, New MCvScalar(255), New MCvScalar(255), 100, 1, LINE_TYPE.CV_AA, New Point(0, 0)) 90 | Next 91 | CvInvoke.cvShowImage("6", imgContours) 92 | End If 93 | 94 | Dim listOfListsOfMatchingCharsInPlate As List(Of List(Of PossibleChar)) = findListOfListsOfMatchingChars(listOfPossibleCharsInPlate) 95 | 96 | If (frmMain.cbShowSteps.Checked = True) Then 97 | Dim imgContours As Image(Of Bgr, Byte) = New Image(Of Bgr, Byte)(possiblePlate.imgThresh.Size()) 98 | 99 | For Each listOfMatchingChars As List(Of PossibleChar) In listOfListsOfMatchingCharsInPlate 100 | Dim intRandomBlue = random.Next(0, 256) 101 | Dim intRandomGreen = random.Next(0, 256) 102 | Dim intRandomRed = random.Next(0, 256) 103 | For Each matchingChar As PossibleChar In listOfMatchingChars 104 | CvInvoke.cvDrawContours(imgContours, matchingChar.contour, New MCvScalar(intRandomBlue, intRandomGreen, intRandomRed), New MCvScalar(intRandomBlue, intRandomGreen, intRandomRed), 100, 1, LINE_TYPE.CV_AA, New Point(0, 0)) 105 | Next 106 | Next 107 | CvInvoke.cvShowImage("7", imgContours) 108 | End If 109 | 110 | If (listOfListsOfMatchingCharsInPlate Is Nothing) Then 111 | If (frmMain.cbShowSteps.Checked = True) Then 112 | frmMain.txtInfo.AppendText("chars found in plate number " + intPlateCounter.ToString + " = (none), click on any image and press a key to continue . . ." + vbCrLf) 113 | intPlateCounter = intPlateCounter + 1 114 | CvInvoke.cvDestroyWindow("8") 115 | CvInvoke.cvDestroyWindow("9") 116 | CvInvoke.cvDestroyWindow("10") 117 | CvInvoke.cvWaitKey(0) 118 | End If 119 | 120 | possiblePlate.strChars = "" 121 | Continue For 122 | ElseIf (listOfListsOfMatchingCharsInPlate.Count = 0) Then 123 | If (frmMain.cbShowSteps.Checked = True) Then 124 | frmMain.txtInfo.AppendText("chars found in plate number " + intPlateCounter.ToString + " = (none), click on any image and press a key to continue . . ." + vbCrLf) 125 | intPlateCounter = intPlateCounter + 1 126 | CvInvoke.cvDestroyWindow("8") 127 | CvInvoke.cvDestroyWindow("9") 128 | CvInvoke.cvDestroyWindow("10") 129 | CvInvoke.cvWaitKey(0) 130 | End If 131 | 132 | possiblePlate.strChars = "" 133 | Continue For 134 | End If 135 | 136 | For i As Integer = 0 To listOfListsOfMatchingCharsInPlate.Count - 1 137 | listOfListsOfMatchingCharsInPlate(i).Sort(Function(oneChar, otherChar) oneChar.boundingRect.X.CompareTo(otherChar.boundingRect.X)) 138 | listOfListsOfMatchingCharsInPlate(i) = removeInnerOverlappingChars(listOfListsOfMatchingCharsInPlate(i)) 139 | Next 140 | 141 | If (frmMain.cbShowSteps.Checked = True) Then 142 | Dim imgContours As Image(Of Bgr, Byte) = New Image(Of Bgr, Byte)(possiblePlate.imgThresh.Size()) 143 | 144 | For Each listOfMatchingChars As List(Of PossibleChar) In listOfListsOfMatchingCharsInPlate 145 | Dim intRandomBlue = random.Next(0, 256) 146 | Dim intRandomGreen = random.Next(0, 256) 147 | Dim intRandomRed = random.Next(0, 256) 148 | For Each matchingChar As PossibleChar In listOfMatchingChars 149 | CvInvoke.cvDrawContours(imgContours, matchingChar.contour, New MCvScalar(intRandomBlue, intRandomGreen, intRandomRed), New MCvScalar(intRandomBlue, intRandomGreen, intRandomRed), 100, 1, LINE_TYPE.CV_AA, New Point(0, 0)) 150 | Next 151 | Next 152 | CvInvoke.cvShowImage("8", imgContours) 153 | End If 154 | 155 | 'within each possible plate, suppose the longest list of potential matching chars is the actual list of chars 156 | 157 | Dim intLenOfLongestListOfChars As Integer = 0 158 | Dim intIndexOfLongestListOfChars As Integer = 0 159 | 160 | For i As Integer = 0 To listOfListsOfMatchingCharsInPlate.Count - 1 'find index of longest list of matching chars, 161 | If (listOfListsOfMatchingCharsInPlate(i).Count > intLenOfLongestListOfChars) Then 'we will suppose this is the "best" or "correct" list of chars 162 | intLenOfLongestListOfChars = listOfListsOfMatchingCharsInPlate(i).Count 163 | intIndexOfLongestListOfChars = i 164 | End If 165 | Next 166 | 167 | Dim longestListOfMatchingCharsInPlate As List(Of PossibleChar) = listOfListsOfMatchingCharsInPlate(intIndexOfLongestListOfChars) 168 | 169 | If (frmMain.cbShowSteps.Checked = True) Then 170 | Dim imgContours As Image(Of Gray, Byte) = New Image(Of Gray, Byte)(possiblePlate.imgThresh.Size()) 171 | 172 | For Each matchingChar As PossibleChar In longestListOfMatchingCharsInPlate 173 | CvInvoke.cvDrawContours(imgContours, matchingChar.contour, New MCvScalar(255), New MCvScalar(255), 100, 1, LINE_TYPE.CV_AA, New Point(0, 0)) 174 | Next 175 | CvInvoke.cvShowImage("9", imgContours) 176 | End If 177 | 178 | possiblePlate.strChars = recognizeCharsInPlate(possiblePlate.imgThresh, longestListOfMatchingCharsInPlate) 179 | 180 | If (frmMain.cbShowSteps.Checked = True) Then 181 | frmMain.txtInfo.AppendText("chars found in plate number " + intPlateCounter.ToString + " = " + possiblePlate.strChars + ", click on any image and press a key to continue . . ." + vbCrLf) 182 | intPlateCounter = intPlateCounter + 1 183 | CvInvoke.cvWaitKey(0) 184 | End If 185 | Next 186 | 187 | If (frmMain.cbShowSteps.Checked = True) Then 188 | frmMain.txtInfo.AppendText(vbCrLf + "char detection complete, click on any image and press a key to continue . . ." + vbCrLf) 189 | CvInvoke.cvWaitKey(0) 190 | End If 191 | 192 | Return listOfPossiblePlates 193 | End Function 194 | 195 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 196 | Function findPossibleCharsInPlate(imgGrayscale As Image(Of Gray, Byte), imgThresh As Image(Of Gray, Byte)) As List(Of PossibleChar) 197 | Dim listOfPossibleChars As List(Of PossibleChar) = New List(Of PossibleChar) 'this will be the return value 198 | Dim imgThreshCopy As Image(Of Gray, Byte) 199 | Dim contours As Contour(Of Point) 200 | Dim listOfContours As List(Of Contour(Of Point)) = New List(Of Contour(Of Point))() 201 | 202 | imgThreshCopy = imgThresh.Clone() 'make a copy of the thresh image, this in necessary b/c findContours modifies the image 203 | 204 | contours = imgThreshCopy.FindContours(CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE, RETR_TYPE.CV_RETR_LIST) 205 | 206 | While (Not contours Is Nothing) 'for each contour 207 | Dim contour As Contour(Of Point) = contours.ApproxPoly(contours.Perimeter * 0.0001) 'get the current contour, note that the lower the multiplier, the higher the precision 208 | If (contour.Area >= MIN_CONTOUR_AREA) Then 209 | listOfContours.Add(contour) 210 | End If 211 | contours = contours.HNext 'move on to next contour 212 | End While 213 | 'sort contours from left to right 214 | listOfContours.Sort(Function(oneContour, otherContour) oneContour.BoundingRectangle.X.CompareTo(otherContour.BoundingRectangle.X)) 215 | 216 | For Each contour As Contour(Of Point) In listOfContours 217 | Dim possibleChar As PossibleChar = New PossibleChar(contour) 218 | If (checkIfPossibleChar(possibleChar)) Then 219 | listOfPossibleChars.Add(possibleChar) 220 | End If 221 | Next 222 | 223 | Return listOfPossibleChars 224 | End Function 225 | 226 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 227 | Function checkIfPossibleChar(possibleChar As PossibleChar) As Boolean 228 | If (possibleChar.boundingRect.Width > MIN_PIXEL_WIDTH And possibleChar.boundingRect.Height > MIN_PIXEL_HEIGHT And _ 229 | MIN_ASPECT_RATIO < possibleChar.dblAspectRatio And possibleChar.dblAspectRatio < MAX_ASPECT_RATIO And _ 230 | possibleChar.lngArea > MIN_PIXEL_AREA) Then 231 | Return True 232 | Else 233 | Return False 234 | End If 235 | End Function 236 | 237 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 238 | Function findListOfListsOfMatchingChars(listOfPossibleChars As List(Of PossibleChar)) As List(Of List(Of PossibleChar)) 239 | 240 | Dim listOfListsOfMatchingChars As List(Of List(Of PossibleChar)) = New List(Of List(Of PossibleChar)) 'this will be the return value 241 | 242 | For Each possibleChar As PossibleChar In listOfPossibleChars 243 | 'get list of chars that match the current char 244 | Dim listOfMatchingChars As List(Of PossibleChar) = findListOfMatchingChars(possibleChar, listOfPossibleChars) 245 | 246 | listOfMatchingChars.Add(possibleChar) 'also add the current char to the list of potential matching chars 247 | 248 | If (listOfMatchingChars.Count < MIN_NUMBER_OF_MATCHING_CHARS) Then 'check if the list of chars is long enough to constitute a "group" or "cluster" of matching chars 249 | Continue For 'if not, continue for, this will go on to the next possible char 250 | End If 251 | 'if we get here, the current list passed test as a "group" or "cluster" of matching chars 252 | listOfListsOfMatchingChars.Add(listOfMatchingChars) 'so add to our list of lists of matching chars 253 | 254 | Dim listOfPossibleCharsWithCurrentMatchesRemoved As List(Of PossibleChar) = listOfPossibleChars.Except(listOfMatchingChars).ToList() 255 | 256 | Dim recursiveListOfListsOfMatchingChars As List(Of List(Of PossibleChar)) = New List(Of List(Of PossibleChar)) 257 | 258 | recursiveListOfListsOfMatchingChars = findListOfListsOfMatchingChars(listOfPossibleCharsWithCurrentMatchesRemoved) 'recursive call !! 259 | 260 | For Each recursiveListOfMatchingChars As List(Of PossibleChar) In recursiveListOfListsOfMatchingChars 261 | listOfListsOfMatchingChars.Add(recursiveListOfMatchingChars) 262 | Next 263 | Exit For 264 | Next 265 | 266 | Return listOfListsOfMatchingChars 267 | End Function 268 | 269 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 270 | Function findListOfMatchingChars(possibleChar As PossibleChar, listOfChars As List(Of PossibleChar)) As List(Of PossibleChar) 271 | Dim listOfMatchingChars As List(Of PossibleChar) = New List(Of PossibleChar) 'this will be the return value 272 | 273 | For Each possibleMatchingChar As PossibleChar In listOfChars 274 | 275 | If (possibleMatchingChar.Equals(possibleChar)) Then 276 | Continue For 277 | End If 278 | 279 | Dim dblDistanceBetweenChars As Double = distanceBetweenChars(possibleChar, possibleMatchingChar) 280 | 281 | Dim dblAngleBetweenChars As Double = angleBetweenChars(possibleChar, possibleMatchingChar) 282 | 283 | Dim dblChangeInArea As Double = Math.Abs(possibleMatchingChar.lngArea - possibleChar.lngArea) / possibleChar.lngArea 284 | 285 | Dim dblChangeInWidth As Double = Math.Abs(possibleMatchingChar.boundingRect.Width - possibleChar.boundingRect.Width) / possibleChar.boundingRect.Width 286 | Dim dblChangeInHeight As Double = Math.Abs(possibleMatchingChar.boundingRect.Height - possibleChar.boundingRect.Height) / possibleChar.boundingRect.Height 287 | 288 | If (dblDistanceBetweenChars < (possibleChar.dblDiagonalSize * MAX_DIAG_SIZE_MULTIPLE_AWAY) And _ 289 | dblAngleBetweenChars < MAX_ANGLE_BETWEEN_CHARS And _ 290 | dblChangeInArea < MAX_CHANGE_IN_AREA And _ 291 | dblChangeInWidth < MAX_CHANGE_IN_WIDTH And _ 292 | dblChangeInHeight < MAX_CHANGE_IN_HEIGHT) Then 293 | 294 | listOfMatchingChars.Add(possibleMatchingChar) 295 | End If 296 | 297 | Next 298 | 299 | Return listOfMatchingChars 300 | End Function 301 | 302 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 303 | Function distanceBetweenChars(firstChar As PossibleChar, secondChar As PossibleChar) As Double 304 | Dim lngX As Long = Math.Abs(firstChar.lngCenterX - secondChar.lngCenterX) 305 | Dim lngY As Long = Math.Abs(firstChar.lngCenterY - secondChar.lngCenterY) 306 | 307 | Return Math.Sqrt((lngX ^ 2) + (lngY ^ 2)) 308 | End Function 309 | 310 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 311 | Function angleBetweenChars(firstChar As PossibleChar, secondChar As PossibleChar) As Double 312 | Dim dblAdj As Double = CDbl(Math.Abs(firstChar.lngCenterX - secondChar.lngCenterX)) 313 | Dim dblOpp As Double = CDbl(Math.Abs(firstChar.lngCenterY - secondChar.lngCenterY)) 314 | 315 | Dim dblAngleInRad As Double = Math.Atan(dblOpp / dblAdj) 316 | 317 | Dim dblAngleInDeg As Double = dblAngleInRad * (180.0 / Math.PI) 318 | 319 | Return dblAngleInDeg 320 | End Function 321 | 322 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 323 | Function removeInnerOverlappingChars(listOfMatchingChars As List(Of PossibleChar)) As List(Of PossibleChar) 324 | 325 | Dim listOfMatchingCharsWithInnerCharRemoved As List(Of PossibleChar) = New List(Of PossibleChar)(listOfMatchingChars) 326 | 327 | For Each currentChar As PossibleChar In listOfMatchingChars 328 | For Each otherChar As PossibleChar In listOfMatchingChars 329 | If (Not currentChar.Equals(otherChar)) Then 'if current char and other char are not the same char . . . 330 | 'if current char and other char have center points at almost the same location . . . 331 | If (distanceBetweenChars(currentChar, otherChar) < currentChar.dblDiagonalSize * MIN_DIAG_SIZE_MULTIPLE_AWAY) Then 332 | 'if we get in here we have found overlapping chars 333 | 'next we identify which char is smaller, then if that char was not already removed on a previous pass, remove it 334 | If (currentChar.lngArea < otherChar.lngArea) Then 'if current char is smaller than other char 335 | If (listOfMatchingCharsWithInnerCharRemoved.Contains(currentChar)) Then 'if current char was not already removed on a previous pass . . . 336 | listOfMatchingCharsWithInnerCharRemoved.Remove(currentChar) 'then remove current char 337 | End If 338 | Else 'else if other char is smaller than current char 339 | If (listOfMatchingCharsWithInnerCharRemoved.Contains(otherChar)) Then 'if other char was not already removed on a previous pass . . . 340 | listOfMatchingCharsWithInnerCharRemoved.Remove(otherChar) 'then remove other char 341 | End If 342 | 343 | End If 344 | End If 345 | End If 346 | Next 347 | Next 348 | 349 | Return listOfMatchingCharsWithInnerCharRemoved 350 | End Function 351 | 352 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 353 | Function recognizeCharsInPlate(imgThresh As Image(Of Gray, Byte), listOfMatchingChars As List(Of PossibleChar)) As String 354 | Dim strChars As String = "" 'this will be the return value, the chars in the lic plate 355 | 356 | Dim imgThreshColor As Image(Of Bgr, Byte) 357 | 358 | listOfMatchingChars.Sort(Function(oneChar, otherChar) oneChar.boundingRect.X.CompareTo(otherChar.boundingRect.X)) 359 | 360 | imgThreshColor = imgThresh.Convert(Of Bgr, Byte)() 361 | 362 | For Each currentChar As PossibleChar In listOfMatchingChars 363 | imgThreshColor.Draw(currentChar.boundingRect, New Bgr(Color.Green), 2) 364 | 365 | Dim imgROI As Image(Of Gray, Byte) = imgThresh.Copy(currentChar.boundingRect) 366 | 367 | Dim imgROIResized As Image(Of Gray, Byte) = imgROI.Resize(RESIZED_CHAR_IMAGE_WIDTH, RESIZED_CHAR_IMAGE_HEIGHT, INTER.CV_INTER_LINEAR) 368 | 369 | Dim mtxTemp As Matrix(Of Single) = New Matrix(Of Single)(imgROIResized.Size()) 370 | Dim mtxTempReshaped As Matrix(Of Single) = New Matrix(Of Single)(1, RESIZED_CHAR_IMAGE_WIDTH * RESIZED_CHAR_IMAGE_HEIGHT) 371 | 372 | CvInvoke.cvConvert(imgROIResized, mtxTemp) 373 | 374 | For intRow As Integer = 0 To RESIZED_CHAR_IMAGE_HEIGHT - 1 'flatten Matrix into one row by RESIZED_IMAGE_WIDTH * RESIZED_IMAGE_HEIGHT number of columns 375 | For intCol As Integer = 0 To RESIZED_CHAR_IMAGE_WIDTH - 1 376 | mtxTempReshaped(0, (intRow * RESIZED_CHAR_IMAGE_WIDTH) + intCol) = mtxTemp(intRow, intCol) 377 | Next 378 | Next 379 | 380 | Dim sngCurrentChar As Single = kNearest.FindNearest(mtxTempReshaped, 1, Nothing, Nothing, Nothing, Nothing) 381 | 382 | strChars = strChars + Chr(Convert.ToInt32(sngCurrentChar)) 383 | Next 384 | 385 | If (frmMain.cbShowSteps.Checked = True) Then 386 | CvInvoke.cvShowImage("10", imgThreshColor) 387 | End If 388 | 389 | Return strChars 390 | End Function 391 | 392 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 393 | Function loadKNNDataAndTrainKNN() As Boolean 394 | 395 | 'note: we effectively have to read the first XML file twice 396 | 'first, we read the file to get the number of rows (which is the same as the number of samples) 397 | 'the first time reading the file we can't get the data yet, since we don't know how many rows of data there are 398 | 'next, reinstantiate our classifications Matrix and training images Matrix with the correct number of rows 399 | 'then, read the file again and this time read the data into our resized classifications Matrix and training images Matrix 400 | 401 | Dim mtxClassifications As Matrix(Of Single) = New Matrix(Of Single)(1, 1) 'for the first time through, declare these to be 1 row by 1 column 402 | Dim mtxTrainingImages As Matrix(Of Single) = New Matrix(Of Single)(1, 1) 'we will resize these when we know the number of rows (i.e. number of training samples) 403 | 404 | Dim xmlSerializer As XmlSerializer = New XmlSerializer(mtxClassifications.GetType) 'these variables are for 405 | Dim streamReader As StreamReader 'reading from the XML files 406 | 407 | Try 408 | streamReader = new StreamReader("classifications.xml") 'attempt to open classifications file 409 | Catch ex As Exception 'if error is encountered, show error and return 410 | frmMain.txtInfo.Text = vbCrLf + frmMain.txtInfo.Text + "unable to open 'classifications.xml', error:" + vbCrLf 411 | frmMain.txtInfo.Text = frmMain.txtInfo.Text + ex.Message + vbCrLf + vbCrLf 412 | Return False 413 | End Try 414 | 415 | 'read from the classifications file the 1st time, this is only to get the number of rows, not the actual data 416 | mtxClassifications = CType(xmlSerializer.Deserialize(streamReader), Global.Emgu.CV.Matrix(Of Single)) 417 | 418 | streamReader.Close() 'close the classifications XML file 419 | 420 | Dim intNumberOfTrainingSamples As Integer = mtxClassifications.Rows 'get the number of rows, i.e. the number of training samples 421 | 422 | 'now that we know the number of rows, reinstantiate classifications Matrix and training images Matrix with the actual number of rows 423 | mtxClassifications = New Matrix(Of Single)(intNumberOfTrainingSamples, 1) 424 | mtxTrainingImages = New Matrix(Of Single)(intNumberOfTrainingSamples, RESIZED_CHAR_IMAGE_WIDTH * RESIZED_CHAR_IMAGE_HEIGHT) 425 | 426 | Try 427 | streamReader = new StreamReader("classifications.xml") 'reinitialize the stream reader 428 | Catch ex As Exception 'if error is encountered, show error and return 429 | frmMain.txtInfo.Text = vbCrLf + frmMain.txtInfo.Text + "unable to open 'classifications.xml', error:" + vbCrLf 430 | frmMain.txtInfo.Text = frmMain.txtInfo.Text + ex.Message + vbCrLf + vbCrLf 431 | Return False 432 | End Try 433 | 434 | 'read from the classifications file again, this time we can get the actual data 435 | mtxClassifications = CType(xmlSerializer.Deserialize(streamReader), Global.Emgu.CV.Matrix(Of Single)) 436 | 437 | streamReader.Close() 'close the classifications XML file 438 | 439 | xmlSerializer = New XmlSerializer(mtxTrainingImages.GetType) 'reinstantiate file reading variables 440 | 441 | Try 442 | streamReader = New StreamReader("images.xml") 443 | Catch ex As Exception 'if error is encountered, show error and return 444 | frmMain.txtInfo.Text = vbCrLf + frmMain.txtInfo.Text + "unable to open 'images.xml', error:" + vbCrLf 445 | frmMain.txtInfo.Text = frmMain.txtInfo.Text + ex.Message + vbCrLf + vbCrLf 446 | Return False 447 | End Try 448 | 449 | mtxTrainingImages = CType(xmlSerializer.Deserialize(streamReader), Global.Emgu.CV.Matrix(Of Single)) 'read from training images file 450 | streamReader.Close() 'close the training images XML file 451 | 452 | ' train ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 453 | 454 | kNearest = New KNearest() 'instantiate KNN object 455 | kNearest.Train(mtxTrainingImages, mtxClassifications, Nothing, False, 3, False) 'call to train 456 | 457 | Return True 458 | End Function 459 | 460 | End Module 461 | 462 | 463 | 464 | 465 | -------------------------------------------------------------------------------- /LicensePlateRecognition/DetectPlates.vb: -------------------------------------------------------------------------------- 1 | 'DetectPlates.vb 2 | ' 3 | 'Emgu CV 2.4.10 4 | 5 | Option Explicit On 'require explicit declaration of variables, this is NOT Python !! 6 | Option Strict On 'restrict implicit data type conversions to only widening conversions 7 | 8 | Imports Emgu.CV ' 9 | Imports Emgu.CV.CvEnum 'Emgu Cv imports 10 | Imports Emgu.CV.Structure ' 11 | Imports Emgu.CV.UI ' 12 | 13 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 14 | Module DetectPlates 15 | 16 | ' module level variables '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 17 | Const PLATE_WIDTH_PADDING_FACTOR As Double = 1.5 18 | Const PLATE_HEIGHT_PADDING_FACTOR As Double = 1.5 19 | 20 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 21 | Function detectPlatesInScene(imgOriginalScene As Image(Of Bgr, Byte)) As List(Of PossiblePlate) 22 | Dim imgGrayscaleScene As Image(Of Gray, Byte) = Nothing 23 | Dim imgThreshScene As Image(Of Gray, Byte) = Nothing 24 | Dim imgContours As Image(Of Bgr, Byte) = Nothing 'this is only used for showing steps 25 | Dim random As New Random() 'this is only used for showing steps 26 | 27 | closePreviousShowStepsWindows() 28 | 29 | If (frmMain.cbShowSteps.Checked = True) Then 30 | CvInvoke.cvShowImage("0", imgOriginalScene) 31 | End If 32 | 33 | Preprocess.preprocess(imgOriginalScene, imgGrayscaleScene, imgThreshScene) 34 | 35 | If (frmMain.cbShowSteps.Checked = True) Then 36 | CvInvoke.cvShowImage("1a", imgGrayscaleScene) 37 | CvInvoke.cvShowImage("1b", imgThreshScene) 38 | End If 39 | 40 | Dim listOfPossibleCharsInScene As List(Of PossibleChar) = findPossibleCharsInScene(imgThreshScene) 41 | 42 | If (frmMain.cbShowSteps.Checked = True) Then 43 | frmMain.txtInfo.AppendText("step 2 - listOfPossibleCharsInScene.Count = " + listOfPossibleCharsInScene.Count.ToString + vbCrLf) '175 with MCLRNF1 image 44 | 45 | imgContours = New Image(Of Bgr, Byte)(imgOriginalScene.Size()) 46 | 47 | For Each possibleChar As PossibleChar In listOfPossibleCharsInScene 48 | CvInvoke.cvDrawContours(imgContours, possibleChar.contour, New MCvScalar(255, 255, 255), New MCvScalar(255, 255, 255), 100, 1, LINE_TYPE.CV_AA, New Point(0, 0)) 49 | Next 50 | 51 | CvInvoke.cvShowImage("2b", imgContours) 52 | End If 53 | 54 | Dim listOfListsOfMatchingCharsInScene As List(Of List(Of PossibleChar)) = findListOfListsOfMatchingChars(listOfPossibleCharsInScene) 55 | 56 | If (frmMain.cbShowSteps.Checked = True) Then 57 | frmMain.txtInfo.AppendText("step 3 - listOfListsOfMatchingCharsInScene.Count = " + listOfListsOfMatchingCharsInScene.Count.ToString + vbCrLf) '13 with MCLRNF1 image 58 | 59 | imgContours = New Image(Of Bgr, Byte)(imgOriginalScene.Size()) 're-instantiate imgContours to clear it 60 | 61 | For Each listOfMatchingChars As List(Of PossibleChar) In listOfListsOfMatchingCharsInScene 62 | Dim intRandomBlue = random.Next(0, 256) 63 | Dim intRandomGreen = random.Next(0, 256) 64 | Dim intRandomRed = random.Next(0, 256) 65 | For Each matchingChar As PossibleChar In listOfMatchingChars 66 | CvInvoke.cvDrawContours(imgContours, matchingChar.contour, New MCvScalar(intRandomBlue, intRandomGreen, intRandomRed), New MCvScalar(intRandomBlue, intRandomGreen, intRandomRed), 100, 1, LINE_TYPE.CV_AA, New Point(0, 0)) 67 | Next 68 | Next 69 | CvInvoke.cvShowImage("3", imgContours) 70 | End If 71 | 72 | Dim listOfPossiblePlates As List(Of PossiblePlate) = New List(Of PossiblePlate) 'this will be the return value 73 | 74 | For Each listOfMatchingChars As List(Of PossibleChar) In listOfListsOfMatchingCharsInScene 75 | Dim possiblePlate = extractPlate(imgOriginalScene, listOfMatchingChars) 76 | 77 | If (Not possiblePlate.imgPlate Is Nothing) Then 78 | listOfPossiblePlates.Add(possiblePlate) 79 | End If 80 | Next 81 | 82 | frmMain.txtInfo.AppendText(vbCrLf + listOfPossiblePlates.Count.ToString + " possible plates found" + vbCrLf) 83 | 84 | If (frmMain.cbShowSteps.Checked = True) Then 85 | frmMain.txtInfo.AppendText(vbCrLf) 86 | CvInvoke.cvShowImage("4a", imgContours) 87 | 88 | For i As Integer = 0 To listOfPossiblePlates.Count - 1 89 | imgContours.Draw(listOfPossiblePlates(i).b2dLocationOfPlateInScene, New Bgr(Color.Red), 2) 'draw red rectangle around plate 90 | CvInvoke.cvShowImage("4a", imgContours) 91 | frmMain.txtInfo.AppendText("possible plate " + i.ToString + ", click on any image and press a key to continue . . ." + vbCrLf) 92 | CvInvoke.cvShowImage("4b", listOfPossiblePlates(i).imgPlate) 93 | CvInvoke.cvWaitKey(0) 94 | Next 95 | frmMain.txtInfo.AppendText(vbCrLf + "plate detection complete, click on any image and press a key to begin char recognition . . ." + vbCrLf + vbCrLf) 96 | CvInvoke.cvWaitKey(0) 97 | End If 98 | 99 | Return listOfPossiblePlates 100 | End Function 101 | 102 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 103 | Function findPossibleCharsInScene(imgThresh As Image(Of Gray, Byte)) As List(Of PossibleChar) 104 | Dim imgContours As New Image(Of Gray, Byte)(imgThresh.Size()) 'this is only for showing steps 105 | Dim intNumOfContours As Integer = 0 'this is only for showing steps 106 | Dim intCountOfPossibleChars As Integer = 0 'this is only for showing steps 107 | 108 | Dim contours As Contour(Of Point) = imgThresh.FindContours(CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE, RETR_TYPE.CV_RETR_LIST) 109 | 110 | Dim listOfPossibleChars As List(Of PossibleChar) = New List(Of PossibleChar)() 'this is the return value 111 | 112 | While (Not contours Is Nothing) 113 | intNumOfContours = intNumOfContours + 1 114 | Dim contour As Contour(Of Point) = contours.ApproxPoly(contours.Perimeter * 0.0001) 115 | 116 | If (frmMain.cbShowSteps.Checked = True) 117 | CvInvoke.cvDrawContours(imgContours, contour, New MCvScalar(255), New MCvScalar(255), 100, 1, LINE_TYPE.CV_AA, New Point(0, 0)) 118 | End If 119 | 120 | Dim possibleChar As PossibleChar = New PossibleChar(contour) 121 | 122 | If (checkIfPossibleChar(possibleChar)) Then 123 | intCountOfPossibleChars = intCountOfPossibleChars + 1 124 | listOfPossibleChars.Add(possibleChar) 125 | End If 126 | 127 | contours = contours.HNext 128 | End While 129 | 130 | If (frmMain.cbShowSteps.Checked) Then 131 | frmMain.txtInfo.AppendText(vbCrLf + "step 2 - intNumOfContours = " + intNumOfContours.ToString + vbCrLf) '2362 with MCLRNF1 image 132 | frmMain.txtInfo.AppendText("step 2 - intCountOfPossibleChars = " + intCountOfPossibleChars.ToString + vbCrLf) '175 with MCLRNF1 image 133 | CvInvoke.cvShowImage("2a", imgContours) 134 | End If 135 | 136 | Return listOfPossibleChars 137 | End Function 138 | 139 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 140 | Function extractPlate(imgOriginal As Image(Of Bgr, Byte), listOfMatchingChars As List(Of PossibleChar)) As PossiblePlate 141 | Dim possiblePlate As PossiblePlate = New PossiblePlate 'this will be the return value 142 | 143 | 'sort chars from left to right based on x position 144 | listOfMatchingChars.Sort(Function(firstChar, secondChar) firstChar.lngCenterX.CompareTo(secondChar.lngCenterX)) 145 | 146 | 'calculate the center point of the plate 147 | Dim sngPlateCenterX As Single = CSng(CSng(listOfMatchingChars(0).lngCenterX + listOfMatchingChars(listOfMatchingChars.Count - 1).lngCenterX) / 2.0) 148 | Dim sngPlateCenterY As Single = CSng(CSng(listOfMatchingChars(0).lngCenterY + listOfMatchingChars(listOfMatchingChars.Count - 1).lngCenterY) / 2.0) 149 | Dim ptfPlateCenter As PointF = New PointF(sngPlateCenterX, sngPlateCenterY) 150 | 151 | 'calculate plate width and height 152 | Dim intPlateWidth As Integer = CInt((listOfMatchingChars(listOfMatchingChars.Count-1).boundingRect.Right - listOfMatchingChars(0).boundingRect.Left) * PLATE_WIDTH_PADDING_FACTOR) 153 | Dim intPlateHeight As Integer = CInt(listOfMatchingChars(0).dblDiagonalSize * PLATE_HEIGHT_PADDING_FACTOR) 154 | 155 | 'calculate correction angle of plate region 156 | Dim sngOpposite = CSng(listOfMatchingChars(listOfMatchingChars.Count - 1).lngCenterY - listOfMatchingChars(0).lngCenterY) 157 | Dim sngHypotenuse = CSng(distanceBetweenChars(listOfMatchingChars(0), listOfMatchingChars(listOfMatchingChars.Count - 1))) 158 | Dim sngCorrectionAngleInRad = CSng(Math.Asin(sngOpposite / sngHypotenuse)) 159 | Dim sngCorrectionAngleInDeg As Single = sngCorrectionAngleInRad * CSng(180.0 / Math.PI) 160 | 161 | 'calculate bounding rect of plate region, 162 | 'use MCvBox2D so rect can be tilted by correction angle (type Rectangle cannot be tilted) 163 | possiblePlate.b2dLocationOfPlateInScene = New MCvBox2D(ptfPlateCenter, New SizeF(CSng(intPlateWidth), CSng(intPlateHeight)), sngCorrectionAngleInDeg) 164 | 165 | possiblePlate.imgPlate = imgOriginal.Copy(possiblePlate.b2dLocationOfPlateInScene) 'assign crop of plate region to return variable 166 | 167 | Return possiblePlate 168 | End Function 169 | 170 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 171 | Sub closePreviousShowStepsWindows() 172 | CvInvoke.cvDestroyWindow("0") 173 | CvInvoke.cvDestroyWindow("1a") 174 | CvInvoke.cvDestroyWindow("1b") 175 | CvInvoke.cvDestroyWindow("2a") 176 | CvInvoke.cvDestroyWindow("2b") 177 | CvInvoke.cvDestroyWindow("3") 178 | CvInvoke.cvDestroyWindow("4a") 179 | CvInvoke.cvDestroyWindow("4b") 180 | CvInvoke.cvDestroyWindow("5a") 181 | CvInvoke.cvDestroyWindow("5b") 182 | CvInvoke.cvDestroyWindow("5c") 183 | CvInvoke.cvDestroyWindow("5d") 184 | CvInvoke.cvDestroyWindow("6") 185 | CvInvoke.cvDestroyWindow("7") 186 | CvInvoke.cvDestroyWindow("8") 187 | CvInvoke.cvDestroyWindow("9") 188 | CvInvoke.cvDestroyWindow("10") 189 | End Sub 190 | 191 | End Module 192 | 193 | 194 | 195 | -------------------------------------------------------------------------------- /LicensePlateRecognition/PossibleChar.vb: -------------------------------------------------------------------------------- 1 | 'PossibleChar.vb 2 | ' 3 | 'Emgu CV 2.4.10 4 | 5 | Option Explicit On 'require explicit declaration of variables, this is NOT Python !! 6 | Option Strict On 'restrict implicit data type conversions to only widening conversions 7 | 8 | Imports System.Math 9 | 10 | Imports Emgu.CV ' 11 | Imports Emgu.CV.CvEnum 'Emgu Cv imports 12 | Imports Emgu.CV.Structure ' 13 | Imports Emgu.CV.UI ' 14 | 15 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 16 | Public Class PossibleChar 17 | 18 | ' member variables '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 19 | Public contour As Contour(Of Point) 20 | 21 | Public boundingRect As Rectangle 22 | 23 | Public lngCenterX As Long 24 | Public lngCenterY As Long 25 | 26 | Public dblDiagonalSize As Double 27 | Public dblAspectRatio As Double 28 | Public lngArea As Long 29 | 30 | ' constructor ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 31 | Sub New(_contour As Contour(Of Point)) 32 | contour = _contour 33 | 34 | boundingRect = contour.BoundingRectangle() 35 | 36 | lngCenterX = CLng((boundingRect.Left + boundingRect.Right) / 2) 37 | lngCenterY = CLng((boundingRect.Top + boundingRect.Bottom) / 2) 38 | 39 | dblDiagonalSize = Math.Sqrt((boundingRect.Width ^ 2) + (boundingRect.Height ^ 2)) 40 | 41 | dblAspectRatio = CDbl(boundingRect.Width) / CDbl(boundingRect.Height) 42 | 43 | lngArea = boundingRect.Width * boundingRect.Height 44 | End Sub 45 | 46 | End Class 47 | 48 | 49 | -------------------------------------------------------------------------------- /LicensePlateRecognition/PossiblePlate.vb: -------------------------------------------------------------------------------- 1 | 'PossiblePlate.vb 2 | ' 3 | 'Emgu CV 2.4.10 4 | 5 | Option Explicit On 'require explicit declaration of variables, this is NOT Python !! 6 | Option Strict On 'restrict implicit data type conversions to only widening conversions 7 | 8 | Imports Emgu.CV ' 9 | Imports Emgu.CV.CvEnum 'Emgu Cv imports 10 | Imports Emgu.CV.Structure ' 11 | Imports Emgu.CV.UI ' 12 | 13 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 14 | Public Class PossiblePlate 15 | 16 | ' member variables '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 17 | Public imgPlate As Image(Of Bgr, Byte) 18 | Public imgGrayscale As Image(Of Gray, Byte) 19 | Public imgThresh As Image(Of Gray, Byte) 20 | 21 | Public b2dLocationOfPlateInScene As MCvBox2D 22 | 23 | Public strChars As String 24 | 25 | ' constructor ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 26 | Sub New 27 | 'initialize values 28 | imgPlate = Nothing 29 | imgGrayscale = Nothing 30 | imgThresh = Nothing 31 | 32 | strChars = "" 33 | End Sub 34 | 35 | End Class 36 | 37 | 38 | 39 | 40 | 41 | -------------------------------------------------------------------------------- /LicensePlateRecognition/Preprocess.vb: -------------------------------------------------------------------------------- 1 | 'Preprocess.vb 2 | ' 3 | 'Emgu CV 2.4.10 4 | 5 | Option Explicit On 'require explicit declaration of variables, this is NOT Python !! 6 | Option Strict On 'restrict implicit data type conversions to only widening conversions 7 | 8 | Imports Emgu.CV ' 9 | Imports Emgu.CV.CvEnum 'Emgu Cv imports 10 | Imports Emgu.CV.Structure ' 11 | Imports Emgu.CV.UI ' 12 | 13 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 14 | Module Preprocess 15 | 16 | ' module level variables '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 17 | Const GAUSSIAN_SMOOTH_FILTER_SIZE As Integer = 5 18 | Const ADAPTIVE_THRESH_BLOCK_SIZE As Integer = 19 19 | Const ADAPTIVE_THRESH_WEIGHT As Integer = 9 20 | 21 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 22 | Sub preprocess(imgOriginal As Image(Of Bgr, Byte), ByRef imgGrayscale As Image(Of Gray, Byte), ByRef imgThresh As Image(Of Gray, Byte)) 23 | 24 | imgGrayscale = extractValue(imgOriginal) 25 | 26 | Dim imgMaxContrastGrayscale As Image(Of Gray, Byte) = maximizeContrast(imgGrayscale) 27 | 28 | Dim imgBlurred As Image(Of Gray, Byte) = imgMaxContrastGrayscale.SmoothGaussian(GAUSSIAN_SMOOTH_FILTER_SIZE) 29 | 30 | imgThresh = imgBlurred.ThresholdAdaptive(New Gray(255), ADAPTIVE_THRESHOLD_TYPE.CV_ADAPTIVE_THRESH_GAUSSIAN_C, THRESH.CV_THRESH_BINARY_INV, ADAPTIVE_THRESH_BLOCK_SIZE, New Gray(ADAPTIVE_THRESH_WEIGHT)) 31 | End Sub 32 | 33 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 34 | Function extractValue(imgOriginal As Image(Of Bgr, Byte)) As Image(Of Gray, Byte) 35 | Dim imgHSV As Image(Of Hsv, Byte) = imgOriginal.Convert(Of Hsv, Byte)() 36 | Dim imgChannels As Image(Of Gray, Byte)() = imgHSV.Split() 37 | Dim imgValue As Image(Of Gray, Byte) = imgChannels(2) 38 | 39 | Return imgValue 40 | End Function 41 | 42 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 43 | Function maximizeContrast(imgGrayscale As Image(Of Gray, Byte)) As Image(Of Gray, Byte) 44 | Dim imgTopHat As Image(Of Gray, Byte) 45 | Dim imgBlackHat As Image(Of Gray, Byte) 46 | Dim imgGrayscalePlusTopHat As Image(Of Gray, Byte) 47 | Dim imgGrayscalePlusTopHatMinusBlackHat As Image(Of Gray, Byte) 48 | 49 | Dim structuringElementEx As StructuringElementEx = New StructuringElementEx(3, 3, 1, 1, CV_ELEMENT_SHAPE.CV_SHAPE_RECT) 50 | 51 | imgTopHat = imgGrayscale.MorphologyEx(structuringElementEx, CV_MORPH_OP.CV_MOP_TOPHAT, 1) 52 | imgBlackHat = imgGrayscale.MorphologyEx(structuringElementEx, CV_MORPH_OP.CV_MOP_BLACKHAT, 1) 53 | 54 | imgGrayscalePlusTopHat = imgGrayscale + imgTopHat 55 | imgGrayscalePlusTopHatMinusBlackHat = imgGrayscalePlusTopHat - imgBlackHat 56 | 57 | Return imgGrayscalePlusTopHatMinusBlackHat 58 | End Function 59 | 60 | End Module 61 | 62 | -------------------------------------------------------------------------------- /LicensePlateRecognition/frmMain.vb: -------------------------------------------------------------------------------- 1 | 'frmMain.vb 2 | ' 3 | 'Emgu CV 2.4.10 4 | ' 5 | 'add the following components to your form: 6 | 'tableLayoutPanel (TableLayoutPanel) 7 | 'btnOpenFile (Button) 8 | 'lblChosenFile (Label) 9 | 'ibOriginal (ImageBox) 10 | 'txtInfo (TextBox) 11 | 'cbShowSteps (CheckBox) 12 | 'ofdOpenFile (OpenFileDialog) 13 | 14 | Option Explicit On 'require explicit declaration of variables, this is NOT Python !! 15 | Option Strict On 'restrict implicit data type conversions to only widening conversions 16 | 17 | Imports Emgu.CV ' 18 | Imports Emgu.CV.CvEnum 'Emgu Cv imports 19 | Imports Emgu.CV.Structure ' 20 | Imports Emgu.CV.UI ' 21 | 22 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 23 | Public Class frmMain 24 | 25 | ' module level variables '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 26 | Const IMAGE_BOX_PCT_SHOW_STEPS_NOT_CHECKED As Single = 75 27 | Const TEXT_BOX_PCT_SHOW_STEPS_NOT_CHECKED As Single = 25 28 | 29 | Const IMAGE_BOX_PCT_SHOW_STEPS_CHECKED As Single = 55 30 | Const TEXT_BOX_PCT_SHOW_STEPS_CHECKED As Single = 45 31 | 32 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 33 | Private Sub frmMain_Load( sender As Object, e As EventArgs) Handles MyBase.Load 34 | cbShowSteps_CheckedChanged(New Object, New EventArgs) 35 | End Sub 36 | 37 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 38 | Private Sub cbShowSteps_CheckedChanged( sender As Object, e As EventArgs) Handles cbShowSteps.CheckedChanged 39 | If (cbShowSteps.Checked = False) Then 40 | tableLayoutPanel.RowStyles.Item(1).Height = IMAGE_BOX_PCT_SHOW_STEPS_NOT_CHECKED 41 | tableLayoutPanel.RowStyles.Item(2).Height = TEXT_BOX_PCT_SHOW_STEPS_NOT_CHECKED 42 | ElseIf (cbShowSteps.Checked = True) Then 43 | tableLayoutPanel.RowStyles.Item(1).Height = IMAGE_BOX_PCT_SHOW_STEPS_CHECKED 44 | tableLayoutPanel.RowStyles.Item(2).Height = TEXT_BOX_PCT_SHOW_STEPS_CHECKED 45 | End If 46 | End Sub 47 | 48 | ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' 49 | Private Sub btnOpenFile_Click( sender As Object, e As EventArgs) Handles btnOpenFile.Click 50 | Dim drChosenFile As DialogResult 51 | 52 | drChosenFile = ofdOpenFile.ShowDialog() 'open file dialog 53 | 54 | If (drChosenFile <> Windows.Forms.DialogResult.OK Or ofdOpenFile.FileName = "") Then 'if user chose Cancel or filename is blank . . . 55 | lblChosenFile.Text = "file not chosen" 'show error message on label 56 | Return 'and exit function 57 | End If 58 | 59 | Dim imgOriginalScene As Image(Of Bgr, Byte) 'this is the main input image 60 | 61 | Try 62 | imgOriginalScene = New Image(Of Bgr, Byte)(ofdOpenFile.FileName) 'open image 63 | Catch ex As Exception 'if error occurred 64 | lblChosenFile.Text = "unable to open image, error: " + ex.Message 'show error message on label 65 | Return 'and exit function 66 | End Try 67 | 68 | If imgOriginalScene Is Nothing Then 'if image could not be opened 69 | lblChosenFile.Text = "unable to open image" 'show error message on label 70 | Return 'and exit function 71 | End If 72 | 73 | lblChosenFile.Text = ofdOpenFile.FileName 'update label with file name 74 | 75 | CvInvoke.cvDestroyWindow("final imgPlate") 76 | CvInvoke.cvDestroyWindow("final imgThresh") 77 | 78 | ibOriginal.Image = imgOriginalScene 'show original image on main form 79 | 80 | Dim listOfPossiblePlates As List(Of PossiblePlate) = DetectPlates.detectPlatesInScene(imgOriginalScene) 81 | 82 | Dim blnKNNTrainingSuccessful As Boolean = loadKNNDataAndTrainKNN() 83 | 84 | If (blnKNNTrainingSuccessful = False) Then 85 | txtInfo.AppendText(vbCrLf + "error: KNN traning was not successful" + vbCrLf) 86 | Return 87 | End If 88 | 89 | listOfPossiblePlates = DetectChars.detectCharsInPlates(listOfPossiblePlates) 90 | 91 | If (listOfPossiblePlates Is Nothing) Then 92 | txtInfo.AppendText(vbCrLf + "no license plates were detected" + vbCrLf) 93 | ElseIf (listOfPossiblePlates.Count = 0) Then 94 | txtInfo.AppendText(vbCrLf + "no license plates were detected" + vbCrLf) 95 | Else 96 | 'if we get in here list of possible plates has at leat one plate 97 | 98 | 'sort the list of possible plates in DESCENDING order (most number of chars to least number of chars) 99 | listOfPossiblePlates.Sort(Function(onePlate, otherPlate) otherPlate.strChars.Length.CompareTo(onePlate.strChars.Length)) 100 | 101 | 'suppose the plate with the most recognized chars 102 | Dim licPlate As PossiblePlate = listOfPossiblePlates(0) '(the first plate in sorted by string length descending order) 103 | 'is the actual plate 104 | 105 | CvInvoke.cvShowImage("final imgPlate", licPlate.imgPlate) 'show the final color plate image 106 | CvInvoke.cvShowImage("final imgThresh", licPlate.imgThresh) 'show the final thresh plate image 107 | CvInvoke.cvSaveImage("imgThresh.png", licPlate.imgThresh, Nothing) 'save thresh image to file 108 | 109 | If (licPlate.strChars.Length = 0) Then 'if no chars are present in the lic plate, 110 | txtInfo.AppendText(vbCrLf + "no characters were detected" + licPlate.strChars + vbCrLf) 'update info text box 111 | Return 'and return 112 | End If 113 | 114 | imgOriginalScene.Draw(licPlate.b2dLocationOfPlateInScene, New Bgr(Color.Red), 2) 'draw red rectangle around plate 115 | 116 | txtInfo.AppendText(vbCrLf + "license plate read from image = " + licPlate.strChars + vbCrLf) 'update info text box with license plate read 117 | txtInfo.AppendText(vbCrLf + "----------------------------------------" + vbCrLf) 118 | 119 | Dim font As MCvFont = New MCvFont(CvEnum.FONT.CV_FONT_HERSHEY_SIMPLEX, 2.0, 2.0) 'use plane jane font 120 | font.thickness = 3 'make text bold 121 | 122 | Dim textSize As Size 'variable to hold the size the text will occupy on the image 123 | 124 | CvInvoke.cvGetTextSize(licPlate.strChars, font, textSize, 0) 'get the size the text will be so we can correctly position the chars 125 | 126 | 'determine the starting point on the image for writing the chars 127 | Dim ptBottomLeftOfFirstChar As New Point(CInt(licPlate.b2dLocationOfPlateInScene.center.X - CSng(textSize.Width) / 2.0), _ 128 | CInt(licPlate.b2dLocationOfPlateInScene.center.Y + licPlate.b2dLocationOfPlateInScene.size.Height + CSng(textSize.Height))) 129 | 130 | imgOriginalScene.Draw(licPlate.strChars, font, ptBottomLeftOfFirstChar, New Bgr(Color.Yellow)) 'write text of license plate on the image 131 | 132 | ibOriginal.Image = imgOriginalScene 'update form with updated original image 133 | End If 134 | 135 | End Sub 136 | 137 | End Class 138 | 139 | 140 | 141 | -------------------------------------------------------------------------------- /presentation/Steps With Images.docx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/presentation/Steps With Images.docx -------------------------------------------------------------------------------- /presentation/Steps With Images.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/presentation/Steps With Images.pdf -------------------------------------------------------------------------------- /presentation/form_design.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/presentation/form_design.png -------------------------------------------------------------------------------- /presentation/steps.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrocontrollersAndMore/License_Plate_Recognition_Visual_Basic/e12cb5a46de97cd868f355e27bafb3ec5c2bab84/presentation/steps.png -------------------------------------------------------------------------------- /presentation/steps.txt: -------------------------------------------------------------------------------- 1 | 2 | 2 classes: 3 | 4 | PossiblePlate 5 | 6 | PossibleChar 7 | 8 | 9 | 10 | find plates 11 | 12 | find chars within plates 13 | 14 | 15 | detectPlatesInScene() 16 | 17 | detectCharsInPlates() 18 | 19 | 20 | imgOriginalScene 21 | 22 | preprocess() 23 | 24 | imgGrayscaleScene, imgThreshScene 25 | 26 | findPossibleCharsInScene() 27 | 28 | listOfPossibleCharsInScene 29 | 30 | findListOfListsOfMatchingChars() 31 | 32 | listOfListsOfMatchingCharsInScene 33 | 34 | extractPlate() 35 | 36 | listOfPossiblePlates 37 | 38 | 39 | 40 | 41 | loadKNNDataAndTrainKNN() 42 | 43 | 44 | listOfPossiblePlates 45 | 46 | preprocess() 47 | 48 | possiblePlate.imgGrayscale, possiblePlate.imgThresh 49 | 50 | findPossibleCharsInPlate() 51 | 52 | listOfPossibleCharsInPlate 53 | 54 | findListOfListsOfMatchingChars() 55 | 56 | listOfListsOfMatchingCharsInPlate 57 | 58 | removeInnerOverlappingChars() 59 | 60 | listOfListsOfMatchingCharsInPlate 61 | 62 | within each possible plate, suppose the longest list of potential matching chars is the actual list of chars 63 | 64 | longestListOfMatchingCharsInPlate 65 | 66 | recognizeCharsInPlate() 67 | 68 | possiblePlate.strChars 69 | 70 | 71 | 72 | listOfPossiblePlates 73 | 74 | suppose the plate with the most recognized chars is the actual plate 75 | 76 | licPlate 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | -------------------------------------------------------------------------------- /readme.txt: -------------------------------------------------------------------------------- 1 | Video for this project: 2 | https://www.youtube.com/watch?v=8AIFEN5oODM 3 | 4 | If you found this project / video helpful, please consider supporting me on Patreon 5 | https://www.patreon.com/18F4550videos?ty=h 6 | --------------------------------------------------------------------------------