├── fonts └── arialbd.ttf ├── README.md ├── .gitignore ├── demo.py └── main.py /fonts/arialbd.ttf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/oianmol/licence_plate_torch/master/fonts/arialbd.ttf -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # License Plate Character Recognition 3 | 4 | This repository contains a pipeline for generating synthetic character datasets, preprocessing images, and training a Convolutional Neural Network (CNN) model to recognize alphanumeric characters from license plates. The implementation uses PyTorch and leverages synthetic data generation with the `TextRecognitionDataGenerator`. 5 | 6 | --- 7 | 8 | ## Features 9 | 10 | 1. **Synthetic Dataset Generation**: Generate images for alphanumeric characters (`0-9`, `A-Z`) using a specified font. 11 | 2. **Preprocessing**: Normalize and resize images to a uniform size and save them as tensors. 12 | 3. **Custom Dataset Class**: Load the preprocessed data for training and evaluation. 13 | 4. **CNN Model**: A convolutional neural network for character classification with batch normalization and dropout for regularization. 14 | 5. **Training and Evaluation**: Train the model on the generated dataset and evaluate its accuracy. 15 | 16 | --- 17 | 18 | ## Installation 19 | 20 | 1. Clone the repository: 21 | 22 | ```bash 23 | git clone https://github.com/oianmol/license-plate-recognition.git 24 | cd license-plate-recognition 25 | ``` 26 | 27 | 2. Install dependencies: 28 | 29 | ```bash 30 | pip install -r requirements.txt 31 | ``` 32 | 33 | 3. Ensure you have the following additional packages installed: 34 | - `Pillow` 35 | - `torch` 36 | - `torchvision` 37 | - `trdg` (TextRecognitionDataGenerator) 38 | 39 | 4. Download a suitable TTF font (e.g., Arial Bold) and specify its path in the script. 40 | 41 | --- 42 | 43 | ## Usage 44 | 45 | ### 1. Generate Synthetic Dataset 46 | Run the script to generate synthetic images for characters (`0-9`, `A-Z`): 47 | 48 | ```python 49 | generate_characters(output_dir="dataset/characters", font_path="/path/to/font.ttf", count_per_char=100) 50 | ``` 51 | 52 | ### 2. Preprocess Images 53 | Preprocess the generated images by resizing, normalizing, and converting them to tensors: 54 | 55 | ```python 56 | preprocess_characters(input_dir="dataset/characters", output_dir="dataset/preprocessed") 57 | ``` 58 | 59 | ### 3. Train the Model 60 | Train the character recognition model on the preprocessed dataset: 61 | 62 | ```bash 63 | python main.py 64 | ``` 65 | 66 | The model will be saved as `trained_model.pth` in the dataset directory upon completion. 67 | 68 | --- 69 | 70 | ## Model Architecture 71 | 72 | The CNN model includes: 73 | 74 | 1. **Convolutional Layers**: Extract spatial features from the input images with ReLU activation. 75 | 2. **Batch Normalization**: Normalize activations for stable training. 76 | 3. **Pooling Layers**: Reduce the spatial dimensions while retaining key features. 77 | 4. **Fully Connected Layers**: Perform classification on the extracted features. 78 | 79 | --- 80 | 81 | ## File Structure 82 | 83 | ```plaintext 84 | . 85 | ├── dataset/ 86 | │ ├── characters/ # Synthetic character images 87 | │ └── preprocessed/ # Preprocessed tensors 88 | ├── fonts/ # Directory to store fonts 89 | ├── main.py # Main script for training and evaluation 90 | ├── requirements.txt # Python dependencies 91 | └── README.md # Project documentation 92 | ``` 93 | 94 | --- 95 | 96 | ## Dependencies 97 | 98 | - Python 3.8+ 99 | - PyTorch 1.9+ 100 | - torchvision 0.10+ 101 | - Pillow 102 | - TextRecognitionDataGenerator 103 | 104 | --- 105 | 106 | ## Future Enhancements 107 | 108 | - Integration with real-world datasets for transfer learning. 109 | - Extend the model to support multi-line text recognition. 110 | - Add functionality for license plate segmentation and character extraction. 111 | 112 | --- 113 | 114 | ## License 115 | 116 | This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details. 117 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | ### Python template 2 | # Byte-compiled / optimized / DLL files 3 | __pycache__/ 4 | *.py[cod] 5 | *$py.class 6 | 7 | # C extensions 8 | *.so 9 | 10 | # Distribution / packaging 11 | .Python 12 | build/ 13 | develop-eggs/ 14 | dist/ 15 | downloads/ 16 | eggs/ 17 | .eggs/ 18 | lib/ 19 | lib64/ 20 | parts/ 21 | sdist/ 22 | var/ 23 | wheels/ 24 | share/python-wheels/ 25 | *.egg-info/ 26 | .installed.cfg 27 | *.egg 28 | MANIFEST 29 | 30 | # PyInstaller 31 | # Usually these files are written by a python script from a template 32 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 33 | *.manifest 34 | *.spec 35 | 36 | # Installer logs 37 | pip-log.txt 38 | pip-delete-this-directory.txt 39 | 40 | # Unit test / coverage reports 41 | htmlcov/ 42 | .tox/ 43 | .nox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | *.py,cover 51 | .hypothesis/ 52 | .pytest_cache/ 53 | cover/ 54 | 55 | # Translations 56 | *.mo 57 | *.pot 58 | 59 | # Django stuff: 60 | *.log 61 | local_settings.py 62 | db.sqlite3 63 | db.sqlite3-journal 64 | 65 | # Flask stuff: 66 | instance/ 67 | .webassets-cache 68 | 69 | # Scrapy stuff: 70 | .scrapy 71 | 72 | # Sphinx documentation 73 | docs/_build/ 74 | 75 | # PyBuilder 76 | .pybuilder/ 77 | target/ 78 | 79 | # Jupyter Notebook 80 | .ipynb_checkpoints 81 | 82 | # IPython 83 | profile_default/ 84 | ipython_config.py 85 | 86 | # pyenv 87 | # For a library or package, you might want to ignore these files since the code is 88 | # intended to run in multiple environments; otherwise, check them in: 89 | # .python-version 90 | 91 | # pipenv 92 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 93 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 94 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 95 | # install all needed dependencies. 96 | #Pipfile.lock 97 | 98 | # poetry 99 | # Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control. 100 | # This is especially recommended for binary packages to ensure reproducibility, and is more 101 | # commonly ignored for libraries. 102 | # https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control 103 | #poetry.lock 104 | 105 | # pdm 106 | # Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control. 107 | #pdm.lock 108 | # pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it 109 | # in version control. 110 | # https://pdm.fming.dev/#use-with-ide 111 | .pdm.toml 112 | 113 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm 114 | __pypackages__/ 115 | 116 | # Celery stuff 117 | celerybeat-schedule 118 | celerybeat.pid 119 | 120 | # SageMath parsed files 121 | *.sage.py 122 | dataset 123 | # Environments 124 | .env 125 | .venv 126 | env/ 127 | venv/ 128 | ENV/ 129 | env.bak/ 130 | venv.bak/ 131 | 132 | # Spyder project settings 133 | .spyderproject 134 | .spyproject 135 | 136 | # Rope project settings 137 | .ropeproject 138 | 139 | # mkdocs documentation 140 | /site 141 | 142 | # mypy 143 | .mypy_cache/ 144 | .dmypy.json 145 | dmypy.json 146 | 147 | # Pyre type checker 148 | .pyre/ 149 | 150 | # pytype static type analyzer 151 | .pytype/ 152 | 153 | # Cython debug symbols 154 | cython_debug/ 155 | 156 | # PyCharm 157 | # JetBrains specific template is maintained in a separate JetBrains.gitignore that can 158 | # be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore 159 | # and can be added to the global gitignore or merged into this file. For a more nuclear 160 | # option (not recommended) you can uncomment the following to ignore the entire idea folder. 161 | .idea/ 162 | segmented_with_contours.jpg 163 | segmented_characters -------------------------------------------------------------------------------- /demo.py: -------------------------------------------------------------------------------- 1 | from pathlib import Path 2 | 3 | import cv2 4 | import os 5 | import torch 6 | from torchvision import transforms 7 | from PIL import Image 8 | 9 | from main import LicensePlateRecognizer 10 | 11 | # Character mapping (used during training) 12 | int_to_char = {idx: char for idx, char in enumerate("0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ")} 13 | 14 | # Preprocessing function (same as during training) 15 | def preprocess_image(image): 16 | transform = transforms.Compose([ 17 | transforms.Grayscale(), 18 | transforms.Resize((28, 28)), 19 | transforms.ToTensor(), 20 | transforms.Normalize((0.5,), (0.5,)) 21 | ]) 22 | return transform(image).unsqueeze(0) # Add batch dimension 23 | 24 | # Load the trained model 25 | def load_model(model_path, num_classes=36): 26 | model = LicensePlateRecognizer(num_classes) 27 | model.load_state_dict(torch.load(model_path, map_location=torch.device("cpu"))) 28 | model.eval() # Set the model to evaluation mode 29 | return model 30 | 31 | # Predict a single character 32 | def predict_character(model, image_tensor): 33 | with torch.no_grad(): 34 | output = model(image_tensor) 35 | _, predicted = torch.max(output, 1) 36 | return int_to_char[predicted.item()] 37 | 38 | # Segment characters from a license plate 39 | def segment_characters(license_plate_path, output_path="segmented_with_contours.jpg"): 40 | """ 41 | Segment characters from a license plate image, draw contours on the original image, 42 | and save each segmented character as an image. 43 | """ 44 | # Load the image 45 | image = cv2.imread(license_plate_path) 46 | original_image = image.copy() # Keep a copy of the original image 47 | gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) # Convert to grayscale 48 | _, thresh = cv2.threshold(gray, 128, 255, cv2.THRESH_BINARY_INV) # Thresholding 49 | 50 | # Detect contours 51 | contours, _ = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) 52 | 53 | # Sort contours from left to right 54 | contours = sorted(contours, key=lambda ctr: cv2.boundingRect(ctr)[0]) 55 | 56 | characters = [] 57 | for i, contour in enumerate(contours): 58 | x, y, w, h = cv2.boundingRect(contour) 59 | if w > 6 and h > 6: # Filter out noise 60 | # Draw a bounding box around each character on the original image 61 | cv2.rectangle(original_image, (x, y), (x + w, y + h), (0, 255, 0), 2) 62 | 63 | # Extract the character region 64 | char = extract_character_with_padding(gray, x, y, w, h) 65 | 66 | char = cv2.resize(char, (28, 28)) # Resize to match training 67 | characters.append(char) 68 | 69 | # Save the image with drawn contours 70 | cv2.imwrite(output_path, original_image) 71 | print(f"Image with contours saved to {output_path}") 72 | 73 | return characters 74 | def extract_character_with_padding(gray, x, y, w, h, padding=5): 75 | """ 76 | Extract a character from the image with padding, ensuring it doesn't exceed the image boundaries. 77 | 78 | Args: 79 | gray (ndarray): Grayscale input image. 80 | x, y, w, h (int): Bounding box coordinates and dimensions. 81 | padding (int): Amount of padding to add around the character. 82 | 83 | Returns: 84 | ndarray: Padded character region. 85 | """ 86 | height, width = gray.shape 87 | 88 | # Calculate padded coordinates 89 | x_start = max(0, x - padding) 90 | y_start = max(0, y - padding) 91 | x_end = min(width, x + w + padding) 92 | y_end = min(height, y + h + padding) 93 | 94 | # Extract the padded region 95 | return gray[y_start:y_end, x_start:x_end] 96 | # Predict the full license plate number 97 | def predict_license_plate(model, license_plate_path, char_output_dir="segmented_characters"): 98 | """ 99 | Predict the full license plate from an image of the license plate. 100 | """ 101 | # Segment characters from the license plate 102 | segmented_characters = segment_characters(license_plate_path) 103 | 104 | # Create output directory for segmented characters 105 | os.makedirs(char_output_dir, exist_ok=True) 106 | 107 | # Predict each character 108 | index = 0 109 | license_plate = "" 110 | for char_image in segmented_characters: 111 | char_image_pil = Image.fromarray(char_image) # Convert to PIL Image 112 | image_tensor = preprocess_image(char_image_pil) 113 | predicted_char = predict_character(model, image_tensor) 114 | # Save the character image 115 | char_path = os.path.join(char_output_dir, f"char_{index}_{predicted_char}.png") 116 | cv2.imwrite(char_path, char_image) 117 | print(f"Saved character image: {char_path}") 118 | index = index+1 119 | license_plate += predicted_char 120 | 121 | return license_plate 122 | 123 | # Example Usage 124 | if __name__ == "__main__": 125 | # Paths 126 | model_path = f"{Path.home()}/hellotorch/dataset/trained_model.pth" 127 | license_plate_image = f"{Path.home()}/hellotorch/dataset/license_plate_dataset/plate_4.png" 128 | 129 | # Load the trained model 130 | model = load_model(model_path) 131 | 132 | # Predict the license plate number 133 | predicted_plate = predict_license_plate(model, license_plate_image) 134 | print(f"Predicted License Plate: {predicted_plate}") -------------------------------------------------------------------------------- /main.py: -------------------------------------------------------------------------------- 1 | import os 2 | from pathlib import Path 3 | 4 | from PIL import Image 5 | from trdg.generators import GeneratorFromStrings 6 | import torch 7 | from torchvision import transforms 8 | from torch.utils.data import Dataset, DataLoader 9 | import torch.nn as nn 10 | import torch.optim as optim 11 | 12 | 13 | # Step 1: Generate Synthetic Character Dataset 14 | def generate_characters(output_dir, font_path, count_per_char=1): 15 | """ 16 | Generate images for characters (0-9, A-Z) using TextRecognitionDataGenerator. 17 | """ 18 | os.makedirs(output_dir, exist_ok=True) 19 | characters = list("0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ") 20 | 21 | generator = GeneratorFromStrings( 22 | characters, 23 | count=count_per_char * len(characters), 24 | fonts=[font_path], 25 | size=64, 26 | random_blur=False, 27 | skewing_angle=0, 28 | background_type=0, 29 | distorsion_type=0, 30 | ) 31 | 32 | for idx, (image, label) in enumerate(generator): 33 | char_output_dir = os.path.join(output_dir, label) 34 | os.makedirs(char_output_dir, exist_ok=True) 35 | image.save(os.path.join(char_output_dir, f"{label}_{idx}.png")) 36 | print(f"Generated: {label}_{idx}.png") 37 | 38 | 39 | # Step 2: Preprocess the Dataset 40 | def preprocess_characters(input_dir, output_dir): 41 | """ 42 | Preprocess character images: Resize, normalize, and save as tensors. 43 | """ 44 | transform = transforms.Compose([ 45 | transforms.Grayscale(), 46 | transforms.Resize((28, 28)), 47 | transforms.ToTensor(), 48 | transforms.Normalize((0.5,), (0.5,)) 49 | ]) 50 | 51 | os.makedirs(output_dir, exist_ok=True) 52 | 53 | for char_label in os.listdir(input_dir): 54 | char_input_dir = os.path.join(input_dir, char_label) 55 | char_output_dir = os.path.join(output_dir, char_label) 56 | os.makedirs(char_output_dir, exist_ok=True) 57 | 58 | for image_file in os.listdir(char_input_dir): 59 | image_path = os.path.join(char_input_dir, image_file) 60 | image = Image.open(image_path) 61 | processed_image = transform(image) 62 | torch.save(processed_image, os.path.join(char_output_dir, f"{image_file}.pt")) 63 | print(f"Preprocessed: {image_file} -> {char_label}") 64 | 65 | 66 | # Step 3: Define the Dataset Class 67 | class CharacterDataset(Dataset): 68 | def __init__(self, data_dir): 69 | """ 70 | Initialize the dataset with preprocessed character data. 71 | """ 72 | self.samples = [] 73 | self.data_dir = data_dir 74 | 75 | # Define a mapping from characters to integers 76 | self.char_to_int = {char: idx for idx, char in enumerate("0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ")} 77 | 78 | for char_label in os.listdir(data_dir): 79 | if char_label not in self.char_to_int: 80 | print(f"Skipping unknown label: {char_label}") 81 | continue 82 | 83 | char_dir = os.path.join(data_dir, char_label) 84 | for file in os.listdir(char_dir): 85 | self.samples.append((os.path.join(char_dir, file), self.char_to_int[char_label])) 86 | 87 | def __len__(self): 88 | return len(self.samples) 89 | 90 | def __getitem__(self, idx): 91 | file_path, label = self.samples[idx] 92 | tensor = torch.load(file_path) 93 | return tensor, label 94 | 95 | 96 | # Step 4: Define the Model 97 | class LicensePlateRecognizer(nn.Module): 98 | def __init__(self, num_classes): 99 | super(LicensePlateRecognizer, self).__init__() 100 | 101 | # Convolutional layers 102 | self.conv_layer = nn.Sequential( 103 | nn.Conv2d(1, 32, kernel_size=3, padding=1), 104 | nn.ReLU(), 105 | nn.BatchNorm2d(32), # Batch normalization 106 | nn.MaxPool2d(2, 2), 107 | 108 | nn.Conv2d(32, 64, kernel_size=3, padding=1), 109 | nn.ReLU(), 110 | nn.BatchNorm2d(64), # Batch normalization 111 | nn.MaxPool2d(2, 2), 112 | 113 | nn.Conv2d(64, 128, kernel_size=3, padding=1), # Additional layer 114 | nn.ReLU(), 115 | nn.BatchNorm2d(128), # Batch normalization 116 | nn.MaxPool2d(2, 2) 117 | ) 118 | 119 | # Fully connected layers 120 | self.fc_layer = nn.Sequential( 121 | nn.Flatten(), 122 | nn.Linear(128 * 3 * 3, 128), # Adjust based on input size 123 | nn.ReLU(), 124 | nn.Dropout(0.5), # Dropout for regularization 125 | nn.Linear(128, num_classes) # Output layer for classification 126 | ) 127 | 128 | def forward(self, x): 129 | x = self.conv_layer(x) 130 | x = self.fc_layer(x) 131 | return x 132 | 133 | 134 | # Step 5: Train and Evaluate the Model 135 | def train_model(model, dataloader, criterion, optimizer, epochs=10): 136 | model.train() 137 | for epoch in range(epochs): 138 | total_loss = 0 139 | for images, labels in dataloader: 140 | images, labels = images.to(device), labels.to(device) 141 | 142 | # Forward pass 143 | outputs = model(images) 144 | loss = criterion(outputs, labels) 145 | 146 | # Backward pass and optimization 147 | optimizer.zero_grad() 148 | loss.backward() 149 | optimizer.step() 150 | 151 | total_loss += loss.item() 152 | 153 | print(f"Epoch [{epoch + 1}/{epochs}], Loss: {total_loss / len(dataloader):.4f}") 154 | 155 | 156 | def evaluate_model(model, dataloader): 157 | model.eval() 158 | correct = 0 159 | total = 0 160 | with torch.no_grad(): 161 | for images, labels in dataloader: 162 | images, labels = images.to(device), labels.to(device) 163 | outputs = model(images) 164 | _, predicted = torch.max(outputs, 1) 165 | total += labels.size(0) 166 | correct += (predicted == labels).sum().item() 167 | 168 | print(f"Accuracy: {100 * correct / total:.2f}%") 169 | 170 | 171 | # Main Pipeline 172 | if __name__ == "__main__": 173 | # Paths 174 | font_path = f"{Path.home()}/hellotorch/fonts/arialbd.ttf" 175 | character_output_dir = f"{Path.home()}/hellotorch/dataset/characters" 176 | preprocessed_output_dir = f"{Path.home()}/hellotorch/dataset/preprocessed" 177 | 178 | # Step 1: Generate synthetic character images 179 | generate_characters(output_dir=character_output_dir, font_path=font_path) 180 | 181 | # Step 2: Preprocess character images 182 | preprocess_characters(input_dir=character_output_dir, output_dir=preprocessed_output_dir) 183 | 184 | # Step 3: Load Dataset 185 | dataset = CharacterDataset(preprocessed_output_dir) 186 | dataloader = DataLoader(dataset, batch_size=32, shuffle=True) 187 | 188 | # Step 4: Define Model, Loss, Optimizer 189 | num_classes = 36 # 0-9, A-Z 190 | device = torch.device("cuda" if torch.cuda.is_available() else "cpu") 191 | model = LicensePlateRecognizer(num_classes).to(device) 192 | criterion = nn.CrossEntropyLoss() 193 | optimizer = optim.Adam(model.parameters(), lr=0.001) 194 | 195 | # Step 5: Train and Evaluate 196 | train_model(model, dataloader, criterion, optimizer, epochs=10) 197 | evaluate_model(model, dataloader) 198 | 199 | # Save the trained model 200 | torch.save(model.state_dict(), f"{Path.home()}/hellotorch/dataset/trained_model.pth") 201 | print("Model saved as trained_model.pth") --------------------------------------------------------------------------------