├── .github └── FUNDING.yml ├── .gitignore ├── README.md ├── custom_components └── codeproject_ai_alpr │ ├── __init__.py │ ├── image_processing.py │ └── manifest.json ├── docs ├── card.png ├── event.png └── main.png └── hacs.json /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | # These are supported funding model platforms 2 | 3 | github: robmarkcole 4 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | pip-wheel-metadata/ 24 | share/python-wheels/ 25 | *.egg-info/ 26 | .installed.cfg 27 | *.egg 28 | MANIFEST 29 | 30 | # PyInstaller 31 | # Usually these files are written by a python script from a template 32 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 33 | *.manifest 34 | *.spec 35 | 36 | # Installer logs 37 | pip-log.txt 38 | pip-delete-this-directory.txt 39 | 40 | # Unit test / coverage reports 41 | htmlcov/ 42 | .tox/ 43 | .nox/ 44 | .coverage 45 | .coverage.* 46 | .cache 47 | nosetests.xml 48 | coverage.xml 49 | *.cover 50 | *.py,cover 51 | .hypothesis/ 52 | .pytest_cache/ 53 | 54 | # Translations 55 | *.mo 56 | *.pot 57 | 58 | # Django stuff: 59 | *.log 60 | local_settings.py 61 | db.sqlite3 62 | db.sqlite3-journal 63 | 64 | # Flask stuff: 65 | instance/ 66 | .webassets-cache 67 | 68 | # Scrapy stuff: 69 | .scrapy 70 | 71 | # Sphinx documentation 72 | docs/_build/ 73 | 74 | # PyBuilder 75 | target/ 76 | 77 | # Jupyter Notebook 78 | .ipynb_checkpoints 79 | 80 | # IPython 81 | profile_default/ 82 | ipython_config.py 83 | 84 | # pyenv 85 | .python-version 86 | 87 | # pipenv 88 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 89 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 90 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 91 | # install all needed dependencies. 92 | #Pipfile.lock 93 | 94 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow 95 | __pypackages__/ 96 | 97 | # Celery stuff 98 | celerybeat-schedule 99 | celerybeat.pid 100 | 101 | # SageMath parsed files 102 | *.sage.py 103 | 104 | # Environments 105 | .env 106 | .venv 107 | env/ 108 | venv/ 109 | ENV/ 110 | env.bak/ 111 | venv.bak/ 112 | 113 | # Spyder project settings 114 | .spyderproject 115 | .spyproject 116 | 117 | # Rope project settings 118 | .ropeproject 119 | 120 | # mkdocs documentation 121 | /site 122 | 123 | # mypy 124 | .mypy_cache/ 125 | .dmypy.json 126 | dmypy.json 127 | 128 | # Pyre type checker 129 | .pyre/ 130 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # CodeProject.AI Home Assistant License Plate Reader custom component 2 | 3 | This component is a direct port of the [HASS-plate-recognizer](https://github.com/robmarkcole/HASS-plate-recognizer) component by [Robin Cole](https://github.com/robmarkcole). This component provides AI-based Object Detection capabilities using [CodeProject.AI Server](https://codeproject.com/ai). 4 | 5 | [CodeProject.AI Server](https://codeproject.com/ai) is a service which runs either in a Docker container or as a Windows Service and exposes various an API for many AI inferencing operations via a REST API. The Object Detection capabilities use the [YOLO](https://arxiv.org/pdf/1506.02640.pdf) algorithm as implemented by Ultralytics and others. It can identify 80 different kinds of objects by default, but custom models are also available that focus on specific objects such as animals, license plates or objects typically encountered by home webcams. CodeProject.AI Server is free, locally installed, and can run without an external internet connection, is is comatible with Windows, Linux, macOS. It can run on Raspberry Pi, and supports CUDA and embedded Intel GPUs. 6 | 7 | On the machine in which you are running CodeProject.AI server, either ensure the service is running, or if using Docker, [start a Docker container](https://www.codeproject.com/ai/docs/why/running_in_docker.html#launching-a-container). 8 | 9 | This integration adds an image processing entity where the state of the entity is the number of license plates found in a processed image. Information about the vehicle which has the license plate is provided in the entity attributes, and includes the license plate number, and confidence (in a scale 0 to 1) in this prediction. For each vehicle an `codeproject_ai_alpr.vehicle_detected` event is fired, containing the same information just listed. 10 | 11 | **Note** this integration does NOT automatically process images, it is necessary to call the `image_processing.scan` service to trigger processing. 12 | 13 | ## Home Assistant setup 14 | Place the `custom_components` folder in your configuration directory (or add its contents to an existing `custom_components` folder). Then configure as below: 15 | 16 | ```yaml 17 | image_processing: 18 | - platform: codeproject_ai_alpr 19 | server: http://yoururl:8080/v1/vision/alpr/ 20 | watched_plates: 21 | - kbw46ba 22 | - kfab726 23 | save_file_folder: /config/images/codeproject_ai_alpr/ 24 | save_timestamped_file: True 25 | always_save_latest_file: True 26 | source: 27 | - entity_id: camera.yours 28 | ``` 29 | Then, **restart** your Home Assistant 30 | 31 | Configuration variables: 32 | - **server**: (CodeProject.AI ALPS instance URL) 33 | - **watched_plates**: (Optional) A list of number plates to watch for, which will identify a plate even if a couple of digits are incorrect in the prediction (fuzzy matching). If configured this adds an attribute to the entity with a boolean for each watched plate to indicate if it is detected. 34 | - **save_file_folder**: (Optional) The folder to save processed images to. Note that folder path should be added to [whitelist_external_dirs](https://www.home-assistant.io/docs/configuration/basic/) 35 | - **save_timestamped_file**: (Optional, default `False`, requires `save_file_folder` to be configured) Save the processed image with the time of detection in the filename. 36 | - **always_save_latest_file**: (Optional, default `False`, requires `save_file_folder` to be configured) Always save the last processed image, no matter there were detections or not. 37 | - **source**: Must be a camera. 38 | - **unique_id**: Unique id of the entity. 39 | 40 | ## Making a sensor for individual plates 41 | If you have configured `watched_plates` you can create a binary sensor for each watched plate, using a [template sensor](https://www.home-assistant.io/integrations/template/) as below, which is an example for plate `kbw46ba`: 42 | 43 | ```yaml 44 | sensor: 45 | - platform: template 46 | sensors: 47 | my_plate: 48 | friendly_name: "kbw46ba" 49 | value_template: "{{ state_attr("image_processing.codeproject_ai_alpr_1", "watched_plates")["kbw46ba"] }}" 50 | ``` 51 | -------------------------------------------------------------------------------- /custom_components/codeproject_ai_alpr/__init__.py: -------------------------------------------------------------------------------- 1 | """The codeproject_ai_alpr integration.""" 2 | -------------------------------------------------------------------------------- /custom_components/codeproject_ai_alpr/image_processing.py: -------------------------------------------------------------------------------- 1 | """Vehicle detection using Plate Recognizer cloud service.""" 2 | import logging 3 | import requests 4 | import voluptuous as vol 5 | import re 6 | import io 7 | from typing import List, Dict 8 | import json 9 | 10 | from PIL import Image, ImageDraw, UnidentifiedImageError 11 | from pathlib import Path 12 | 13 | from homeassistant.components.image_processing import ( 14 | CONF_ENTITY_ID, 15 | CONF_NAME, 16 | CONF_SOURCE, 17 | PLATFORM_SCHEMA, 18 | ImageProcessingEntity, 19 | ) 20 | from homeassistant.const import ATTR_ENTITY_ID 21 | from homeassistant.core import split_entity_id 22 | import homeassistant.helpers.config_validation as cv 23 | import homeassistant.util.dt as dt_util 24 | from homeassistant.util.pil import draw_box 25 | 26 | _LOGGER = logging.getLogger(__name__) 27 | 28 | EVENT_VEHICLE_DETECTED = "codeproject_ai_alpr.vehicle_detected" 29 | 30 | ATTR_PLATE = "plate" 31 | ATTR_CONFIDENCE = "confidence" 32 | ATTR_BOX_Y_CENTRE = "box_y_centre" 33 | ATTR_BOX_X_CENTRE = "box_x_centre" 34 | 35 | CONF_SAVE_FILE_FOLDER = "save_file_folder" 36 | CONF_SAVE_TIMESTAMPTED_FILE = "save_timestamped_file" 37 | CONF_ALWAYS_SAVE_LATEST_FILE = "always_save_latest_file" 38 | CONF_WATCHED_PLATES = "watched_plates" 39 | CONF_SERVER = "server" 40 | CONF_UNIQUE_ID = "unique_id" 41 | 42 | DATETIME_FORMAT = "%Y-%m-%d_%H-%M-%S" 43 | RED = (255, 0, 0) # For objects within the ROI 44 | 45 | PLATFORM_SCHEMA = PLATFORM_SCHEMA.extend( 46 | { 47 | vol.Required(CONF_SERVER): cv.string, 48 | vol.Optional(CONF_UNIQUE_ID): cv.string, 49 | vol.Optional(CONF_SAVE_FILE_FOLDER): cv.isdir, 50 | vol.Optional(CONF_SAVE_TIMESTAMPTED_FILE, default=False): cv.boolean, 51 | vol.Optional(CONF_ALWAYS_SAVE_LATEST_FILE, default=False): cv.boolean, 52 | vol.Optional(CONF_WATCHED_PLATES): vol.All( 53 | cv.ensure_list, [cv.string] 54 | ), 55 | } 56 | ) 57 | 58 | def setup_platform(hass, config, add_entities, discovery_info=None): 59 | """Set up the platform.""" 60 | # Validate credentials by processing image. 61 | save_file_folder = config.get(CONF_SAVE_FILE_FOLDER) 62 | if save_file_folder: 63 | save_file_folder = Path(save_file_folder) 64 | 65 | entities = [] 66 | for camera in config[CONF_SOURCE]: 67 | codeproject_ai_alpr = codeproject_ai_alprEntity( 68 | save_file_folder=save_file_folder, 69 | save_timestamped_file=config.get(CONF_SAVE_TIMESTAMPTED_FILE), 70 | always_save_latest_file=config.get(CONF_ALWAYS_SAVE_LATEST_FILE), 71 | watched_plates=config.get(CONF_WATCHED_PLATES), 72 | camera_entity=camera[CONF_ENTITY_ID], 73 | name=camera.get(CONF_NAME), 74 | server=config.get(CONF_SERVER), 75 | unique_id=config.get(CONF_UNIQUE_ID), 76 | 77 | ) 78 | entities.append(codeproject_ai_alpr) 79 | add_entities(entities) 80 | 81 | 82 | class codeproject_ai_alprEntity(ImageProcessingEntity): 83 | """Create entity.""" 84 | 85 | def __init__( 86 | self, 87 | save_file_folder, 88 | save_timestamped_file, 89 | always_save_latest_file, 90 | watched_plates, 91 | camera_entity, 92 | name, 93 | server, 94 | unique_id, 95 | ): 96 | """Init.""" 97 | self._headers = "" 98 | self._camera = camera_entity 99 | if name: 100 | self._name = name 101 | else: 102 | camera_name = split_entity_id(camera_entity)[1] 103 | self._name = f"codeproject_ai_alpr_{camera_name}" 104 | self._save_file_folder = save_file_folder 105 | self._save_timestamped_file = save_timestamped_file 106 | self._always_save_latest_file = always_save_latest_file 107 | self._watched_plates = watched_plates 108 | self._server = server 109 | self._state = None 110 | self._results = {} 111 | self._vehicles = [{}] 112 | self._last_detection = None 113 | self._image_width = None 114 | self._image_height = None 115 | self._image = None 116 | self._config = {} 117 | self._unique_id = unique_id 118 | self._inference_time = None 119 | 120 | def process_image(self, image): 121 | """Process an image.""" 122 | self._state = None 123 | self._results = {} 124 | self._vehicles = [{}] 125 | self._image = Image.open(io.BytesIO(bytearray(image))) 126 | self._image_width, self._image_height = self._image.size 127 | 128 | try: 129 | _LOGGER.debug("Config: " + str(json.dumps(self._config))) 130 | response = requests.post( 131 | self._server, 132 | files={"upload": image}, 133 | headers=self._headers 134 | ).json() 135 | self._results = response["predictions"] 136 | self._inference_time = response["inferenceMs"] 137 | self._vehicles = [ 138 | { 139 | ATTR_PLATE: r["plate"], 140 | ATTR_CONFIDENCE: r["confidence"], 141 | ATTR_BOX_Y_CENTRE: (r["y_min"] + ((r["y_max"] - r["y_min"]) /2)), 142 | ATTR_BOX_X_CENTRE: (r["x_min"] + ((r["x_max"] - r["x_min"]) /2)), 143 | } 144 | for r in self._results 145 | ] 146 | except Exception as exc: 147 | _LOGGER.error("codeproject_ai_alpr error: %s", exc) 148 | _LOGGER.error(f"codeproject_ai_alpr api response: {response}") 149 | 150 | self._state = len(self._vehicles) 151 | if self._state > 0: 152 | self._last_detection = dt_util.now().strftime(DATETIME_FORMAT) 153 | for vehicle in self._vehicles: 154 | self.fire_vehicle_detected_event(vehicle) 155 | if self._save_file_folder: 156 | if self._state > 0 or self._always_save_latest_file: 157 | self.save_image() 158 | 159 | def fire_vehicle_detected_event(self, vehicle): 160 | """Send event.""" 161 | vehicle_copy = vehicle.copy() 162 | vehicle_copy.update({ATTR_ENTITY_ID: self.entity_id}) 163 | self.hass.bus.fire(EVENT_VEHICLE_DETECTED, vehicle_copy) 164 | 165 | def save_image(self): 166 | """Save a timestamped image with bounding boxes around plates.""" 167 | draw = ImageDraw.Draw(self._image) 168 | 169 | decimal_places = 3 170 | for vehicle in self._results: 171 | box = ( 172 | round(vehicle["y_min"] / self._image_height, decimal_places), 173 | round(vehicle["x_min"] / self._image_width, decimal_places), 174 | round(vehicle["y_max"] / self._image_height, decimal_places), 175 | round(vehicle["x_max"] / self._image_width, decimal_places), 176 | ) 177 | text = vehicle['label'] 178 | draw_box( 179 | draw, 180 | box, 181 | self._image_width, 182 | self._image_height, 183 | text=text, 184 | color=RED, 185 | ) 186 | 187 | latest_save_path = self._save_file_folder / f"{self._name}_latest.png" 188 | self._image.save(latest_save_path) 189 | 190 | if self._save_timestamped_file: 191 | timestamp_save_path = self._save_file_folder / f"{self._name}_{self._last_detection}.png" 192 | self._image.save(timestamp_save_path) 193 | _LOGGER.info("codeproject_ai_alpr saved file %s", timestamp_save_path) 194 | 195 | @property 196 | def camera_entity(self): 197 | """Return camera entity id from process pictures.""" 198 | return self._camera 199 | 200 | @property 201 | def name(self): 202 | """Return the name of the sensor.""" 203 | return self._name 204 | 205 | @property 206 | def unique_id(self): 207 | """Return the unique id of the sensor.""" 208 | return self._unique_id 209 | 210 | @property 211 | def should_poll(self): 212 | """Return the polling state.""" 213 | return False 214 | 215 | @property 216 | def state(self): 217 | """Return the state of the entity.""" 218 | return self._state 219 | 220 | @property 221 | def unit_of_measurement(self): 222 | """Return the unit of measurement.""" 223 | return ATTR_PLATE 224 | 225 | @property 226 | def extra_state_attributes(self): 227 | """Return the attributes.""" 228 | attr = {} 229 | attr.update({"last_detection": self._last_detection}) 230 | attr.update({"vehicles": self._vehicles}) 231 | detected_plates = [ r["plate"] for r in self._results ] 232 | if self._watched_plates: 233 | watched_plates_results = {plate : False for plate in self._watched_plates} 234 | for plate in self._watched_plates: 235 | if plate in detected_plates: 236 | watched_plates_results.update({plate: True}) 237 | attr[CONF_WATCHED_PLATES] = watched_plates_results 238 | attr[CONF_SERVER] = str(self._server) 239 | attr.update({"inference_time": self._inference_time}) 240 | if self._save_file_folder: 241 | attr[CONF_SAVE_FILE_FOLDER] = str(self._save_file_folder) 242 | attr[CONF_SAVE_TIMESTAMPTED_FILE] = self._save_timestamped_file 243 | attr[CONF_ALWAYS_SAVE_LATEST_FILE] = self._always_save_latest_file 244 | return attr 245 | -------------------------------------------------------------------------------- /custom_components/codeproject_ai_alpr/manifest.json: -------------------------------------------------------------------------------- 1 | { 2 | "domain": "codeproject_ai_alpr", 3 | "name": "CodeProject.AI Server ALPR custom integration", 4 | "documentation": "https://github.com/grinco/CodeProject.AI-HomeAssist-ALPR", 5 | "version": "0.2.0", 6 | "iot_class": "local_polling", 7 | "requirements": ["pillow", "requests"], 8 | "dependencies": [], 9 | "codeowners": [ 10 | "@grinco" 11 | ] 12 | } 13 | -------------------------------------------------------------------------------- /docs/card.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/grinco/CodeProject.AI-HomeAssist-ALPR/779f13c2254eb5f1b6b8964d2a52ec97d55560fc/docs/card.png -------------------------------------------------------------------------------- /docs/event.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/grinco/CodeProject.AI-HomeAssist-ALPR/779f13c2254eb5f1b6b8964d2a52ec97d55560fc/docs/event.png -------------------------------------------------------------------------------- /docs/main.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/grinco/CodeProject.AI-HomeAssist-ALPR/779f13c2254eb5f1b6b8964d2a52ec97d55560fc/docs/main.png -------------------------------------------------------------------------------- /hacs.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "CodeProject.AI Server ALPR custom integration" 3 | } 4 | --------------------------------------------------------------------------------