├── README.md ├── captcha.ipynb ├── captcha_extractor.py ├── captcha_extractor_model.hdf5 ├── captcha_labels └── test_sample_captcha.png /README.md: -------------------------------------------------------------------------------- 1 | # Captcha-text-extraction 2 | 3 | 4 | ## Introduction 5 | This captcha text extractor uses open cv and keras to extract the text. It uses a CNN model trained on the seprate images of letters of the captcha. 6 | It takes an input image, output folder, cnn model and a lablebinarizier file and returns an output image with predicted captcha text as image name and also displays the output showing the predicted captcha text. 7 | 8 | ## How it works 9 | 10 | 1. Read the image using open cv 11 | 2. Apply threshold 12 | 3. Find contours 13 | 4. Extract the image regions containing letters 14 | 5. Use the trained model to predict the letters 15 | 6. Combine the predicted letters 16 | 7. Display the results 17 | 18 | 19 | ## Files Description 20 | - captcha.ipynb: It extracts the captcha images into separate images of letters. It also containes the complete project as jupyter notebook. 21 | - captcha_extractor.py: Extract the captch text, save and display the output captcha image. 22 | - captcha_extractor_model.hdf5: Trained CNN model used for prediction. 23 | - captcha_labels: Holds the labelbinarizer needed to predict the letters. 24 | - test_sample_captcha.png: It is a sample captcha image for testing. 25 | 26 | 27 | ## Libraries required: 28 | 1. open cv 29 | 2. keras 30 | 31 | 32 | ## How to use 33 | captcha_extractor.py needs few arguments: 34 | 1. --input: Path to the input png captcha image. 35 | 2. --output: path to the output folder. 36 | 3. --model: path to the 'captcha_extractor_model.hdf5' file. 37 | 4. --labels: Path to the captcha_labels file 38 | 39 | ## Sample 40 | ``` 41 | $python captcha_extractor.py --input test_sample_captcha.png --output ./out --model captcha_extractor_model.hdf5 --labels captcha_labels 42 | ``` 43 | -------------------------------------------------------------------------------- /captcha.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "code", 5 | "execution_count": 1, 6 | "metadata": {}, 7 | "outputs": [], 8 | "source": [ 9 | "import pandas as pd\n", 10 | "import numpy as np\n", 11 | "import cv2\n", 12 | "import glob\n", 13 | "import imutils\n", 14 | "from imutils import paths\n", 15 | "import os\n", 16 | "import os.path" 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "metadata": {}, 22 | "source": [ 23 | "## Extracting letters from each captcha for training the model" 24 | ] 25 | }, 26 | { 27 | "cell_type": "code", 28 | "execution_count": 2, 29 | "metadata": {}, 30 | "outputs": [ 31 | { 32 | "name": "stdout", 33 | "output_type": "stream", 34 | "text": [ 35 | "processing image 1/939\n", 36 | "processing image 2/939\n", 37 | "processing image 3/939\n", 38 | "processing image 4/939\n", 39 | "processing image 5/939\n", 40 | "processing image 6/939\n", 41 | "processing image 7/939\n", 42 | "processing image 8/939\n", 43 | "processing image 9/939\n", 44 | "processing image 10/939\n", 45 | "processing image 11/939\n", 46 | "processing image 12/939\n", 47 | "processing image 13/939\n", 48 | "processing image 14/939\n", 49 | "processing image 15/939\n", 50 | "processing image 16/939\n", 51 | "processing image 17/939\n", 52 | "processing image 18/939\n", 53 | "processing image 19/939\n", 54 | "processing image 20/939\n", 55 | "processing image 21/939\n", 56 | "processing image 22/939\n", 57 | "processing image 23/939\n", 58 | "processing image 24/939\n", 59 | "processing image 25/939\n", 60 | "processing image 26/939\n", 61 | "processing image 27/939\n", 62 | "processing image 28/939\n", 63 | "processing image 29/939\n", 64 | "processing image 30/939\n", 65 | "processing image 31/939\n", 66 | "processing image 32/939\n", 67 | "processing image 33/939\n", 68 | "processing image 34/939\n", 69 | "processing image 35/939\n", 70 | "processing image 36/939\n", 71 | "processing image 37/939\n", 72 | "processing image 38/939\n", 73 | "processing image 39/939\n", 74 | "processing image 40/939\n", 75 | "processing image 41/939\n", 76 | "processing image 42/939\n", 77 | "processing image 43/939\n", 78 | "processing image 44/939\n", 79 | "processing image 45/939\n", 80 | "processing image 46/939\n", 81 | "processing image 47/939\n", 82 | "processing image 48/939\n", 83 | "processing image 49/939\n", 84 | "processing image 50/939\n", 85 | "processing image 51/939\n", 86 | "processing image 52/939\n", 87 | "processing image 53/939\n", 88 | "processing image 54/939\n", 89 | "processing image 55/939\n", 90 | "processing image 56/939\n", 91 | "processing image 57/939\n", 92 | "processing image 58/939\n", 93 | "processing image 59/939\n", 94 | "processing image 60/939\n", 95 | "processing image 61/939\n", 96 | "processing image 62/939\n", 97 | "processing image 63/939\n", 98 | "processing image 64/939\n", 99 | "processing image 65/939\n", 100 | "processing image 66/939\n", 101 | "processing image 67/939\n", 102 | "processing image 68/939\n", 103 | "processing image 69/939\n", 104 | "processing image 70/939\n", 105 | "processing image 71/939\n", 106 | "processing image 72/939\n", 107 | "processing image 73/939\n", 108 | "processing image 74/939\n", 109 | "processing image 75/939\n", 110 | "processing image 76/939\n", 111 | "processing image 77/939\n", 112 | "processing image 78/939\n", 113 | "processing image 79/939\n", 114 | "processing image 80/939\n", 115 | "processing image 81/939\n", 116 | "processing image 82/939\n", 117 | "processing image 83/939\n", 118 | "processing image 84/939\n", 119 | "processing image 85/939\n", 120 | "processing image 86/939\n", 121 | "processing image 87/939\n", 122 | "processing image 88/939\n", 123 | "processing image 89/939\n", 124 | "processing image 90/939\n", 125 | "processing image 91/939\n", 126 | "processing image 92/939\n", 127 | "processing image 93/939\n", 128 | "processing image 94/939\n", 129 | "processing image 95/939\n", 130 | "processing image 96/939\n", 131 | "processing image 97/939\n", 132 | "processing image 98/939\n", 133 | "processing image 99/939\n", 134 | "processing image 100/939\n", 135 | "processing image 101/939\n", 136 | "processing image 102/939\n", 137 | "processing image 103/939\n", 138 | "processing image 104/939\n", 139 | "processing image 105/939\n", 140 | "processing image 106/939\n", 141 | "processing image 107/939\n", 142 | "processing image 108/939\n", 143 | "processing image 109/939\n", 144 | "processing image 110/939\n", 145 | "processing image 111/939\n", 146 | "processing image 112/939\n", 147 | "processing image 113/939\n", 148 | "processing image 114/939\n", 149 | "processing image 115/939\n", 150 | "processing image 116/939\n", 151 | "processing image 117/939\n", 152 | "processing image 118/939\n", 153 | "processing image 119/939\n", 154 | "processing image 120/939\n", 155 | "processing image 121/939\n", 156 | "processing image 122/939\n", 157 | "processing image 123/939\n", 158 | "processing image 124/939\n", 159 | "processing image 125/939\n", 160 | "processing image 126/939\n", 161 | "processing image 127/939\n", 162 | "processing image 128/939\n", 163 | "processing image 129/939\n", 164 | "processing image 130/939\n", 165 | "processing image 131/939\n", 166 | "processing image 132/939\n", 167 | "processing image 133/939\n", 168 | "processing image 134/939\n", 169 | "processing image 135/939\n", 170 | "processing image 136/939\n", 171 | "processing image 137/939\n", 172 | "processing image 138/939\n", 173 | "processing image 139/939\n", 174 | "processing image 140/939\n", 175 | "processing image 141/939\n", 176 | "processing image 142/939\n", 177 | "processing image 143/939\n", 178 | "processing image 144/939\n", 179 | "processing image 145/939\n", 180 | "processing image 146/939\n", 181 | "processing image 147/939\n", 182 | "processing image 148/939\n", 183 | "processing image 149/939\n", 184 | "processing image 150/939\n", 185 | "processing image 151/939\n", 186 | "processing image 152/939\n", 187 | "processing image 153/939\n", 188 | "processing image 154/939\n", 189 | "processing image 155/939\n", 190 | "processing image 156/939\n", 191 | "processing image 157/939\n", 192 | "processing image 158/939\n", 193 | "processing image 159/939\n", 194 | "processing image 160/939\n", 195 | "processing image 161/939\n", 196 | "processing image 162/939\n", 197 | "processing image 163/939\n", 198 | "processing image 164/939\n", 199 | "processing image 165/939\n", 200 | "processing image 166/939\n", 201 | "processing image 167/939\n", 202 | "processing image 168/939\n", 203 | "processing image 169/939\n", 204 | "processing image 170/939\n", 205 | "processing image 171/939\n", 206 | "processing image 172/939\n", 207 | "processing image 173/939\n", 208 | "processing image 174/939\n", 209 | "processing image 175/939\n", 210 | "processing image 176/939\n", 211 | "processing image 177/939\n", 212 | "processing image 178/939\n", 213 | "processing image 179/939\n", 214 | "processing image 180/939\n", 215 | "processing image 181/939\n", 216 | "processing image 182/939\n", 217 | "processing image 183/939\n", 218 | "processing image 184/939\n", 219 | "processing image 185/939\n", 220 | "processing image 186/939\n", 221 | "processing image 187/939\n", 222 | "processing image 188/939\n", 223 | "processing image 189/939\n", 224 | "processing image 190/939\n", 225 | "processing image 191/939\n", 226 | "processing image 192/939\n", 227 | "processing image 193/939\n", 228 | "processing image 194/939\n", 229 | "processing image 195/939\n", 230 | "processing image 196/939\n", 231 | "processing image 197/939\n", 232 | "processing image 198/939\n", 233 | "processing image 199/939\n", 234 | "processing image 200/939\n", 235 | "processing image 201/939\n", 236 | "processing image 202/939\n", 237 | "processing image 203/939\n", 238 | "processing image 204/939\n", 239 | "processing image 205/939\n", 240 | "processing image 206/939\n", 241 | "processing image 207/939\n", 242 | "processing image 208/939\n", 243 | "processing image 209/939\n", 244 | "processing image 210/939\n", 245 | "processing image 211/939\n", 246 | "processing image 212/939\n", 247 | "processing image 213/939\n", 248 | "processing image 214/939\n", 249 | "processing image 215/939\n", 250 | "processing image 216/939\n", 251 | "processing image 217/939\n", 252 | "processing image 218/939\n", 253 | "processing image 219/939\n", 254 | "processing image 220/939\n", 255 | "processing image 221/939\n", 256 | "processing image 222/939\n", 257 | "processing image 223/939\n", 258 | "processing image 224/939\n", 259 | "processing image 225/939\n", 260 | "processing image 226/939\n", 261 | "processing image 227/939\n", 262 | "processing image 228/939\n", 263 | "processing image 229/939\n", 264 | "processing image 230/939\n", 265 | "processing image 231/939\n", 266 | "processing image 232/939\n", 267 | "processing image 233/939\n", 268 | "processing image 234/939\n", 269 | "processing image 235/939\n", 270 | "processing image 236/939\n", 271 | "processing image 237/939\n", 272 | "processing image 238/939\n", 273 | "processing image 239/939\n", 274 | "processing image 240/939\n", 275 | "processing image 241/939\n", 276 | "processing image 242/939\n", 277 | "processing image 243/939\n", 278 | "processing image 244/939\n", 279 | "processing image 245/939\n", 280 | "processing image 246/939\n", 281 | "processing image 247/939\n", 282 | "processing image 248/939\n", 283 | "processing image 249/939\n", 284 | "processing image 250/939\n", 285 | "processing image 251/939\n", 286 | "processing image 252/939\n", 287 | "processing image 253/939\n", 288 | "processing image 254/939\n", 289 | "processing image 255/939\n", 290 | "processing image 256/939\n", 291 | "processing image 257/939\n", 292 | "processing image 258/939\n", 293 | "processing image 259/939\n", 294 | "processing image 260/939\n", 295 | "processing image 261/939\n", 296 | "processing image 262/939\n", 297 | "processing image 263/939\n", 298 | "processing image 264/939\n", 299 | "processing image 265/939\n", 300 | "processing image 266/939\n", 301 | "processing image 267/939\n", 302 | "processing image 268/939\n", 303 | "processing image 269/939\n", 304 | "processing image 270/939\n", 305 | "processing image 271/939\n", 306 | "processing image 272/939\n", 307 | "processing image 273/939\n", 308 | "processing image 274/939\n", 309 | "processing image 275/939\n", 310 | "processing image 276/939\n", 311 | "processing image 277/939\n", 312 | "processing image 278/939\n", 313 | "processing image 279/939\n", 314 | "processing image 280/939\n", 315 | "processing image 281/939\n", 316 | "processing image 282/939\n", 317 | "processing image 283/939\n", 318 | "processing image 284/939\n", 319 | "processing image 285/939\n", 320 | "processing image 286/939\n", 321 | "processing image 287/939\n", 322 | "processing image 288/939\n", 323 | "processing image 289/939\n", 324 | "processing image 290/939\n", 325 | "processing image 291/939\n", 326 | "processing image 292/939\n", 327 | "processing image 293/939\n", 328 | "processing image 294/939\n", 329 | "processing image 295/939\n", 330 | "processing image 296/939\n", 331 | "processing image 297/939\n", 332 | "processing image 298/939\n", 333 | "processing image 299/939\n", 334 | "processing image 300/939\n", 335 | "processing image 301/939\n", 336 | "processing image 302/939\n", 337 | "processing image 303/939\n", 338 | "processing image 304/939\n", 339 | "processing image 305/939\n", 340 | "processing image 306/939\n", 341 | "processing image 307/939\n", 342 | "processing image 308/939\n", 343 | "processing image 309/939\n", 344 | "processing image 310/939\n", 345 | "processing image 311/939\n", 346 | "processing image 312/939\n", 347 | "processing image 313/939\n", 348 | "processing image 314/939\n", 349 | "processing image 315/939\n", 350 | "processing image 316/939\n", 351 | "processing image 317/939\n", 352 | "processing image 318/939\n", 353 | "processing image 319/939\n", 354 | "processing image 320/939\n", 355 | "processing image 321/939\n", 356 | "processing image 322/939\n", 357 | "processing image 323/939\n", 358 | "processing image 324/939\n", 359 | "processing image 325/939\n", 360 | "processing image 326/939\n", 361 | "processing image 327/939\n", 362 | "processing image 328/939\n", 363 | "processing image 329/939\n", 364 | "processing image 330/939\n", 365 | "processing image 331/939\n", 366 | "processing image 332/939\n", 367 | "processing image 333/939\n", 368 | "processing image 334/939\n", 369 | "processing image 335/939\n", 370 | "processing image 336/939\n", 371 | "processing image 337/939\n", 372 | "processing image 338/939\n", 373 | "processing image 339/939\n", 374 | "processing image 340/939\n", 375 | "processing image 341/939\n", 376 | "processing image 342/939\n", 377 | "processing image 343/939\n", 378 | "processing image 344/939\n", 379 | "processing image 345/939\n", 380 | "processing image 346/939\n", 381 | "processing image 347/939\n", 382 | "processing image 348/939\n", 383 | "processing image 349/939\n", 384 | "processing image 350/939\n", 385 | "processing image 351/939\n", 386 | "processing image 352/939\n", 387 | "processing image 353/939\n", 388 | "processing image 354/939\n", 389 | "processing image 355/939\n", 390 | "processing image 356/939\n", 391 | "processing image 357/939\n", 392 | "processing image 358/939\n", 393 | "processing image 359/939\n" 394 | ] 395 | }, 396 | { 397 | "name": "stdout", 398 | "output_type": "stream", 399 | "text": [ 400 | "processing image 360/939\n", 401 | "processing image 361/939\n", 402 | "processing image 362/939\n", 403 | "processing image 363/939\n", 404 | "processing image 364/939\n", 405 | "processing image 365/939\n", 406 | "processing image 366/939\n", 407 | "processing image 367/939\n", 408 | "processing image 368/939\n", 409 | "processing image 369/939\n", 410 | "processing image 370/939\n", 411 | "processing image 371/939\n", 412 | "processing image 372/939\n", 413 | "processing image 373/939\n", 414 | "processing image 374/939\n", 415 | "processing image 375/939\n", 416 | "processing image 376/939\n", 417 | "processing image 377/939\n", 418 | "processing image 378/939\n", 419 | "processing image 379/939\n", 420 | "processing image 380/939\n", 421 | "processing image 381/939\n", 422 | "processing image 382/939\n", 423 | "processing image 383/939\n", 424 | "processing image 384/939\n", 425 | "processing image 385/939\n", 426 | "processing image 386/939\n", 427 | "processing image 387/939\n", 428 | "processing image 388/939\n", 429 | "processing image 389/939\n", 430 | "processing image 390/939\n", 431 | "processing image 391/939\n", 432 | "processing image 392/939\n", 433 | "processing image 393/939\n", 434 | "processing image 394/939\n", 435 | "processing image 395/939\n", 436 | "processing image 396/939\n", 437 | "processing image 397/939\n", 438 | "processing image 398/939\n", 439 | "processing image 399/939\n", 440 | "processing image 400/939\n", 441 | "processing image 401/939\n", 442 | "processing image 402/939\n", 443 | "processing image 403/939\n", 444 | "processing image 404/939\n", 445 | "processing image 405/939\n", 446 | "processing image 406/939\n", 447 | "processing image 407/939\n", 448 | "processing image 408/939\n", 449 | "processing image 409/939\n", 450 | "processing image 410/939\n", 451 | "processing image 411/939\n", 452 | "processing image 412/939\n", 453 | "processing image 413/939\n", 454 | "processing image 414/939\n", 455 | "processing image 415/939\n", 456 | "processing image 416/939\n", 457 | "processing image 417/939\n", 458 | "processing image 418/939\n", 459 | "processing image 419/939\n", 460 | "processing image 420/939\n", 461 | "processing image 421/939\n", 462 | "processing image 422/939\n", 463 | "processing image 423/939\n", 464 | "processing image 424/939\n", 465 | "processing image 425/939\n", 466 | "processing image 426/939\n", 467 | "processing image 427/939\n", 468 | "processing image 428/939\n", 469 | "processing image 429/939\n", 470 | "processing image 430/939\n", 471 | "processing image 431/939\n", 472 | "processing image 432/939\n", 473 | "processing image 433/939\n", 474 | "processing image 434/939\n", 475 | "processing image 435/939\n", 476 | "processing image 436/939\n", 477 | "processing image 437/939\n", 478 | "processing image 438/939\n", 479 | "processing image 439/939\n", 480 | "processing image 440/939\n", 481 | "processing image 441/939\n", 482 | "processing image 442/939\n", 483 | "processing image 443/939\n", 484 | "processing image 444/939\n", 485 | "processing image 445/939\n", 486 | "processing image 446/939\n", 487 | "processing image 447/939\n", 488 | "processing image 448/939\n", 489 | "processing image 449/939\n", 490 | "processing image 450/939\n", 491 | "processing image 451/939\n", 492 | "processing image 452/939\n", 493 | "processing image 453/939\n", 494 | "processing image 454/939\n", 495 | "processing image 455/939\n", 496 | "processing image 456/939\n", 497 | "processing image 457/939\n", 498 | "processing image 458/939\n", 499 | "processing image 459/939\n", 500 | "processing image 460/939\n", 501 | "processing image 461/939\n", 502 | "processing image 462/939\n", 503 | "processing image 463/939\n", 504 | "processing image 464/939\n", 505 | "processing image 465/939\n", 506 | "processing image 466/939\n", 507 | "processing image 467/939\n", 508 | "processing image 468/939\n", 509 | "processing image 469/939\n", 510 | "processing image 470/939\n", 511 | "processing image 471/939\n", 512 | "processing image 472/939\n", 513 | "processing image 473/939\n", 514 | "processing image 474/939\n", 515 | "processing image 475/939\n", 516 | "processing image 476/939\n", 517 | "processing image 477/939\n", 518 | "processing image 478/939\n", 519 | "processing image 479/939\n", 520 | "processing image 480/939\n", 521 | "processing image 481/939\n", 522 | "processing image 482/939\n", 523 | "processing image 483/939\n", 524 | "processing image 484/939\n", 525 | "processing image 485/939\n", 526 | "processing image 486/939\n", 527 | "processing image 487/939\n", 528 | "processing image 488/939\n", 529 | "processing image 489/939\n", 530 | "processing image 490/939\n", 531 | "processing image 491/939\n", 532 | "processing image 492/939\n", 533 | "processing image 493/939\n", 534 | "processing image 494/939\n", 535 | "processing image 495/939\n", 536 | "processing image 496/939\n", 537 | "processing image 497/939\n", 538 | "processing image 498/939\n", 539 | "processing image 499/939\n", 540 | "processing image 500/939\n", 541 | "processing image 501/939\n", 542 | "processing image 502/939\n", 543 | "processing image 503/939\n", 544 | "processing image 504/939\n", 545 | "processing image 505/939\n", 546 | "processing image 506/939\n", 547 | "processing image 507/939\n", 548 | "processing image 508/939\n", 549 | "processing image 509/939\n", 550 | "processing image 510/939\n", 551 | "processing image 511/939\n", 552 | "processing image 512/939\n", 553 | "processing image 513/939\n", 554 | "processing image 514/939\n", 555 | "processing image 515/939\n", 556 | "processing image 516/939\n", 557 | "processing image 517/939\n", 558 | "processing image 518/939\n", 559 | "processing image 519/939\n", 560 | "processing image 520/939\n", 561 | "processing image 521/939\n", 562 | "processing image 522/939\n", 563 | "processing image 523/939\n", 564 | "processing image 524/939\n", 565 | "processing image 525/939\n", 566 | "processing image 526/939\n", 567 | "processing image 527/939\n", 568 | "processing image 528/939\n", 569 | "processing image 529/939\n", 570 | "processing image 530/939\n", 571 | "processing image 531/939\n", 572 | "processing image 532/939\n", 573 | "processing image 533/939\n", 574 | "processing image 534/939\n", 575 | "processing image 535/939\n", 576 | "processing image 536/939\n", 577 | "processing image 537/939\n", 578 | "processing image 538/939\n", 579 | "processing image 539/939\n", 580 | "processing image 540/939\n", 581 | "processing image 541/939\n", 582 | "processing image 542/939\n", 583 | "processing image 543/939\n", 584 | "processing image 544/939\n", 585 | "processing image 545/939\n", 586 | "processing image 546/939\n", 587 | "processing image 547/939\n", 588 | "processing image 548/939\n", 589 | "processing image 549/939\n", 590 | "processing image 550/939\n", 591 | "processing image 551/939\n", 592 | "processing image 552/939\n", 593 | "processing image 553/939\n", 594 | "processing image 554/939\n", 595 | "processing image 555/939\n", 596 | "processing image 556/939\n", 597 | "processing image 557/939\n", 598 | "processing image 558/939\n", 599 | "processing image 559/939\n", 600 | "processing image 560/939\n", 601 | "processing image 561/939\n", 602 | "processing image 562/939\n", 603 | "processing image 563/939\n", 604 | "processing image 564/939\n", 605 | "processing image 565/939\n", 606 | "processing image 566/939\n", 607 | "processing image 567/939\n", 608 | "processing image 568/939\n", 609 | "processing image 569/939\n", 610 | "processing image 570/939\n", 611 | "processing image 571/939\n", 612 | "processing image 572/939\n", 613 | "processing image 573/939\n", 614 | "processing image 574/939\n", 615 | "processing image 575/939\n", 616 | "processing image 576/939\n", 617 | "processing image 577/939\n", 618 | "processing image 578/939\n", 619 | "processing image 579/939\n", 620 | "processing image 580/939\n", 621 | "processing image 581/939\n", 622 | "processing image 582/939\n", 623 | "processing image 583/939\n", 624 | "processing image 584/939\n", 625 | "processing image 585/939\n", 626 | "processing image 586/939\n", 627 | "processing image 587/939\n", 628 | "processing image 588/939\n", 629 | "processing image 589/939\n", 630 | "processing image 590/939\n", 631 | "processing image 591/939\n", 632 | "processing image 592/939\n", 633 | "processing image 593/939\n", 634 | "processing image 594/939\n", 635 | "processing image 595/939\n", 636 | "processing image 596/939\n", 637 | "processing image 597/939\n", 638 | "processing image 598/939\n", 639 | "processing image 599/939\n", 640 | "processing image 600/939\n", 641 | "processing image 601/939\n", 642 | "processing image 602/939\n", 643 | "processing image 603/939\n", 644 | "processing image 604/939\n", 645 | "processing image 605/939\n", 646 | "processing image 606/939\n", 647 | "processing image 607/939\n", 648 | "processing image 608/939\n", 649 | "processing image 609/939\n", 650 | "processing image 610/939\n", 651 | "processing image 611/939\n", 652 | "processing image 612/939\n", 653 | "processing image 613/939\n", 654 | "processing image 614/939\n", 655 | "processing image 615/939\n", 656 | "processing image 616/939\n", 657 | "processing image 617/939\n", 658 | "processing image 618/939\n", 659 | "processing image 619/939\n", 660 | "processing image 620/939\n", 661 | "processing image 621/939\n", 662 | "processing image 622/939\n", 663 | "processing image 623/939\n", 664 | "processing image 624/939\n", 665 | "processing image 625/939\n", 666 | "processing image 626/939\n", 667 | "processing image 627/939\n", 668 | "processing image 628/939\n", 669 | "processing image 629/939\n", 670 | "processing image 630/939\n", 671 | "processing image 631/939\n", 672 | "processing image 632/939\n", 673 | "processing image 633/939\n", 674 | "processing image 634/939\n", 675 | "processing image 635/939\n", 676 | "processing image 636/939\n", 677 | "processing image 637/939\n", 678 | "processing image 638/939\n", 679 | "processing image 639/939\n", 680 | "processing image 640/939\n", 681 | "processing image 641/939\n", 682 | "processing image 642/939\n", 683 | "processing image 643/939\n", 684 | "processing image 644/939\n", 685 | "processing image 645/939\n", 686 | "processing image 646/939\n", 687 | "processing image 647/939\n", 688 | "processing image 648/939\n", 689 | "processing image 649/939\n", 690 | "processing image 650/939\n", 691 | "processing image 651/939\n", 692 | "processing image 652/939\n", 693 | "processing image 653/939\n", 694 | "processing image 654/939\n", 695 | "processing image 655/939\n", 696 | "processing image 656/939\n", 697 | "processing image 657/939\n", 698 | "processing image 658/939\n", 699 | "processing image 659/939\n", 700 | "processing image 660/939\n", 701 | "processing image 661/939\n", 702 | "processing image 662/939\n", 703 | "processing image 663/939\n", 704 | "processing image 664/939\n", 705 | "processing image 665/939\n", 706 | "processing image 666/939\n", 707 | "processing image 667/939\n", 708 | "processing image 668/939\n", 709 | "processing image 669/939\n", 710 | "processing image 670/939\n", 711 | "processing image 671/939\n", 712 | "processing image 672/939\n", 713 | "processing image 673/939\n", 714 | "processing image 674/939\n", 715 | "processing image 675/939\n", 716 | "processing image 676/939\n", 717 | "processing image 677/939\n", 718 | "processing image 678/939\n", 719 | "processing image 679/939\n", 720 | "processing image 680/939\n", 721 | "processing image 681/939\n", 722 | "processing image 682/939\n", 723 | "processing image 683/939\n", 724 | "processing image 684/939\n", 725 | "processing image 685/939\n", 726 | "processing image 686/939\n", 727 | "processing image 687/939\n", 728 | "processing image 688/939\n", 729 | "processing image 689/939\n", 730 | "processing image 690/939\n", 731 | "processing image 691/939\n", 732 | "processing image 692/939\n", 733 | "processing image 693/939\n", 734 | "processing image 694/939\n", 735 | "processing image 695/939\n", 736 | "processing image 696/939\n", 737 | "processing image 697/939\n", 738 | "processing image 698/939\n", 739 | "processing image 699/939\n", 740 | "processing image 700/939\n", 741 | "processing image 701/939\n", 742 | "processing image 702/939\n", 743 | "processing image 703/939\n", 744 | "processing image 704/939\n", 745 | "processing image 705/939\n", 746 | "processing image 706/939\n", 747 | "processing image 707/939\n", 748 | "processing image 708/939\n", 749 | "processing image 709/939\n", 750 | "processing image 710/939\n", 751 | "processing image 711/939\n", 752 | "processing image 712/939\n", 753 | "processing image 713/939\n", 754 | "processing image 714/939\n", 755 | "processing image 715/939\n", 756 | "processing image 716/939\n", 757 | "processing image 717/939\n", 758 | "processing image 718/939\n", 759 | "processing image 719/939\n", 760 | "processing image 720/939\n", 761 | "processing image 721/939\n", 762 | "processing image 722/939\n", 763 | "processing image 723/939\n", 764 | "processing image 724/939\n", 765 | "processing image 725/939\n", 766 | "processing image 726/939\n", 767 | "processing image 727/939\n", 768 | "processing image 728/939\n", 769 | "processing image 729/939\n", 770 | "processing image 730/939\n", 771 | "processing image 731/939\n", 772 | "processing image 732/939\n", 773 | "processing image 733/939\n", 774 | "processing image 734/939\n", 775 | "processing image 735/939\n" 776 | ] 777 | }, 778 | { 779 | "name": "stdout", 780 | "output_type": "stream", 781 | "text": [ 782 | "processing image 736/939\n", 783 | "processing image 737/939\n", 784 | "processing image 738/939\n", 785 | "processing image 739/939\n", 786 | "processing image 740/939\n", 787 | "processing image 741/939\n", 788 | "processing image 742/939\n", 789 | "processing image 743/939\n", 790 | "processing image 744/939\n", 791 | "processing image 745/939\n", 792 | "processing image 746/939\n", 793 | "processing image 747/939\n", 794 | "processing image 748/939\n", 795 | "processing image 749/939\n", 796 | "processing image 750/939\n", 797 | "processing image 751/939\n", 798 | "processing image 752/939\n", 799 | "processing image 753/939\n", 800 | "processing image 754/939\n", 801 | "processing image 755/939\n", 802 | "processing image 756/939\n", 803 | "processing image 757/939\n", 804 | "processing image 758/939\n", 805 | "processing image 759/939\n", 806 | "processing image 760/939\n", 807 | "processing image 761/939\n", 808 | "processing image 762/939\n", 809 | "processing image 763/939\n", 810 | "processing image 764/939\n", 811 | "processing image 765/939\n", 812 | "processing image 766/939\n", 813 | "processing image 767/939\n", 814 | "processing image 768/939\n", 815 | "processing image 769/939\n", 816 | "processing image 770/939\n", 817 | "processing image 771/939\n", 818 | "processing image 772/939\n", 819 | "processing image 773/939\n", 820 | "processing image 774/939\n", 821 | "processing image 775/939\n", 822 | "processing image 776/939\n", 823 | "processing image 777/939\n", 824 | "processing image 778/939\n", 825 | "processing image 779/939\n", 826 | "processing image 780/939\n", 827 | "processing image 781/939\n", 828 | "processing image 782/939\n", 829 | "processing image 783/939\n", 830 | "processing image 784/939\n", 831 | "processing image 785/939\n", 832 | "processing image 786/939\n", 833 | "processing image 787/939\n", 834 | "processing image 788/939\n", 835 | "processing image 789/939\n", 836 | "processing image 790/939\n", 837 | "processing image 791/939\n", 838 | "processing image 792/939\n", 839 | "processing image 793/939\n", 840 | "processing image 794/939\n", 841 | "processing image 795/939\n", 842 | "processing image 796/939\n", 843 | "processing image 797/939\n", 844 | "processing image 798/939\n", 845 | "processing image 799/939\n", 846 | "processing image 800/939\n", 847 | "processing image 801/939\n", 848 | "processing image 802/939\n", 849 | "processing image 803/939\n", 850 | "processing image 804/939\n", 851 | "processing image 805/939\n", 852 | "processing image 806/939\n", 853 | "processing image 807/939\n", 854 | "processing image 808/939\n", 855 | "processing image 809/939\n", 856 | "processing image 810/939\n", 857 | "processing image 811/939\n", 858 | "processing image 812/939\n", 859 | "processing image 813/939\n", 860 | "processing image 814/939\n", 861 | "processing image 815/939\n", 862 | "processing image 816/939\n", 863 | "processing image 817/939\n", 864 | "processing image 818/939\n", 865 | "processing image 819/939\n", 866 | "processing image 820/939\n", 867 | "processing image 821/939\n", 868 | "processing image 822/939\n", 869 | "processing image 823/939\n", 870 | "processing image 824/939\n", 871 | "processing image 825/939\n", 872 | "processing image 826/939\n", 873 | "processing image 827/939\n", 874 | "processing image 828/939\n", 875 | "processing image 829/939\n", 876 | "processing image 830/939\n", 877 | "processing image 831/939\n", 878 | "processing image 832/939\n", 879 | "processing image 833/939\n", 880 | "processing image 834/939\n", 881 | "processing image 835/939\n", 882 | "processing image 836/939\n", 883 | "processing image 837/939\n", 884 | "processing image 838/939\n", 885 | "processing image 839/939\n", 886 | "processing image 840/939\n", 887 | "processing image 841/939\n", 888 | "processing image 842/939\n", 889 | "processing image 843/939\n", 890 | "processing image 844/939\n", 891 | "processing image 845/939\n", 892 | "processing image 846/939\n", 893 | "processing image 847/939\n", 894 | "processing image 848/939\n", 895 | "processing image 849/939\n", 896 | "processing image 850/939\n", 897 | "processing image 851/939\n", 898 | "processing image 852/939\n", 899 | "processing image 853/939\n", 900 | "processing image 854/939\n", 901 | "processing image 855/939\n", 902 | "processing image 856/939\n", 903 | "processing image 857/939\n", 904 | "processing image 858/939\n", 905 | "processing image 859/939\n", 906 | "processing image 860/939\n", 907 | "processing image 861/939\n", 908 | "processing image 862/939\n", 909 | "processing image 863/939\n", 910 | "processing image 864/939\n", 911 | "processing image 865/939\n", 912 | "processing image 866/939\n", 913 | "processing image 867/939\n", 914 | "processing image 868/939\n", 915 | "processing image 869/939\n", 916 | "processing image 870/939\n", 917 | "processing image 871/939\n", 918 | "processing image 872/939\n", 919 | "processing image 873/939\n", 920 | "processing image 874/939\n", 921 | "processing image 875/939\n", 922 | "processing image 876/939\n", 923 | "processing image 877/939\n", 924 | "processing image 878/939\n", 925 | "processing image 879/939\n", 926 | "processing image 880/939\n", 927 | "processing image 881/939\n", 928 | "processing image 882/939\n", 929 | "processing image 883/939\n", 930 | "processing image 884/939\n", 931 | "processing image 885/939\n", 932 | "processing image 886/939\n", 933 | "processing image 887/939\n", 934 | "processing image 888/939\n", 935 | "processing image 889/939\n", 936 | "processing image 890/939\n", 937 | "processing image 891/939\n", 938 | "processing image 892/939\n", 939 | "processing image 893/939\n", 940 | "processing image 894/939\n", 941 | "processing image 895/939\n", 942 | "processing image 896/939\n", 943 | "processing image 897/939\n", 944 | "processing image 898/939\n", 945 | "processing image 899/939\n", 946 | "processing image 900/939\n", 947 | "processing image 901/939\n", 948 | "processing image 902/939\n", 949 | "processing image 903/939\n", 950 | "processing image 904/939\n", 951 | "processing image 905/939\n", 952 | "processing image 906/939\n", 953 | "processing image 907/939\n", 954 | "processing image 908/939\n", 955 | "processing image 909/939\n", 956 | "processing image 910/939\n", 957 | "processing image 911/939\n", 958 | "processing image 912/939\n", 959 | "processing image 913/939\n", 960 | "processing image 914/939\n", 961 | "processing image 915/939\n", 962 | "processing image 916/939\n", 963 | "processing image 917/939\n", 964 | "processing image 918/939\n", 965 | "processing image 919/939\n", 966 | "processing image 920/939\n", 967 | "processing image 921/939\n", 968 | "processing image 922/939\n", 969 | "processing image 923/939\n", 970 | "processing image 924/939\n", 971 | "processing image 925/939\n", 972 | "processing image 926/939\n", 973 | "processing image 927/939\n", 974 | "processing image 928/939\n", 975 | "processing image 929/939\n", 976 | "processing image 930/939\n", 977 | "processing image 931/939\n", 978 | "processing image 932/939\n", 979 | "processing image 933/939\n", 980 | "processing image 934/939\n", 981 | "processing image 935/939\n", 982 | "processing image 936/939\n", 983 | "processing image 937/939\n", 984 | "processing image 938/939\n", 985 | "processing image 939/939\n" 986 | ] 987 | } 988 | ], 989 | "source": [ 990 | "SOLVED_CAPTCHA_FOLDER = \"solved-captchas\"\n", 991 | "OUTPUT_FOLDER = \"extracted_letters\"\n", 992 | "\n", 993 | "\n", 994 | "# Get the path of all the solved captcha images\n", 995 | "solved_captchas = glob.glob(os.path.join(SOLVED_CAPTCHA_FOLDER, \"*\"))\n", 996 | "counts = {}\n", 997 | "\n", 998 | "# loop over the image paths\n", 999 | "for (i, captcha) in enumerate(solved_captchas):\n", 1000 | " print(\"processing image {}/{}\".format(i + 1, len(solved_captchas)))\n", 1001 | "\n", 1002 | " \n", 1003 | " # grab the base filename as the text\n", 1004 | " filename = os.path.basename(captcha)\n", 1005 | " captcha_text = os.path.splitext(filename)[0]\n", 1006 | "\n", 1007 | " # Load the image and convert it to grayscale\n", 1008 | " image = cv2.imread(captcha)\n", 1009 | " gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n", 1010 | " \n", 1011 | " # Add some extra padding around the image\n", 1012 | " gray = cv2.copyMakeBorder(gray, 8, 8, 8, 8, cv2.BORDER_REPLICATE)\n", 1013 | "\n", 1014 | " # applying threshold\n", 1015 | " thresh = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY_INV, cv2.THRESH_OTSU)[1]\n", 1016 | "\n", 1017 | " # finding the contours\n", 1018 | " contours, hierarchy = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)\n", 1019 | "\n", 1020 | " # creating empty list for holding the coordinates of the letters\n", 1021 | " letter_image_regions = []\n", 1022 | "\n", 1023 | " # Now we will loop through each of the contours and extract the letter\n", 1024 | " for contour in contours:\n", 1025 | " # Get the rectangle that contains the contour\n", 1026 | " (x, y, w, h) = cv2.boundingRect(contour)\n", 1027 | " \n", 1028 | " # checking if any counter is too wide\n", 1029 | " # if countour is too wide then there could be two letters joined together or are very close to each other\n", 1030 | " if w / h > 1.25:\n", 1031 | " # Split it in half into two letter regions\n", 1032 | " half_width = int(w / 2)\n", 1033 | " letter_image_regions.append((x, y, half_width, h))\n", 1034 | " letter_image_regions.append((x + half_width, y, half_width, h))\n", 1035 | " else:\n", 1036 | " \n", 1037 | " letter_image_regions.append((x, y, w, h))\n", 1038 | " \n", 1039 | "\n", 1040 | " \n", 1041 | " # Sort the detected letter images based on the x coordinate to make sure\n", 1042 | " # we get them from left-to-right so that we match the right image with the right letter\n", 1043 | " letter_image_regions = sorted(letter_image_regions, key=lambda x: x[0])\n", 1044 | " \n", 1045 | " # Save each letter as a single image\n", 1046 | " for letter_bounding_box, letter_text in zip(letter_image_regions, captcha_text):\n", 1047 | " # Grab the coordinates of the letter in the image\n", 1048 | " x, y, w, h = letter_bounding_box\n", 1049 | "\n", 1050 | " # Extract the letter from the original image with a 2-pixel margin around the edge\n", 1051 | " letter_image = gray[y - 2:y + h + 2, x - 2:x + w + 2]\n", 1052 | "\n", 1053 | "\n", 1054 | " # Get the folder to save the image in\n", 1055 | " save_path = os.path.join(OUTPUT_FOLDER, letter_text)\n", 1056 | "\n", 1057 | " # creating different output folder for storing different letters\n", 1058 | " if not os.path.exists(save_path):\n", 1059 | " os.makedirs(save_path)\n", 1060 | "\n", 1061 | " # write the letter image to a file\n", 1062 | " count = counts.get(letter_text, 1)\n", 1063 | " p = os.path.join(save_path, \"{}.png\".format(str(count)))\n", 1064 | " cv2.imwrite(p, letter_image)\n", 1065 | "\n", 1066 | " # increment the count\n", 1067 | " counts[letter_text] = count + 1\n" 1068 | ] 1069 | }, 1070 | { 1071 | "cell_type": "markdown", 1072 | "metadata": {}, 1073 | "source": [ 1074 | "## Preparing training data" 1075 | ] 1076 | }, 1077 | { 1078 | "cell_type": "code", 1079 | "execution_count": 3, 1080 | "metadata": {}, 1081 | "outputs": [ 1082 | { 1083 | "name": "stdout", 1084 | "output_type": "stream", 1085 | "text": [ 1086 | "(4695, 30, 30, 1) (4695,)\n" 1087 | ] 1088 | } 1089 | ], 1090 | "source": [ 1091 | "letter_folder = 'extracted_letters'\n", 1092 | "\n", 1093 | "#creating empty lists for storing image data and labels\n", 1094 | "data = []\n", 1095 | "labels = []\n", 1096 | "for image in paths.list_images(letter_folder):\n", 1097 | " img = cv2.imread(image)\n", 1098 | " img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n", 1099 | " img = cv2.resize(img, (30,30))\n", 1100 | " \n", 1101 | " # adding a 3rd dimension to the image\n", 1102 | " img = np.expand_dims(img, axis = 2)\n", 1103 | " \n", 1104 | " #grabing the name of the letter based on the folder it is present in\n", 1105 | " label = image.split(os.path.sep)[-2]\n", 1106 | " \n", 1107 | " # appending to the empty lists\n", 1108 | " data.append(img)\n", 1109 | " labels.append(label)\n", 1110 | "\n", 1111 | "#converting data and labels to np array\n", 1112 | "data = np.array(data, dtype = \"float\")\n", 1113 | "labels = np.array(labels)\n", 1114 | "\n", 1115 | "print(data.shape, labels.shape)\n", 1116 | " " 1117 | ] 1118 | }, 1119 | { 1120 | "cell_type": "code", 1121 | "execution_count": 4, 1122 | "metadata": {}, 1123 | "outputs": [], 1124 | "source": [ 1125 | "#scaling the values of data between 0 and 1\n", 1126 | "data = data/255.0" 1127 | ] 1128 | }, 1129 | { 1130 | "cell_type": "code", 1131 | "execution_count": 5, 1132 | "metadata": {}, 1133 | "outputs": [ 1134 | { 1135 | "name": "stdout", 1136 | "output_type": "stream", 1137 | "text": [ 1138 | "(3756, 30, 30, 1) (939, 30, 30, 1) (3756,) (939,)\n" 1139 | ] 1140 | } 1141 | ], 1142 | "source": [ 1143 | "# Split the training data into separate train and test sets\n", 1144 | "from sklearn.model_selection import train_test_split\n", 1145 | "(train_x, val_x, train_y, val_y) = train_test_split(data, labels, test_size=0.2, random_state=0)\n", 1146 | "print(train_x.shape, val_x.shape, train_y.shape, val_y.shape)" 1147 | ] 1148 | }, 1149 | { 1150 | "cell_type": "code", 1151 | "execution_count": 8, 1152 | "metadata": { 1153 | "scrolled": true 1154 | }, 1155 | "outputs": [ 1156 | { 1157 | "name": "stdout", 1158 | "output_type": "stream", 1159 | "text": [ 1160 | "(3756, 9) (939, 9)\n" 1161 | ] 1162 | } 1163 | ], 1164 | "source": [ 1165 | "# one hot encoding our target variable 'labels'\n", 1166 | "from sklearn.preprocessing import LabelBinarizer\n", 1167 | "import pickle\n", 1168 | "lb = LabelBinarizer().fit(train_y)\n", 1169 | "train_y = lb.transform(train_y)\n", 1170 | "val_y = lb.transform(val_y)\n", 1171 | "\n", 1172 | "bin = pickle.dumps(lb)\n", 1173 | "with open(\"captcha_labels.pickle\", \"wb\") as f:\n", 1174 | " pickle.dump(lb, f)\n", 1175 | "\n", 1176 | "print(train_y.shape, val_y.shape)" 1177 | ] 1178 | }, 1179 | { 1180 | "cell_type": "code", 1181 | "execution_count": 7, 1182 | "metadata": {}, 1183 | "outputs": [ 1184 | { 1185 | "data": { 1186 | "text/plain": [ 1187 | "b'\\x80\\x03csklearn.preprocessing.label\\nLabelBinarizer\\nq\\x00)\\x81q\\x01}q\\x02(X\\t\\x00\\x00\\x00neg_labelq\\x03K\\x00X\\t\\x00\\x00\\x00pos_labelq\\x04K\\x01X\\r\\x00\\x00\\x00sparse_outputq\\x05\\x89X\\x07\\x00\\x00\\x00y_type_q\\x06X\\n\\x00\\x00\\x00multiclassq\\x07X\\r\\x00\\x00\\x00sparse_input_q\\x08\\x89X\\x08\\x00\\x00\\x00classes_q\\tcnumpy.core.multiarray\\n_reconstruct\\nq\\ncnumpy\\nndarray\\nq\\x0bK\\x00\\x85q\\x0cC\\x01bq\\r\\x87q\\x0eRq\\x0f(K\\x01K\\t\\x85q\\x10cnumpy\\ndtype\\nq\\x11X\\x02\\x00\\x00\\x00U1q\\x12K\\x00K\\x01\\x87q\\x13Rq\\x14(K\\x03X\\x01\\x00\\x00\\x00" 1330 | ] 1331 | }, 1332 | "execution_count": 10, 1333 | "metadata": {}, 1334 | "output_type": "execute_result" 1335 | } 1336 | ], 1337 | "source": [ 1338 | "model.fit(train_x, train_y, validation_data=(val_x, val_y), batch_size=32, epochs=50, verbose=1, callbacks = [estop])" 1339 | ] 1340 | }, 1341 | { 1342 | "cell_type": "markdown", 1343 | "metadata": {}, 1344 | "source": [ 1345 | "### Testing model on an unseen captcha" 1346 | ] 1347 | }, 1348 | { 1349 | "cell_type": "code", 1350 | "execution_count": 11, 1351 | "metadata": {}, 1352 | "outputs": [ 1353 | { 1354 | "name": "stdout", 1355 | "output_type": "stream", 1356 | "text": [ 1357 | "CAPTCHA text is: 66372\n" 1358 | ] 1359 | }, 1360 | { 1361 | "data": { 1362 | "text/plain": [ 1363 | "" 1364 | ] 1365 | }, 1366 | "execution_count": 11, 1367 | "metadata": {}, 1368 | "output_type": "execute_result" 1369 | }, 1370 | { 1371 | "data": { 1372 | "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAACnCAYAAAABvqdmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAE5VJREFUeJzt3X2wXHV9x/H3xwSwiFxCJCFNmAZruJU6BTQjD+lAigIRmARmpEIdSKeZyTi204B0IMDIiKJCi0LrWOUKQmBSAo0GMhmQRh6CKBOTgEJCIIEIGkkJIISnsRr49o89Z7PZu/fu856zJ58Xk9nztPd893f3fvme3zm/cxQRmJlZ/3tP1gGYmVlnOKGbmRWEE7qZWUE4oZuZFYQTuplZQTihm5kVhBO6mVlBtJXQJc2S9LSkZyQt7FRQZmbWPLU6sEjSGGATcBKwFVgDnBMRT3YuPDMza9TYNt77ceCZiNgCIGkJMAcYMaEPDAzEwQcf3MYu+9um929q6/2HvXFYhyIpnnbbtha3t+XFpk2bXo6Ig+pt105Cnwz8pmJ+K3B09UaS5gPzASZOnMjQ0FAbu+xPM0+Y2ZGfM7Rqz2u7Vri9rWhmzpz5fCPbtdOHrhrLhvXfRMRQREyPiOkDAwNt7M7MzEbTTkLfChxSMT8FeKG9cMzMrFXtdLmsAaZJOhT4LXA28HcdiWoP9OCqB4HRuwsq16Xb23CjtU0j3TFuW+tXLSf0iNgp6Z+Ae4ExwPcjYkPHIjMzs6a0U6ETEXcDd3colj1SdTVYOV9dTRapcszbZ8t6/2ad4JGiZmYF0VaFbqNrpL+2yH26zVw+WGvbTn3uZn8P/dreZq7QzcwKwgndzKwg3OXSB9LugH7pCujUSM2stNrenfrcWf2ei9z9t6dwhW5mVhCu0PtIkU7cNTOQqt3POtqloO3qxtFIL4/Imo2/344W9zSu0M3MCsIVesbaHabeT2p91tEq9W5UgdU/s9027mb1303txlqko8UicYVuZlYQrtAzUrSqplODqKB/26aZuLOq5lu9kqUX5zqsfa7QzcwKwgndzKwg3OViudPKIXyvTqr2QlZxj7bfRi4ztezVrdAlfV/SdknrK5YdKGmlpM3J67juhmlmZvUoYthjQHffQDoeeBO4JSI+kiz7V+B3EXGVpIXAuIi4uN7OBgcHo1sPiT7hhBO68nM7QTUfv1oSwx/D2vb7sjBarO3qZhtl1cZZ/27T/Te6r0Z+v3n7TmZl1apVHf+ZM2fOXBcR0+ttV7dCj4iHgN9VLZ4DLEqmFwFnNB2hmZl1VKt96BMjYhtARGyTNGGkDSXNB+YDTJw4scXdFVc3K9u8abXSblUe2zbryrwb+3Jlnh9dv8olIoYiYnpETB8YGOj27szM9litJvQXJU0CSF63dy4kMzNrRasJfTkwN5meC9zVmXCKKZL/OvVz+u0Qt99i7rd4u0UV/1l/aOSyxduAR4BBSVslzQOuAk6StBk4KZk3M7MM1T0pGhHnjLDqEx2OpfBqVX15OUnWr5pt00be32n99jv2JYr9y0P/zcwKwkP/M1aUSqfyc1RXeM0OYulkLNY4V+b9zxW6mVlBOKGbmRWEu1x6KOvLv7I+XO7V589LO/fDydBG2yov8droXKGbmRWEK/SM9LLi6XXF2kiF2szPyer9zcr6yKAZPgFaTK7QzcwKwhV6TrzyyisAfOUrXykvu+222wB49dVXAfjYxz5WXvfFL34RgFNPPbVXITatusLLqk+58v7U3/jGNwC4//77AfjDH/4AwPjx48vbzJ49G4Avf/nLQPN3Cc1z33kzRxHuX+8/rtDNzArCCd3MrCDc5ZKxt99+G9j1CL233nqrvO7ee+8F4MgjjwTgySefLK9Lu2by3OVSLatD87SbBeDCCy8E4M477wTgPe8p1TTr1q0rbzNjxgwA1qxZA8Cjjz7akzj7Va9HAtvIXKGbmRWEK/SMpdXjhg0bALjjjjvK69LKPHX44YeXp9MTpnmWl8v4li9fXnebI444ojy9c+dOALZs2dLS/qov28xT5dqpS0pr/UzLXiP3Qz9E0gOSNkraIGlBsvxASSslbU5ex3U/XDMzG0kjFfpO4MKIeFTS+4F1klYCfw/cFxFXSVoILAQu7l6oxbRs2bLd5ufMmZNRJN2TxwE+7777LgDr168H4Morryyv23vvvQG4/vrr24ojz5VrnmOz1tWt0CNiW0Q8mky/AWwEJgNzgEXJZouAM7oVpJmZ1dfUSVFJU4GjgNXAxIjYBqWkD0wY4T3zJa2VtHbHjh3tRWtmZiNq+KSopP2AHwDnR8TrUoOjyCKGgCGAwcFBH+dVeeqpp3abv/XWW8vT3/rWtwDYtGkTsPslit/73vcAGDeuP09dVI+MrTzJWz0yNh0VC+1fpjnS9/b4448vT2/evBmAyZMnt7aPnNztsRFZxeoun+5oqEKXtBelZL44In6YLH5R0qRk/SRge3dCNDOzRtSt0FUqaW4ENkbENytWLQfmAlclr3d1JcKCS+8lknrooYfK0w8//PBuy0477bRh71+6dGkXo+usdBAVDB9IlQ6iguEDqSrvb9NuhR5RqgzTk6KrV68G4KKLLipvc9xxxwGwZMkSAI499tjW9pXDk8GjyfqB2da+RrpcZgDnAk9I+kWy7FJKifwOSfOAXwNndSdEMzNrhNKKpRcGBwdjaGioKz87rfjyrLI6Sauhgw46CICXX34ZgHfeeae8TTosPa0mx4wZU163zz77APD73/++4f1mVTGm+62stC+//HJg10Cqs87qXD3Qyud96aWXytMTJpTO70+bNg3YdQ6j0f2myoN4GjzfNJJG/0Zb+dy1vpMrVqwAdr9lwk9/+lMA9t13XwDOO++88rqvfe1rAOy3335dibHfVN7ds1Nmzpy5LiKm19vOQ//NzArCQ/8zlvYJ33LLLQC8+eab5XX7778/sKtSrzQwMNCD6DqrehAV5GcgVa32TI+a2tVohZ0ebaVHBq323bcrveKocmDVhz70IQCuvvpqAC699NLyujfeeAOAm266qVch2ghcoZuZFYQTuplZQbjLJWMXX1y6/U16mPulL32pvO6aa64Bdj0urdLnP//57gfXYdWDqGDXQKp0EBUMH0iVDqKC1gZSnX766eXptL3Te56no5fPP//8Ye+rjKkX0u6M9ATttdde29P9pxYvXjziurT9Krtc0u+uu1yy5wrdzKwgXKFnLL3H+cqVKwG44IILyuvSu/6lDylOH1oMuw+H7xfVg6hg16CpdBBV5bJODaT63Oc+V55OL5d85JFHdtvm6KOPLk/fc889AMyaNavpfTXr+eefL09//etfB3ZVwa3eeqCbav0O0++pZc8VuplZQbhCz4l0YFSRn19Z2f+dXhKY9rtWXppZXRmnA11aVdmHXjmdB5VHZOmgnfS5p3l08803D1tWOcjIsuUK3cysIJzQzcwKwl0u1jOVd0qsHhmbjoqF4SNj+3FUbD0/+clPgN1Hz6aXrFa2RV6k96j/6le/CsDUqVPL66644oosQrIaXKGbmRWEK3TrmfRyPBg+kCodRAXDB1L14yCqehYsWADAXnvtVV6Wt89ZeSfPc889F9h19FR53/7x48f3NjAbUd0KXdJ7Jf1c0i8lbZB0RbL8UEmrJW2WdLskX4xqZpahRir0/wNOjIg3k0fRPSzpHuALwLURsUTSd4F5wHe6GKv1uXQQFQwfSFU5OKV6IFU/DqIayd133w3AY489BsCZZ55ZXpfeGz9r6XmN9MlNAIcddhgAzz77LABjx/rgPo/qVuhRkt7Tda/kXwAnAumwvUXAGV2J0MzMGtLoQ6LHJI+f2w6sBJ4FXouInckmW4Ga45QlzZe0VtLa9EZIZmbWeQ0dN0XEO8CRkg4AlgEfrrXZCO8dAoag9Ai6FuMsnD39Ybm9GBmbxza+4YYbdpv/zGc+k1EkIzvllFMAeOKJJ8rL0vvbuKsl35q6bDEiXgMeBI4BDpCU/nanAC90NjQzM2tG3f/dSjoI+GNEvCbpT4BPAlcDDwCfBpYAc4G7uhloERT5wbh5lFV7j3Zk8OMf/3i3+ZNPPrnb4TTtZz/72bBlU6ZMqfu+Xj5w3mpr5PhpErBI0hhKFf0dEbFC0pPAEklXAo8BN3YxTjMzq6NuQo+Ix4GjaizfAny8G0GZFdXrr7+edQh1udLuXx76b2ZWEE7oZmYF4YRuZlYQTuhmZgXhUQLWdXkc4GNWRK7QzcwKwhW6dY0HUpn1lit0M7OCcIVu1iX9du6g3+K14Vyhm5kVhBO6mVlBuMvFrMP66WRwP8Vq9blCNzMrCCd0M7OCaDihJ88VfUzSimT+UEmrJW2WdLukvev9DDMz655mKvQFwMaK+auBayNiGvAqMK+TgZmZWXMaSuiSpgCnATck8wJOBJYmmywCzuhGgGZm1phGK/TrgIuAd5P58cBrEbEzmd8KTK71RknzJa2VtHbHjh1tBWtmZiOrm9AlnQ5sj4h1lYtrbFrz+qeIGIqI6RExfWBgoMUwzcysnkauQ58BzJZ0KvBeYH9KFfsBksYmVfoU4IXuhWlmZvXUrdAj4pKImBIRU4Gzgfsj4rPAA8Cnk83mAnd1LUozM6urnevQLwa+IOkZSn3qN3YmJDMza0VTQ/8j4kHgwWR6C/DxzodkZmat8EhRM7OCcEI3MysIJ3Qzs4JwQjczKwgndDOzgnBCNzMrCCd0M7OCcEI3MysIJ3Qzs4JwQjczKwgndDOzgnBCNzMrCCd0M7OCaOpui3m2atWqrEMwM8tUQwld0nPAG8A7wM6ImC7pQOB2YCrwHPC3EfFqd8I0M7N6muly+ZuIODIipifzC4H7ImIacF8yb2ZmGWmnD30OsCiZXgSc0X44ZmbWqkYTegD/I2mdpPnJsokRsQ0geZ1Q642S5ktaK2ntjh072o/YzMxqavSk6IyIeEHSBGClpKca3UFEDAFDAIODg9FCjGZm1oCGKvSIeCF53Q4so/Qs0RclTQJIXrd3K0gzM6uvbkKX9D5J70+ngZOB9cByYG6y2Vzgrm4FaWZm9TXS5TIRWCYp3f6/IuJHktYAd0iaB/waOKt7YZqZWT11E3pEbAGOqLH8FeAT3QjKzMya56H/ZmYF4YRuZlYQTuhmZgXhhG5mVhBO6GZmBeGEbmZWEE7oZmYF4YRuZlYQTuhmZgXhhG5mVhBO6GZmBaGI3t2iXNJLwFvAyz3baed8AMfdS/0Ydz/GDI6711qJ+88i4qB6G/U0oQNIWlvxXNK+4bh7qx/j7seYwXH3WjfjdpeLmVlBOKGbmRVEFgl9KIN9doLj7q1+jLsfYwbH3Wtdi7vnfehmZtYd7nIxMysIJ3Qzs4LoWUKXNEvS05KekbSwV/ttlqRDJD0gaaOkDZIWJMsPlLRS0ubkdVzWsdYiaYykxyStSOYPlbQ6ift2SXtnHWM1SQdIWirpqaTdj+2H9pZ0QfIdWS/pNknvzWN7S/q+pO2S1lcsq9m+KvmP5O/0cUkfzVnc/5Z8Tx6XtEzSARXrLkniflrSKdlEXTvuinX/IikkfSCZ72h79yShSxoDfBv4FHA4cI6kw3ux7xbsBC6MiA8DxwD/mMS6ELgvIqYB9yXzebQA2FgxfzVwbRL3q8C8TKIa3b8DP4qIv6D0QPKN5Ly9JU0G/hmYHhEfAcYAZ5PP9r4ZmFW1bKT2/RQwLfk3H/hOj2Ks5WaGx70S+EhE/BWwCbgEIPkbPRv4y+Q9/5nknSzczPC4kXQIcBLw64rFnW3viOj6P+BY4N6K+UuAS3qx7w7EflfyS3gamJQsmwQ8nXVsNWKdQumP80RgBSBKI9LG1vo95OEfsD/wK5IT9BXLc93ewGTgN8CBwNikvU/Ja3sDU4H19doXuB44p9Z2eYi7at2ZwOJkerecAtwLHJunuIGllAqW54APdKO9e9Xlkn75U1uTZbkmaSpwFLAamBgR2wCS1wnZRTai64CLgHeT+fHAaxGxM5nPY7t/EHgJuCnpKrpB0vvIeXtHxG+BayhVW9uAHcA68t/eqZHat5/+Vv8BuCeZznXckmYDv42IX1at6mjcvUroqrEs19dLStoP+AFwfkS8nnU89Ug6HdgeEesqF9fYNG/tPhb4KPCdiDiK0r1+ctW9UkvS5zwHOBT4U+B9lA6fq+Wtvevph+8Mki6j1D26OF1UY7NcxC1pX+Ay4PJaq2ssaznuXiX0rcAhFfNTgBd6tO+mSdqLUjJfHBE/TBa/KGlSsn4SsD2r+EYwA5gt6TlgCaVul+uAAySNTbbJY7tvBbZGxOpkfimlBJ/39v4k8KuIeCki/gj8EDiO/Ld3aqT2zf3fqqS5wOnAZyPppyDfcf85pf/x/zL5+5wCPCrpYDocd68S+hpgWnIFwN6UTl4s79G+myJJwI3Axoj4ZsWq5cDcZHoupb713IiISyJiSkRMpdS+90fEZ4EHgE8nm+Ux7v8FfiNpMFn0CeBJct7elLpajpG0b/KdSePOdXtXGKl9lwPnJVdfHAPsSLtm8kDSLOBiYHZEvF2xajlwtqR9JB1K6STjz7OIsVpEPBEREyJiavL3uRX4aPLd72x79/AkwamUzko/C1yW1cmKBuL8a0qHPI8Dv0j+nUqpP/o+YHPyemDWsY7yGWYCK5LpD1L6Yj8D/DewT9bx1Yj3SGBt0uZ3AuP6ob2BK4CngPXArcA+eWxv4DZK/fx/TJLJvJHal1IXwLeTv9MnKF3Fk6e4n6HU55z+bX63YvvLkrifBj6Vp7ir1j/HrpOiHW1vD/03MysIjxQ1MysIJ3Qzs4JwQjczKwgndDOzgnBCNzMrCCd0M7OCcEI3MyuI/wf2xELFvRW4QQAAAABJRU5ErkJggg==\n", 1373 | "text/plain": [ 1374 | "
" 1375 | ] 1376 | }, 1377 | "metadata": { 1378 | "needs_background": "light" 1379 | }, 1380 | "output_type": "display_data" 1381 | } 1382 | ], 1383 | "source": [ 1384 | "import matplotlib.pyplot as plt\n", 1385 | "%matplotlib inline\n", 1386 | "# Load the image and convert it to grayscale\n", 1387 | "image = cv2.imread('./unsolved-captchas/0.png')\n", 1388 | "gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)\n", 1389 | " \n", 1390 | "# Add some extra padding around the image\n", 1391 | "gray = cv2.copyMakeBorder(gray, 8, 8, 8, 8, cv2.BORDER_REPLICATE)\n", 1392 | "\n", 1393 | "# threshold the image\n", 1394 | "thresh = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY_INV, cv2.THRESH_OTSU)[1]\n", 1395 | "\n", 1396 | "# find the contours\n", 1397 | "contours, hierarchy = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)\n", 1398 | "\n", 1399 | "\n", 1400 | " \n", 1401 | "letter_image_regions = []\n", 1402 | "\n", 1403 | "# Now we can loop through each of the contours and extract the letter\n", 1404 | "\n", 1405 | "for contour in contours:\n", 1406 | " # Get the rectangle that contains the contour\n", 1407 | " (x, y, w, h) = cv2.boundingRect(contour)\n", 1408 | " \n", 1409 | " # checking if any counter is too wide\n", 1410 | " # if countour is too wide then there could be two letters joined together or are very close to each other\n", 1411 | " if w / h > 1.25:\n", 1412 | " # Split it in half into two letter regions\n", 1413 | " half_width = int(w / 2)\n", 1414 | " letter_image_regions.append((x, y, half_width, h))\n", 1415 | " letter_image_regions.append((x + half_width, y, half_width, h))\n", 1416 | " else:\n", 1417 | " letter_image_regions.append((x, y, w, h))\n", 1418 | " \n", 1419 | "\n", 1420 | "# Sort the detected letter images based on the x coordinate to make sure\n", 1421 | "# we get them from left-to-right so that we match the right image with the right letter \n", 1422 | "\n", 1423 | "letter_image_regions = sorted(letter_image_regions, key=lambda x: x[0])\n", 1424 | "\n", 1425 | "# Create an output image and a list to hold our predicted letters\n", 1426 | "output = cv2.merge([gray] * 3)\n", 1427 | "predictions = []\n", 1428 | " \n", 1429 | "# Creating an empty list for storing predicted letters\n", 1430 | "predictions = []\n", 1431 | " \n", 1432 | "# Save out each letter as a single image\n", 1433 | "for letter_bounding_box in letter_image_regions:\n", 1434 | " # Grab the coordinates of the letter in the image\n", 1435 | " x, y, w, h = letter_bounding_box\n", 1436 | "\n", 1437 | " # Extract the letter from the original image with a 2-pixel margin around the edge\n", 1438 | " letter_image = gray[y - 2:y + h + 2, x - 2:x + w + 2]\n", 1439 | "\n", 1440 | " letter_image = cv2.resize(letter_image, (30,30))\n", 1441 | " \n", 1442 | " # Turn the single image into a 4d list of images\n", 1443 | " letter_image = np.expand_dims(letter_image, axis=2)\n", 1444 | " letter_image = np.expand_dims(letter_image, axis=0)\n", 1445 | "\n", 1446 | " # making prediction\n", 1447 | " pred = model.predict(letter_image)\n", 1448 | " \n", 1449 | " # Convert the one-hot-encoded prediction back to a normal letter\n", 1450 | " letter = lb.inverse_transform(pred)[0]\n", 1451 | " predictions.append(letter)\n", 1452 | "\n", 1453 | "\n", 1454 | " # draw the prediction on the output image\n", 1455 | " cv2.rectangle(output, (x - 2, y - 2), (x + w + 4, y + h + 4), (0, 255, 0), 1)\n", 1456 | " cv2.putText(output, letter, (x - 5, y - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.55, (0, 255, 0), 2)\n", 1457 | "\n", 1458 | "# Print the captcha's text\n", 1459 | "captcha_text = \"\".join(predictions)\n", 1460 | "print(\"CAPTCHA text is: {}\".format(captcha_text))\n", 1461 | "\n", 1462 | "# Show the annotated image\n", 1463 | "plt.imshow(output)" 1464 | ] 1465 | }, 1466 | { 1467 | "cell_type": "code", 1468 | "execution_count": 13, 1469 | "metadata": {}, 1470 | "outputs": [], 1471 | "source": [ 1472 | "#Saving the model for future use\n", 1473 | "model.save(\"captcha_extractor_model.hdf5\")" 1474 | ] 1475 | }, 1476 | { 1477 | "cell_type": "markdown", 1478 | "metadata": {}, 1479 | "source": [ 1480 | "### Making predictions on test data" 1481 | ] 1482 | }, 1483 | { 1484 | "cell_type": "code", 1485 | "execution_count": null, 1486 | "metadata": {}, 1487 | "outputs": [], 1488 | "source": [ 1489 | "test_image_folder = 'unsolved-captchas'\n", 1490 | "OUTPUT_FOLDER = 'predicted captchas'\n", 1491 | "test_captcha_images = glob.glob(os.path.join(test_image_folder, \"*\"))" 1492 | ] 1493 | }, 1494 | { 1495 | "cell_type": "code", 1496 | "execution_count": null, 1497 | "metadata": {}, 1498 | "outputs": [], 1499 | "source": [ 1500 | "# loop over the image paths\n", 1501 | "for (i, image) in enumerate(test_captcha_images):\n", 1502 | " print(\"processing image {}/{}\".format(i + 1, len(test_captcha_images)))\n", 1503 | "\n", 1504 | " # Load the image and convert it to grayscale\n", 1505 | " img = cv2.imread(image)\n", 1506 | " gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)\n", 1507 | " \n", 1508 | " # Add some extra padding around the image\n", 1509 | " gray = cv2.copyMakeBorder(gray, 8, 8, 8, 8, cv2.BORDER_REPLICATE)\n", 1510 | "\n", 1511 | " # threshold the image (convert it to pure black and white)\n", 1512 | " thresh = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY_INV, cv2.THRESH_OTSU)[1]\n", 1513 | "\n", 1514 | " # find the contours (continuous blobs of pixels) the image\n", 1515 | " contours, hierarchy = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)\n", 1516 | "\n", 1517 | " \n", 1518 | " letter_image_regions = []\n", 1519 | "\n", 1520 | " # Now we can loop through each of the contours and extract the letter\n", 1521 | " for contour in contours:\n", 1522 | " # Get the rectangle that contains the contour\n", 1523 | " (x, y, w, h) = cv2.boundingRect(contour)\n", 1524 | " \n", 1525 | " # checking if any counter is too wide\n", 1526 | " # if countour is too wide then there could be two letters joined together or are very close to each other\n", 1527 | " if w / h > 1.25:\n", 1528 | " # Split it in half into two letter regions\n", 1529 | " half_width = int(w / 2)\n", 1530 | " letter_image_regions.append((x, y, half_width, h))\n", 1531 | " letter_image_regions.append((x + half_width, y, half_width, h))\n", 1532 | " else:\n", 1533 | " letter_image_regions.append((x, y, w, h))\n", 1534 | " \n", 1535 | "\n", 1536 | " \n", 1537 | " # Sort the detected letter images based on the x coordinate to make sure\n", 1538 | " # we get them from left-to-right so that we match the right image with the right letter \n", 1539 | " letter_image_regions = sorted(letter_image_regions, key=lambda x: x[0])\n", 1540 | " \n", 1541 | " # Creating an empty list for storing predicted letters\n", 1542 | " predictions = []\n", 1543 | " \n", 1544 | " # Save out each letter as a single image\n", 1545 | " for letter_bounding_box in letter_image_regions:\n", 1546 | " # Grab the coordinates of the letter in the image\n", 1547 | " x, y, w, h = letter_bounding_box\n", 1548 | "\n", 1549 | " # Extract the letter from the original image with a 2-pixel margin around the edge\n", 1550 | " letter_image = gray[y - 2:y + h + 2, x - 2:x + w + 2]\n", 1551 | "\n", 1552 | " letter_image = cv2.resize(letter_image, (30,30))\n", 1553 | " \n", 1554 | " # Turn the single image into a 4d list of images\n", 1555 | " letter_image = np.expand_dims(letter_image, axis=2)\n", 1556 | " letter_image = np.expand_dims(letter_image, axis=0)\n", 1557 | "\n", 1558 | " # making prediction\n", 1559 | " pred = model.predict(letter_image)\n", 1560 | " \n", 1561 | " # Convert the one-hot-encoded prediction back to a normal letter\n", 1562 | " letter = lb.inverse_transform(pred)[0]\n", 1563 | " predictions.append(letter)\n", 1564 | "\n", 1565 | " \n", 1566 | " # joining predicted captcha's text\n", 1567 | " captcha_text = \"\".join(predictions)\n", 1568 | " \n", 1569 | " # Get the folder to save the image in\n", 1570 | " save_path = os.path.join(OUTPUT_FOLDER, captcha_text)\n", 1571 | " \n", 1572 | " p = os.path.join(save_path+'.png' )\n", 1573 | " #writing the image to the output folder\n", 1574 | " cv2.imwrite(p, img)\n" 1575 | ] 1576 | }, 1577 | { 1578 | "cell_type": "code", 1579 | "execution_count": null, 1580 | "metadata": {}, 1581 | "outputs": [], 1582 | "source": [] 1583 | } 1584 | ], 1585 | "metadata": { 1586 | "kernelspec": { 1587 | "display_name": "Python 3", 1588 | "language": "python", 1589 | "name": "python3" 1590 | }, 1591 | "language_info": { 1592 | "codemirror_mode": { 1593 | "name": "ipython", 1594 | "version": 3 1595 | }, 1596 | "file_extension": ".py", 1597 | "mimetype": "text/x-python", 1598 | "name": "python", 1599 | "nbconvert_exporter": "python", 1600 | "pygments_lexer": "ipython3", 1601 | "version": "3.6.4rc1" 1602 | } 1603 | }, 1604 | "nbformat": 4, 1605 | "nbformat_minor": 2 1606 | } 1607 | -------------------------------------------------------------------------------- /captcha_extractor.py: -------------------------------------------------------------------------------- 1 | import pandas as pd 2 | import numpy as np 3 | import cv2 4 | import imutils 5 | from imutils import paths 6 | import os 7 | import os.path 8 | import pickle 9 | import matplotlib.pyplot as plt 10 | from IPython.display import Image, display 11 | 12 | from keras.models import load_model 13 | from sklearn.preprocessing import LabelBinarizer 14 | import argparse 15 | 16 | 17 | ap = argparse.ArgumentParser() 18 | ap.add_argument("-i", "--input", required=True, help="path to the captcha png image") 19 | ap.add_argument("-o","--output", required=True, help="path to save the output image") 20 | ap.add_argument("-m", "--model", required=True, help="path to the hdf5 model") 21 | ap.add_argument("-lb", "--labels", required=True, help="path to the captcha labels dat file") 22 | args = vars(ap.parse_args()) 23 | 24 | 25 | #Loading the model 26 | model = load_model(args['model']) 27 | with open(args['labels'], "rb") as f: 28 | lb = pickle.load(f) 29 | 30 | # Load the image and convert it to grayscale 31 | image = cv2.imread(args['input']) 32 | gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) 33 | 34 | # Adding some extra padding around the image 35 | gray = cv2.copyMakeBorder(gray, 10, 10, 10, 10, cv2.BORDER_REPLICATE) 36 | 37 | # applying threshold 38 | thresh = cv2.threshold(gray, 127, 255, cv2.THRESH_BINARY_INV, cv2.THRESH_OTSU)[1] 39 | 40 | # find the contours 41 | contours, hierarchy = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) 42 | 43 | 44 | 45 | letter_image_regions = [] 46 | 47 | # Now we can loop through each of the contours and extract the letter 48 | 49 | for contour in contours: 50 | # Get the rectangle that contains the contour 51 | (x, y, w, h) = cv2.boundingRect(contour) 52 | 53 | # checking if any counter is too wide 54 | # if countour is too wide then there could be two letters joined together or are very close to each other 55 | if w / h > 1.25: 56 | # Split it in half into two letter regions 57 | half_width = int(w / 2) 58 | letter_image_regions.append((x, y, half_width, h)) 59 | letter_image_regions.append((x + half_width, y, half_width, h)) 60 | else: 61 | letter_image_regions.append((x, y, w, h)) 62 | 63 | 64 | # Sort the detected letter images based on the x coordinate to make sure 65 | # we get them from left-to-right so that we match the right image with the right letter 66 | 67 | letter_image_regions = sorted(letter_image_regions, key=lambda x: x[0]) 68 | 69 | # Create an output image and a list to hold our predicted letters 70 | output = cv2.merge([gray] * 3) 71 | predictions = [] 72 | 73 | # Creating an empty list for storing predicted letters 74 | predictions = [] 75 | 76 | # Save out each letter as a single image 77 | for letter_bounding_box in letter_image_regions: 78 | # Grab the coordinates of the letter in the image 79 | x, y, w, h = letter_bounding_box 80 | 81 | # Extract the letter from the original image with a 2-pixel margin around the edge 82 | letter_image = gray[y - 2:y + h + 2, x - 2:x + w + 2] 83 | 84 | letter_image = cv2.resize(letter_image, (30,30)) 85 | 86 | # Turn the single image into a 4d list of images 87 | letter_image = np.expand_dims(letter_image, axis=2) 88 | letter_image = np.expand_dims(letter_image, axis=0) 89 | 90 | # making prediction 91 | pred = model.predict(letter_image) 92 | 93 | # Convert the one-hot-encoded prediction back to a normal letter 94 | letter = lb.inverse_transform(pred)[0] 95 | predictions.append(letter) 96 | 97 | 98 | # draw the prediction on the output image 99 | cv2.rectangle(output, (x - 2, y - 2), (x + w + 4, y + h + 4), (0, 255, 0), 1) 100 | cv2.putText(output, letter, (x - 5, y - 5), cv2.FONT_HERSHEY_SIMPLEX, 0.55, (0, 255, 0), 2) 101 | 102 | 103 | 104 | # Print the captcha's text 105 | captcha_text = "".join(predictions) 106 | print("CAPTCHA text is: {}".format(captcha_text)) 107 | 108 | # Get the folder to save the image in 109 | save_path = os.path.join(args['output'], captcha_text) 110 | 111 | p = os.path.join(save_path+'.png' ) 112 | #writing the image to the output folder 113 | cv2.imwrite(p, image) 114 | cv2.imwrite("output.png",output) 115 | print("Output saved to "+args['output']) 116 | # Show the annotated image 117 | # cv2.imshow("Output", output) 118 | # cv2.waitKey() 119 | 120 | -------------------------------------------------------------------------------- /captcha_extractor_model.hdf5: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chauhan01/Captcha-text-extraction/fb2650057daaa4a86ce3a283e0f19ee23310adbe/captcha_extractor_model.hdf5 -------------------------------------------------------------------------------- /captcha_labels: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chauhan01/Captcha-text-extraction/fb2650057daaa4a86ce3a283e0f19ee23310adbe/captcha_labels -------------------------------------------------------------------------------- /test_sample_captcha.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/chauhan01/Captcha-text-extraction/fb2650057daaa4a86ce3a283e0f19ee23310adbe/test_sample_captcha.png --------------------------------------------------------------------------------