├── LICENSE ├── README.md ├── classifier_ns_2 ├── 20201211 │ └── efficientnetb0_ns_fit_normal_vs_falldown_stage1stage2_normal_vs_falldown_beforeafterjangdae_4_99.27884615384616_False_True.pt ├── module_2class.py └── predict_2class.py ├── predict.py ├── requirements.txt ├── source ├── falldown.png └── swoon_person.gif ├── utils ├── README.txt ├── split_each_frame.py ├── txt_to_xml.py └── xml_to_txt.py └── yolov5_test ├── AdamW.py ├── Dockerfile ├── LICENSE ├── data ├── AGC.yaml ├── coco.yaml ├── coco128.yaml ├── hyp.finetune.yaml ├── hyp.scratch.yaml ├── scripts │ ├── get_coco.sh │ └── get_voc.sh └── voc.yaml ├── detect.py ├── detection_module.py ├── hubconf.py ├── models ├── common.py ├── experimental.py ├── export.py ├── hub │ ├── yolov3-spp.yaml │ ├── yolov5-fpn.yaml │ └── yolov5-panet.yaml ├── yolo.py ├── yolov5l.yaml ├── yolov5m.yaml ├── yolov5s.yaml └── yolov5x.yaml └── readme /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | Preamble 9 | 10 | The GNU General Public License is a free, copyleft license for 11 | software and other kinds of works. 12 | 13 | The licenses for most software and other practical works are designed 14 | to take away your freedom to share and change the works. By contrast, 15 | the GNU General Public License is intended to guarantee your freedom to 16 | share and change all versions of a program--to make sure it remains free 17 | software for all its users. We, the Free Software Foundation, use the 18 | GNU General Public License for most of our software; it applies also to 19 | any other work released this way by its authors. You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | them if you wish), that you receive source code or can get it if you 26 | want it, that you can change the software or use pieces of it in new 27 | free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you 30 | these rights or asking you to surrender the rights. Therefore, you have 31 | certain responsibilities if you distribute copies of the software, or if 32 | you modify it: responsibilities to respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must pass on to the recipients the same 36 | freedoms that you received. You must make sure that they, too, receive 37 | or can get the source code. And you must show them these terms so they 38 | know their rights. 39 | 40 | Developers that use the GNU GPL protect your rights with two steps: 41 | (1) assert copyright on the software, and (2) offer you this License 42 | giving you legal permission to copy, distribute and/or modify it. 43 | 44 | For the developers' and authors' protection, the GPL clearly explains 45 | that there is no warranty for this free software. For both users' and 46 | authors' sake, the GPL requires that modified versions be marked as 47 | changed, so that their problems will not be attributed erroneously to 48 | authors of previous versions. 49 | 50 | Some devices are designed to deny users access to install or run 51 | modified versions of the software inside them, although the manufacturer 52 | can do so. This is fundamentally incompatible with the aim of 53 | protecting users' freedom to change the software. The systematic 54 | pattern of such abuse occurs in the area of products for individuals to 55 | use, which is precisely where it is most unacceptable. Therefore, we 56 | have designed this version of the GPL to prohibit the practice for those 57 | products. If such problems arise substantially in other domains, we 58 | stand ready to extend this provision to those domains in future versions 59 | of the GPL, as needed to protect the freedom of users. 60 | 61 | Finally, every program is threatened constantly by software patents. 62 | States should not allow patents to restrict development and use of 63 | software on general-purpose computers, but in those that do, we wish to 64 | avoid the special danger that patents applied to a free program could 65 | make it effectively proprietary. To prevent this, the GPL assures that 66 | patents cannot be used to render the program non-free. 67 | 68 | The precise terms and conditions for copying, distribution and 69 | modification follow. 70 | 71 | TERMS AND CONDITIONS 72 | 73 | 0. Definitions. 74 | 75 | "This License" refers to version 3 of the GNU General Public License. 76 | 77 | "Copyright" also means copyright-like laws that apply to other kinds of 78 | works, such as semiconductor masks. 79 | 80 | "The Program" refers to any copyrightable work licensed under this 81 | License. Each licensee is addressed as "you". "Licensees" and 82 | "recipients" may be individuals or organizations. 83 | 84 | To "modify" a work means to copy from or adapt all or part of the work 85 | in a fashion requiring copyright permission, other than the making of an 86 | exact copy. The resulting work is called a "modified version" of the 87 | earlier work or a work "based on" the earlier work. 88 | 89 | A "covered work" means either the unmodified Program or a work based 90 | on the Program. 91 | 92 | To "propagate" a work means to do anything with it that, without 93 | permission, would make you directly or secondarily liable for 94 | infringement under applicable copyright law, except executing it on a 95 | computer or modifying a private copy. Propagation includes copying, 96 | distribution (with or without modification), making available to the 97 | public, and in some countries other activities as well. 98 | 99 | To "convey" a work means any kind of propagation that enables other 100 | parties to make or receive copies. Mere interaction with a user through 101 | a computer network, with no transfer of a copy, is not conveying. 102 | 103 | An interactive user interface displays "Appropriate Legal Notices" 104 | to the extent that it includes a convenient and prominently visible 105 | feature that (1) displays an appropriate copyright notice, and (2) 106 | tells the user that there is no warranty for the work (except to the 107 | extent that warranties are provided), that licensees may convey the 108 | work under this License, and how to view a copy of this License. If 109 | the interface presents a list of user commands or options, such as a 110 | menu, a prominent item in the list meets this criterion. 111 | 112 | 1. Source Code. 113 | 114 | The "source code" for a work means the preferred form of the work 115 | for making modifications to it. "Object code" means any non-source 116 | form of a work. 117 | 118 | A "Standard Interface" means an interface that either is an official 119 | standard defined by a recognized standards body, or, in the case of 120 | interfaces specified for a particular programming language, one that 121 | is widely used among developers working in that language. 122 | 123 | The "System Libraries" of an executable work include anything, other 124 | than the work as a whole, that (a) is included in the normal form of 125 | packaging a Major Component, but which is not part of that Major 126 | Component, and (b) serves only to enable use of the work with that 127 | Major Component, or to implement a Standard Interface for which an 128 | implementation is available to the public in source code form. A 129 | "Major Component", in this context, means a major essential component 130 | (kernel, window system, and so on) of the specific operating system 131 | (if any) on which the executable work runs, or a compiler used to 132 | produce the work, or an object code interpreter used to run it. 133 | 134 | The "Corresponding Source" for a work in object code form means all 135 | the source code needed to generate, install, and (for an executable 136 | work) run the object code and to modify the work, including scripts to 137 | control those activities. However, it does not include the work's 138 | System Libraries, or general-purpose tools or generally available free 139 | programs which are used unmodified in performing those activities but 140 | which are not part of the work. For example, Corresponding Source 141 | includes interface definition files associated with source files for 142 | the work, and the source code for shared libraries and dynamically 143 | linked subprograms that the work is specifically designed to require, 144 | such as by intimate data communication or control flow between those 145 | subprograms and other parts of the work. 146 | 147 | The Corresponding Source need not include anything that users 148 | can regenerate automatically from other parts of the Corresponding 149 | Source. 150 | 151 | The Corresponding Source for a work in source code form is that 152 | same work. 153 | 154 | 2. Basic Permissions. 155 | 156 | All rights granted under this License are granted for the term of 157 | copyright on the Program, and are irrevocable provided the stated 158 | conditions are met. This License explicitly affirms your unlimited 159 | permission to run the unmodified Program. The output from running a 160 | covered work is covered by this License only if the output, given its 161 | content, constitutes a covered work. This License acknowledges your 162 | rights of fair use or other equivalent, as provided by copyright law. 163 | 164 | You may make, run and propagate covered works that you do not 165 | convey, without conditions so long as your license otherwise remains 166 | in force. You may convey covered works to others for the sole purpose 167 | of having them make modifications exclusively for you, or provide you 168 | with facilities for running those works, provided that you comply with 169 | the terms of this License in conveying all material for which you do 170 | not control copyright. Those thus making or running the covered works 171 | for you must do so exclusively on your behalf, under your direction 172 | and control, on terms that prohibit them from making any copies of 173 | your copyrighted material outside their relationship with you. 174 | 175 | Conveying under any other circumstances is permitted solely under 176 | the conditions stated below. Sublicensing is not allowed; section 10 177 | makes it unnecessary. 178 | 179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 180 | 181 | No covered work shall be deemed part of an effective technological 182 | measure under any applicable law fulfilling obligations under article 183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 184 | similar laws prohibiting or restricting circumvention of such 185 | measures. 186 | 187 | When you convey a covered work, you waive any legal power to forbid 188 | circumvention of technological measures to the extent such circumvention 189 | is effected by exercising rights under this License with respect to 190 | the covered work, and you disclaim any intention to limit operation or 191 | modification of the work as a means of enforcing, against the work's 192 | users, your or third parties' legal rights to forbid circumvention of 193 | technological measures. 194 | 195 | 4. Conveying Verbatim Copies. 196 | 197 | You may convey verbatim copies of the Program's source code as you 198 | receive it, in any medium, provided that you conspicuously and 199 | appropriately publish on each copy an appropriate copyright notice; 200 | keep intact all notices stating that this License and any 201 | non-permissive terms added in accord with section 7 apply to the code; 202 | keep intact all notices of the absence of any warranty; and give all 203 | recipients a copy of this License along with the Program. 204 | 205 | You may charge any price or no price for each copy that you convey, 206 | and you may offer support or warranty protection for a fee. 207 | 208 | 5. Conveying Modified Source Versions. 209 | 210 | You may convey a work based on the Program, or the modifications to 211 | produce it from the Program, in the form of source code under the 212 | terms of section 4, provided that you also meet all of these conditions: 213 | 214 | a) The work must carry prominent notices stating that you modified 215 | it, and giving a relevant date. 216 | 217 | b) The work must carry prominent notices stating that it is 218 | released under this License and any conditions added under section 219 | 7. This requirement modifies the requirement in section 4 to 220 | "keep intact all notices". 221 | 222 | c) You must license the entire work, as a whole, under this 223 | License to anyone who comes into possession of a copy. This 224 | License will therefore apply, along with any applicable section 7 225 | additional terms, to the whole of the work, and all its parts, 226 | regardless of how they are packaged. This License gives no 227 | permission to license the work in any other way, but it does not 228 | invalidate such permission if you have separately received it. 229 | 230 | d) If the work has interactive user interfaces, each must display 231 | Appropriate Legal Notices; however, if the Program has interactive 232 | interfaces that do not display Appropriate Legal Notices, your 233 | work need not make them do so. 234 | 235 | A compilation of a covered work with other separate and independent 236 | works, which are not by their nature extensions of the covered work, 237 | and which are not combined with it such as to form a larger program, 238 | in or on a volume of a storage or distribution medium, is called an 239 | "aggregate" if the compilation and its resulting copyright are not 240 | used to limit the access or legal rights of the compilation's users 241 | beyond what the individual works permit. Inclusion of a covered work 242 | in an aggregate does not cause this License to apply to the other 243 | parts of the aggregate. 244 | 245 | 6. Conveying Non-Source Forms. 246 | 247 | You may convey a covered work in object code form under the terms 248 | of sections 4 and 5, provided that you also convey the 249 | machine-readable Corresponding Source under the terms of this License, 250 | in one of these ways: 251 | 252 | a) Convey the object code in, or embodied in, a physical product 253 | (including a physical distribution medium), accompanied by the 254 | Corresponding Source fixed on a durable physical medium 255 | customarily used for software interchange. 256 | 257 | b) Convey the object code in, or embodied in, a physical product 258 | (including a physical distribution medium), accompanied by a 259 | written offer, valid for at least three years and valid for as 260 | long as you offer spare parts or customer support for that product 261 | model, to give anyone who possesses the object code either (1) a 262 | copy of the Corresponding Source for all the software in the 263 | product that is covered by this License, on a durable physical 264 | medium customarily used for software interchange, for a price no 265 | more than your reasonable cost of physically performing this 266 | conveying of source, or (2) access to copy the 267 | Corresponding Source from a network server at no charge. 268 | 269 | c) Convey individual copies of the object code with a copy of the 270 | written offer to provide the Corresponding Source. This 271 | alternative is allowed only occasionally and noncommercially, and 272 | only if you received the object code with such an offer, in accord 273 | with subsection 6b. 274 | 275 | d) Convey the object code by offering access from a designated 276 | place (gratis or for a charge), and offer equivalent access to the 277 | Corresponding Source in the same way through the same place at no 278 | further charge. You need not require recipients to copy the 279 | Corresponding Source along with the object code. If the place to 280 | copy the object code is a network server, the Corresponding Source 281 | may be on a different server (operated by you or a third party) 282 | that supports equivalent copying facilities, provided you maintain 283 | clear directions next to the object code saying where to find the 284 | Corresponding Source. Regardless of what server hosts the 285 | Corresponding Source, you remain obligated to ensure that it is 286 | available for as long as needed to satisfy these requirements. 287 | 288 | e) Convey the object code using peer-to-peer transmission, provided 289 | you inform other peers where the object code and Corresponding 290 | Source of the work are being offered to the general public at no 291 | charge under subsection 6d. 292 | 293 | A separable portion of the object code, whose source code is excluded 294 | from the Corresponding Source as a System Library, need not be 295 | included in conveying the object code work. 296 | 297 | A "User Product" is either (1) a "consumer product", which means any 298 | tangible personal property which is normally used for personal, family, 299 | or household purposes, or (2) anything designed or sold for incorporation 300 | into a dwelling. In determining whether a product is a consumer product, 301 | doubtful cases shall be resolved in favor of coverage. For a particular 302 | product received by a particular user, "normally used" refers to a 303 | typical or common use of that class of product, regardless of the status 304 | of the particular user or of the way in which the particular user 305 | actually uses, or expects or is expected to use, the product. A product 306 | is a consumer product regardless of whether the product has substantial 307 | commercial, industrial or non-consumer uses, unless such uses represent 308 | the only significant mode of use of the product. 309 | 310 | "Installation Information" for a User Product means any methods, 311 | procedures, authorization keys, or other information required to install 312 | and execute modified versions of a covered work in that User Product from 313 | a modified version of its Corresponding Source. The information must 314 | suffice to ensure that the continued functioning of the modified object 315 | code is in no case prevented or interfered with solely because 316 | modification has been made. 317 | 318 | If you convey an object code work under this section in, or with, or 319 | specifically for use in, a User Product, and the conveying occurs as 320 | part of a transaction in which the right of possession and use of the 321 | User Product is transferred to the recipient in perpetuity or for a 322 | fixed term (regardless of how the transaction is characterized), the 323 | Corresponding Source conveyed under this section must be accompanied 324 | by the Installation Information. But this requirement does not apply 325 | if neither you nor any third party retains the ability to install 326 | modified object code on the User Product (for example, the work has 327 | been installed in ROM). 328 | 329 | The requirement to provide Installation Information does not include a 330 | requirement to continue to provide support service, warranty, or updates 331 | for a work that has been modified or installed by the recipient, or for 332 | the User Product in which it has been modified or installed. Access to a 333 | network may be denied when the modification itself materially and 334 | adversely affects the operation of the network or violates the rules and 335 | protocols for communication across the network. 336 | 337 | Corresponding Source conveyed, and Installation Information provided, 338 | in accord with this section must be in a format that is publicly 339 | documented (and with an implementation available to the public in 340 | source code form), and must require no special password or key for 341 | unpacking, reading or copying. 342 | 343 | 7. Additional Terms. 344 | 345 | "Additional permissions" are terms that supplement the terms of this 346 | License by making exceptions from one or more of its conditions. 347 | Additional permissions that are applicable to the entire Program shall 348 | be treated as though they were included in this License, to the extent 349 | that they are valid under applicable law. If additional permissions 350 | apply only to part of the Program, that part may be used separately 351 | under those permissions, but the entire Program remains governed by 352 | this License without regard to the additional permissions. 353 | 354 | When you convey a copy of a covered work, you may at your option 355 | remove any additional permissions from that copy, or from any part of 356 | it. (Additional permissions may be written to require their own 357 | removal in certain cases when you modify the work.) You may place 358 | additional permissions on material, added by you to a covered work, 359 | for which you have or can give appropriate copyright permission. 360 | 361 | Notwithstanding any other provision of this License, for material you 362 | add to a covered work, you may (if authorized by the copyright holders of 363 | that material) supplement the terms of this License with terms: 364 | 365 | a) Disclaiming warranty or limiting liability differently from the 366 | terms of sections 15 and 16 of this License; or 367 | 368 | b) Requiring preservation of specified reasonable legal notices or 369 | author attributions in that material or in the Appropriate Legal 370 | Notices displayed by works containing it; or 371 | 372 | c) Prohibiting misrepresentation of the origin of that material, or 373 | requiring that modified versions of such material be marked in 374 | reasonable ways as different from the original version; or 375 | 376 | d) Limiting the use for publicity purposes of names of licensors or 377 | authors of the material; or 378 | 379 | e) Declining to grant rights under trademark law for use of some 380 | trade names, trademarks, or service marks; or 381 | 382 | f) Requiring indemnification of licensors and authors of that 383 | material by anyone who conveys the material (or modified versions of 384 | it) with contractual assumptions of liability to the recipient, for 385 | any liability that these contractual assumptions directly impose on 386 | those licensors and authors. 387 | 388 | All other non-permissive additional terms are considered "further 389 | restrictions" within the meaning of section 10. If the Program as you 390 | received it, or any part of it, contains a notice stating that it is 391 | governed by this License along with a term that is a further 392 | restriction, you may remove that term. If a license document contains 393 | a further restriction but permits relicensing or conveying under this 394 | License, you may add to a covered work material governed by the terms 395 | of that license document, provided that the further restriction does 396 | not survive such relicensing or conveying. 397 | 398 | If you add terms to a covered work in accord with this section, you 399 | must place, in the relevant source files, a statement of the 400 | additional terms that apply to those files, or a notice indicating 401 | where to find the applicable terms. 402 | 403 | Additional terms, permissive or non-permissive, may be stated in the 404 | form of a separately written license, or stated as exceptions; 405 | the above requirements apply either way. 406 | 407 | 8. Termination. 408 | 409 | You may not propagate or modify a covered work except as expressly 410 | provided under this License. Any attempt otherwise to propagate or 411 | modify it is void, and will automatically terminate your rights under 412 | this License (including any patent licenses granted under the third 413 | paragraph of section 11). 414 | 415 | However, if you cease all violation of this License, then your 416 | license from a particular copyright holder is reinstated (a) 417 | provisionally, unless and until the copyright holder explicitly and 418 | finally terminates your license, and (b) permanently, if the copyright 419 | holder fails to notify you of the violation by some reasonable means 420 | prior to 60 days after the cessation. 421 | 422 | Moreover, your license from a particular copyright holder is 423 | reinstated permanently if the copyright holder notifies you of the 424 | violation by some reasonable means, this is the first time you have 425 | received notice of violation of this License (for any work) from that 426 | copyright holder, and you cure the violation prior to 30 days after 427 | your receipt of the notice. 428 | 429 | Termination of your rights under this section does not terminate the 430 | licenses of parties who have received copies or rights from you under 431 | this License. If your rights have been terminated and not permanently 432 | reinstated, you do not qualify to receive new licenses for the same 433 | material under section 10. 434 | 435 | 9. Acceptance Not Required for Having Copies. 436 | 437 | You are not required to accept this License in order to receive or 438 | run a copy of the Program. Ancillary propagation of a covered work 439 | occurring solely as a consequence of using peer-to-peer transmission 440 | to receive a copy likewise does not require acceptance. However, 441 | nothing other than this License grants you permission to propagate or 442 | modify any covered work. These actions infringe copyright if you do 443 | not accept this License. Therefore, by modifying or propagating a 444 | covered work, you indicate your acceptance of this License to do so. 445 | 446 | 10. Automatic Licensing of Downstream Recipients. 447 | 448 | Each time you convey a covered work, the recipient automatically 449 | receives a license from the original licensors, to run, modify and 450 | propagate that work, subject to this License. You are not responsible 451 | for enforcing compliance by third parties with this License. 452 | 453 | An "entity transaction" is a transaction transferring control of an 454 | organization, or substantially all assets of one, or subdividing an 455 | organization, or merging organizations. If propagation of a covered 456 | work results from an entity transaction, each party to that 457 | transaction who receives a copy of the work also receives whatever 458 | licenses to the work the party's predecessor in interest had or could 459 | give under the previous paragraph, plus a right to possession of the 460 | Corresponding Source of the work from the predecessor in interest, if 461 | the predecessor has it or can get it with reasonable efforts. 462 | 463 | You may not impose any further restrictions on the exercise of the 464 | rights granted or affirmed under this License. For example, you may 465 | not impose a license fee, royalty, or other charge for exercise of 466 | rights granted under this License, and you may not initiate litigation 467 | (including a cross-claim or counterclaim in a lawsuit) alleging that 468 | any patent claim is infringed by making, using, selling, offering for 469 | sale, or importing the Program or any portion of it. 470 | 471 | 11. Patents. 472 | 473 | A "contributor" is a copyright holder who authorizes use under this 474 | License of the Program or a work on which the Program is based. The 475 | work thus licensed is called the contributor's "contributor version". 476 | 477 | A contributor's "essential patent claims" are all patent claims 478 | owned or controlled by the contributor, whether already acquired or 479 | hereafter acquired, that would be infringed by some manner, permitted 480 | by this License, of making, using, or selling its contributor version, 481 | but do not include claims that would be infringed only as a 482 | consequence of further modification of the contributor version. For 483 | purposes of this definition, "control" includes the right to grant 484 | patent sublicenses in a manner consistent with the requirements of 485 | this License. 486 | 487 | Each contributor grants you a non-exclusive, worldwide, royalty-free 488 | patent license under the contributor's essential patent claims, to 489 | make, use, sell, offer for sale, import and otherwise run, modify and 490 | propagate the contents of its contributor version. 491 | 492 | In the following three paragraphs, a "patent license" is any express 493 | agreement or commitment, however denominated, not to enforce a patent 494 | (such as an express permission to practice a patent or covenant not to 495 | sue for patent infringement). To "grant" such a patent license to a 496 | party means to make such an agreement or commitment not to enforce a 497 | patent against the party. 498 | 499 | If you convey a covered work, knowingly relying on a patent license, 500 | and the Corresponding Source of the work is not available for anyone 501 | to copy, free of charge and under the terms of this License, through a 502 | publicly available network server or other readily accessible means, 503 | then you must either (1) cause the Corresponding Source to be so 504 | available, or (2) arrange to deprive yourself of the benefit of the 505 | patent license for this particular work, or (3) arrange, in a manner 506 | consistent with the requirements of this License, to extend the patent 507 | license to downstream recipients. "Knowingly relying" means you have 508 | actual knowledge that, but for the patent license, your conveying the 509 | covered work in a country, or your recipient's use of the covered work 510 | in a country, would infringe one or more identifiable patents in that 511 | country that you have reason to believe are valid. 512 | 513 | If, pursuant to or in connection with a single transaction or 514 | arrangement, you convey, or propagate by procuring conveyance of, a 515 | covered work, and grant a patent license to some of the parties 516 | receiving the covered work authorizing them to use, propagate, modify 517 | or convey a specific copy of the covered work, then the patent license 518 | you grant is automatically extended to all recipients of the covered 519 | work and works based on it. 520 | 521 | A patent license is "discriminatory" if it does not include within 522 | the scope of its coverage, prohibits the exercise of, or is 523 | conditioned on the non-exercise of one or more of the rights that are 524 | specifically granted under this License. You may not convey a covered 525 | work if you are a party to an arrangement with a third party that is 526 | in the business of distributing software, under which you make payment 527 | to the third party based on the extent of your activity of conveying 528 | the work, and under which the third party grants, to any of the 529 | parties who would receive the covered work from you, a discriminatory 530 | patent license (a) in connection with copies of the covered work 531 | conveyed by you (or copies made from those copies), or (b) primarily 532 | for and in connection with specific products or compilations that 533 | contain the covered work, unless you entered into that arrangement, 534 | or that patent license was granted, prior to 28 March 2007. 535 | 536 | Nothing in this License shall be construed as excluding or limiting 537 | any implied license or other defenses to infringement that may 538 | otherwise be available to you under applicable patent law. 539 | 540 | 12. No Surrender of Others' Freedom. 541 | 542 | If conditions are imposed on you (whether by court order, agreement or 543 | otherwise) that contradict the conditions of this License, they do not 544 | excuse you from the conditions of this License. If you cannot convey a 545 | covered work so as to satisfy simultaneously your obligations under this 546 | License and any other pertinent obligations, then as a consequence you may 547 | not convey it at all. For example, if you agree to terms that obligate you 548 | to collect a royalty for further conveying from those to whom you convey 549 | the Program, the only way you could satisfy both those terms and this 550 | License would be to refrain entirely from conveying the Program. 551 | 552 | 13. Use with the GNU Affero General Public License. 553 | 554 | Notwithstanding any other provision of this License, you have 555 | permission to link or combine any covered work with a work licensed 556 | under version 3 of the GNU Affero General Public License into a single 557 | combined work, and to convey the resulting work. The terms of this 558 | License will continue to apply to the part which is the covered work, 559 | but the special requirements of the GNU Affero General Public License, 560 | section 13, concerning interaction through a network will apply to the 561 | combination as such. 562 | 563 | 14. Revised Versions of this License. 564 | 565 | The Free Software Foundation may publish revised and/or new versions of 566 | the GNU General Public License from time to time. Such new versions will 567 | be similar in spirit to the present version, but may differ in detail to 568 | address new problems or concerns. 569 | 570 | Each version is given a distinguishing version number. If the 571 | Program specifies that a certain numbered version of the GNU General 572 | Public License "or any later version" applies to it, you have the 573 | option of following the terms and conditions either of that numbered 574 | version or of any later version published by the Free Software 575 | Foundation. If the Program does not specify a version number of the 576 | GNU General Public License, you may choose any version ever published 577 | by the Free Software Foundation. 578 | 579 | If the Program specifies that a proxy can decide which future 580 | versions of the GNU General Public License can be used, that proxy's 581 | public statement of acceptance of a version permanently authorizes you 582 | to choose that version for the Program. 583 | 584 | Later license versions may give you additional or different 585 | permissions. However, no additional obligations are imposed on any 586 | author or copyright holder as a result of your choosing to follow a 587 | later version. 588 | 589 | 15. Disclaimer of Warranty. 590 | 591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 599 | 600 | 16. Limitation of Liability. 601 | 602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 610 | SUCH DAMAGES. 611 | 612 | 17. Interpretation of Sections 15 and 16. 613 | 614 | If the disclaimer of warranty and limitation of liability provided 615 | above cannot be given local legal effect according to their terms, 616 | reviewing courts shall apply local law that most closely approximates 617 | an absolute waiver of all civil liability in connection with the 618 | Program, unless a warranty or assumption of liability accompanies a 619 | copy of the Program in return for a fee. 620 | 621 | END OF TERMS AND CONDITIONS 622 | 623 | How to Apply These Terms to Your New Programs 624 | 625 | If you develop a new program, and you want it to be of the greatest 626 | possible use to the public, the best way to achieve this is to make it 627 | free software which everyone can redistribute and change under these terms. 628 | 629 | To do so, attach the following notices to the program. It is safest 630 | to attach them to the start of each source file to most effectively 631 | state the exclusion of warranty; and each file should have at least 632 | the "copyright" line and a pointer to where the full notice is found. 633 | 634 | 635 | Copyright (C) 636 | 637 | This program is free software: you can redistribute it and/or modify 638 | it under the terms of the GNU General Public License as published by 639 | the Free Software Foundation, either version 3 of the License, or 640 | (at your option) any later version. 641 | 642 | This program is distributed in the hope that it will be useful, 643 | but WITHOUT ANY WARRANTY; without even the implied warranty of 644 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 645 | GNU General Public License for more details. 646 | 647 | You should have received a copy of the GNU General Public License 648 | along with this program. If not, see . 649 | 650 | Also add information on how to contact you by electronic and paper mail. 651 | 652 | If the program does terminal interaction, make it output a short 653 | notice like this when it starts in an interactive mode: 654 | 655 | Copyright (C) 656 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 657 | This is free software, and you are welcome to redistribute it 658 | under certain conditions; type `show c' for details. 659 | 660 | The hypothetical commands `show w' and `show c' should show the appropriate 661 | parts of the General Public License. Of course, your program's commands 662 | might be different; for a GUI interface, you would use an "about box". 663 | 664 | You should also get your employer (if you work as a programmer) or school, 665 | if any, to sign a "copyright disclaimer" for the program, if necessary. 666 | For more information on this, and how to apply and follow the GNU GPL, see 667 | . 668 | 669 | The GNU General Public License does not permit incorporating your program 670 | into proprietary programs. If your program is a subroutine library, you 671 | may consider it more useful to permit linking proprietary applications with 672 | the library. If this is what you want to do, use the GNU Lesser General 673 | Public License instead of this License. But first, please read 674 | . 675 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Swoon Detector 2 | --- 3 | This repository is conducted by Lee et al. as a final result of the AI Grand Challenge([Homepage](https://ai-challenge.kr)). 4 | Our task is to detect a fallen person who needed help. We got 3rd place in this challenge. We created this repository to share our results. 5 | 6 | ![image1](./source/falldown.png) 7 | 8 | #### install requirements.txt 9 | `pip install -r requirements.txt` 10 | 11 | #### execute predict.py 12 | you can test our model by executing `python predict.py `. 13 | 14 | [input] \ 15 | `` must follow that below file directory structure 16 | 17 | ~~~ 18 | /your dataset path 19 | /video_1 20 | /video_1_001.jpg 21 | /video_2_002.jpg 22 | /... 23 | /video_2 24 | /video_2_001.jpg 25 | /video_2_002.jpg 26 | /... 27 | 28 | ~~~ 29 | you don't have to follow file name, but maintain file structure 30 | 31 | 32 | [output] 33 | After inference, you can get a json file. 34 | The structure of the json file is as follows. 35 | 36 | ~~~ 37 | { 38 | 'annotations':[ 39 | { 40 | 'file_name': 'video_1_001.jpg', # if two or more swoon people exist in frame 41 | 'box':[{ 42 | 'position':[150, 150, 300, 300], # xyxy 43 | 'confidence_score': '0.9987' 44 | }, 45 | { 46 | 'position':[560, 560, 900, 900], 47 | 'confidence_score': '0.98' 48 | }] 49 | }, 50 | { 51 | 'file_name': 'video_1_002.jpg', # if no swoon person in frame 52 | 'box': [] 53 | }, 54 | { 55 | 'file_name': 'video_1_003.jpg', # if just one swoon person in frame 56 | 'box':[{ 57 | 'position': [200, 200, 400, 400] # xyxy 58 | 'confidence_score': '0.899' 59 | }] 60 | } 61 | ] 62 | } 63 | ~~~ 64 | 65 | you can make video using generated json file\ 66 | example\ 67 | ![swoon_person](./source/swoon_person.gif) 68 | 69 | dataset link : https://drive.google.com/drive/folders/1JfEMxKb70GSEEUKMBqr62UFOsMbpPK8s?usp=sharing 70 | 71 | ### TODO 72 | - trainable code 73 | 74 | -------------------------------------------------------------------------------- /classifier_ns_2/20201211/efficientnetb0_ns_fit_normal_vs_falldown_stage1stage2_normal_vs_falldown_beforeafterjangdae_4_99.27884615384616_False_True.pt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DASH-Lab/SwoonDetector/7ba2a9fe9697847f1e9bf9611b2829837e5b4a94/classifier_ns_2/20201211/efficientnetb0_ns_fit_normal_vs_falldown_stage1stage2_normal_vs_falldown_beforeafterjangdae_4_99.27884615384616_False_True.pt -------------------------------------------------------------------------------- /classifier_ns_2/module_2class.py: -------------------------------------------------------------------------------- 1 | import timm 2 | import torch 3 | import torchvision.transforms as transforms 4 | import numpy as np 5 | import matplotlib.pyplot as plt 6 | 7 | class classifier_sub: 8 | def __init__(self, model_path1, model_path2, input_size = 128,back_vs_person_padding = False, 9 | back_vs_person_normalize = False, normal_vs_falldown_padding = False, 10 | normal_vs_falldown_normalize = False): 11 | 12 | ''' 13 | model path doesn't be used in this script 14 | ''' 15 | self.device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu") 16 | self.model2 = timm.create_model('tf_efficientnet_b0_ns', pretrained=True, num_classes=2) 17 | 18 | self.model2.load_state_dict(torch.load(model_path2)) 19 | self.model2.to(self.device) 20 | self.model2.eval() 21 | 22 | self.input_size = input_size 23 | 24 | 25 | 26 | transform_normal_vs_falldown = [] 27 | transform_normal_vs_falldown.append(transforms.ToTensor()) 28 | if normal_vs_falldown_normalize: 29 | transform_normal_vs_falldown.append(transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])) 30 | if normal_vs_falldown_padding: 31 | transform_normal_vs_falldown.append( 32 | lambda x: transforms.Pad(((128 - x.shape[2]) // 2, (128 - x.shape[1]) // 2), fill=0, 33 | padding_mode="constant")(x)) 34 | transform_normal_vs_falldown.append(transforms.Resize((input_size, input_size))) 35 | self.transform_normal_vs_falldown = transforms.Compose(transform_normal_vs_falldown) 36 | print("normal_vs_falldown: {}".format(self.transform_normal_vs_falldown)) 37 | 38 | def predict(self, input_): 39 | 40 | len_input = len(input_) 41 | PERSON = 1 42 | BACKGROUND = 2 43 | patch_normal_vs_falldown = torch.empty(len_input, 3, self.input_size, self.input_size) 44 | for n, img in enumerate(input_): 45 | try: 46 | img_normal_vs_falldown = self.transform_normal_vs_falldown(img.copy()) 47 | patch_normal_vs_falldown[n] = img_normal_vs_falldown 48 | except: 49 | patch_normal_vs_falldown[n] = torch.zeros((3, self.input_size, self.input_size)) 50 | patch_normal_vs_falldown = patch_normal_vs_falldown.to(self.device) 51 | with torch.no_grad(): 52 | try: 53 | falldown_or_normal_preds = self.model2(patch_normal_vs_falldown) 54 | _, falldown_or_normal = torch.max(falldown_or_normal_preds, -1) 55 | return falldown_or_normal 56 | except: 57 | return [] 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | -------------------------------------------------------------------------------- /classifier_ns_2/predict_2class.py: -------------------------------------------------------------------------------- 1 | import matplotlib.pyplot as plt 2 | import numpy as np 3 | import os 4 | os.environ["CUDA_VISIBLE_DEVICES"] ="2" 5 | import glob 6 | import cv2 7 | import time 8 | from tqdm import tqdm 9 | from jeongho.module_2class import classifier 10 | 11 | 12 | 13 | ''''' 14 | Not used in this project, 15 | ''''' 16 | 17 | img_paths = glob.glob("./test_img(test_falldown)/*") 18 | img_paths.sort() 19 | classifier = classifier(model_path1 = "./best_model/efficientnetb0_ns_fit_background_vs_person_stage1stage2_back_vs_person_7_97.11021505376344_False_False.pt", 20 | model_path2 = "./best_model/efficientnetb0_ns_fit_normal_vs_falldown_stage1stage2_normal_vs_falldown_3_93.26923076923077_False_False.pt", 21 | back_vs_person_padding = False, back_vs_person_normalize = False, 22 | normal_vs_falldown_padding = False, normal_vs_falldown_normalize = False) 23 | 24 | results = [] 25 | count = [0,0,0] 26 | start = time.time() 27 | for n, path in tqdm(enumerate(img_paths)): 28 | img = cv2.imread(path) 29 | img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) 30 | img = img[np.newaxis, :, :, :] 31 | result = classifier.predict(img) 32 | #result.append((path, result_dict[0]["class"], result_dict[0]["confidence"])) 33 | results.append(result) 34 | #count[result_dict[0]["class"]] += 1 35 | #print("time : {}".format(time.time() - start)) 36 | print(results) 37 | #print(count) 38 | -------------------------------------------------------------------------------- /predict.py: -------------------------------------------------------------------------------- 1 | import glob 2 | import os 3 | # os.system('python3 detection_module.py') 4 | os.environ['CUDA_VISIBLE_DEVICES'] = '0' 5 | import time 6 | # from detection_module import * 7 | from yolov5_test.detection_module import * 8 | from classifier_ns_2.module_2class import * 9 | # from classifier_ensemble.module_ensem_2class import * 10 | # from mmdetection_module.detection_module import * 11 | from itertools import product 12 | import torch 13 | import cv2 14 | import json 15 | import pprint 16 | import numpy as np 17 | import imageio 18 | GET_EVERY = 3 19 | FRAME_RATE = 15 20 | IOU_THRESHOLD = 0.2 21 | IOU_NMS_THRESHOLD = 0.4 22 | CONF_THRESHOLD = 0.22 23 | classifier = None 24 | detector = None 25 | 26 | 27 | 28 | class SwoonTracker: 29 | def __init__(self, bbox_coord, filename, initial_frame, swoon_conf): 30 | self.bboxes = [bbox_coord] 31 | self.class_list = [1] 32 | self.filenames = [filename] 33 | self.initial_frame = initial_frame 34 | self.swoon_confidence = [swoon_conf] 35 | self.index_list = [initial_frame] 36 | self.last_bbox = bbox_coord 37 | self.is_swoon = False 38 | self.end_frame = initial_frame 39 | self.swoon_by_end = False 40 | self.sum_frame = np.array(bbox_coord) 41 | self.average_frame = np.array(bbox_coord) 42 | self.swoon_count = 1 43 | 44 | def append(self, bbox_coord, class_result, index, filename, conf): 45 | if len(bbox_coord) != 0: 46 | self.last_bbox = bbox_coord 47 | self.swoon_count += 1 48 | self.sum_frame += np.array(bbox_coord) 49 | #print('sumframe', self.sum_frame) 50 | self.average_frame = self.sum_frame.astype(np.int32) / self.swoon_count 51 | #print('average frame', self.average_frame) 52 | self.bboxes.append(bbox_coord) 53 | self.class_list.append(class_result) 54 | self.index_list.append(index) 55 | self.filenames.append(filename) 56 | self.swoon_confidence.append(conf) 57 | 58 | def __str__(self): 59 | return "[Tracker Print] \n index_list: {} \n bboxes: {}".format(self.index_list, self.bboxes) 60 | def analysis_tracker(tracker_list, image_list): 61 | global detector, classifier 62 | image_list = np.array(image_list) 63 | output_result = [[] for i in range(len(image_list))] 64 | output_confidence = [[] for i in range(len(image_list))] 65 | ELIMINATE_FRAMES = 6 66 | for idx, tracker in enumerate(tracker_list): 67 | 68 | start_index = final_index = tracker.initial_frame 69 | swoon_count = 0 70 | end_index = 0 71 | 72 | for id, (class_, index) in enumerate(zip(tracker.class_list, tracker.index_list)): 73 | if class_ == 1: 74 | final_index = index 75 | swoon_count += 1 76 | end_index = id 77 | if final_index - start_index <= FRAME_RATE * 9: # 9초 이하로 쓰러진 상태면 기각 78 | continue 79 | 80 | swoon_index = tracker.index_list[:end_index+1] #[255, 265, 275, ...] 81 | swoon_class = tracker.class_list[:end_index+1] # [1, 1, 1, ...] 82 | swoon_confidence = tracker.swoon_confidence[:end_index+1] 83 | 84 | if sum(swoon_class) < 0.35 * len(swoon_class): # "쓰러짐 구간 중" 쓰러진 사람이 35% 이하일 경우 기각 85 | continue 86 | 87 | # width or height 가 너무 작은 경우 예외 처리 88 | init_width = tracker.bboxes[0][2] - tracker.bboxes[0][0] 89 | init_height = tracker.bboxes[0][3] - tracker.bboxes[0][1] 90 | if min(init_width, init_height) < 25: 91 | print("0 box cut") 92 | continue 93 | prev_coordinate = tracker.bboxes[0] # 가장 처음 디텍트된 bounding box 94 | count = 0 95 | # print(tracker.bboxes[:len(swoon_class)]) 96 | prev_confidence = tracker.swoon_confidence[0] 97 | for idx_, (class_, coordinate, conf) in enumerate(zip(swoon_class, tracker.bboxes[:len(swoon_class)], swoon_confidence)): # 빈 bbox가 append 되었으면, 앞의 bbox coordinate으로 값을 채운다. 98 | if idx_ == 0: continue 99 | if class_ == 0: 100 | count += 1 101 | else: 102 | if count > 0: 103 | diff_coordinate = (np.array(coordinate, dtype=np.float) - np.array(prev_coordinate, dtype=np.float)) 104 | diff_confidence = conf - prev_confidence 105 | for ii, box in enumerate(tracker.bboxes[idx_-count:idx_]): 106 | tracker.bboxes[idx_-count+ii] = (np.array(prev_coordinate, dtype=np.float) + ((ii+1) / count) * diff_coordinate).astype(np.int).tolist() 107 | tracker.swoon_confidence[idx_-count+ii] = prev_confidence + ((ii+1) / count) * diff_confidence 108 | count = 0 109 | prev_coordinate = coordinate 110 | prev_confidence = conf 111 | # print("revise -->",tracker.bboxes[:len(swoon_class)]) 112 | 113 | swoon_class_ = [1] * len(swoon_class) 114 | tracker_list[idx].class_list[:end_index+1] = swoon_class_ # 11100111 --> 11111111 115 | 116 | #Find first swoon 117 | prev_frame = swoon_index[0] - GET_EVERY + 1 118 | 119 | 120 | index_list = list(range(prev_frame, swoon_index[0])) 121 | 122 | 123 | def get_first_swoon(index_list): #이진탐색으로 최초 쓰러진 위치를 탐색 124 | pivot = (len(index_list) - 1) // 2 125 | 126 | out = detector.predict(cv2.imread(image_list[index_list[pivot]]), IOU_NMS_THRESHOLD, CONF_THRESHOLD) 127 | patch_images = out['img'] 128 | coordinates = out['label'] 129 | 130 | output_class = classifier.predict(patch_images) 131 | swoon_coords = [] 132 | for coord, class_ in zip(coordinates, output_class): # 한 프레임 안에 쓰러진 사람 좌표 append 133 | if class_ == 1: # swoon case 134 | swoon_coords.append(coord) 135 | max_iou = -1 136 | for swoon_coord in swoon_coords: 137 | iou = cal_iou(swoon_coord, tracker.bboxes[0]) 138 | if max_iou < iou: 139 | max_iou = iou 140 | if len(index_list) == 1: 141 | if max_iou < IOU_THRESHOLD: 142 | return index_list[0] + 1 143 | else: 144 | return index_list[0] 145 | else: 146 | if max_iou < IOU_THRESHOLD: 147 | return get_first_swoon(index_list[pivot+1:]) 148 | else: 149 | return get_first_swoon(index_list[:pivot+1]) 150 | 151 | def get_last_swoon(index_list): # 이진탐색으로 최초 쓰러진 위치를 탐색 152 | 153 | pivot = (len(index_list)-1) // 2 154 | out = detector.predict(cv2.imread(image_list[index_list[pivot]]), IOU_NMS_THRESHOLD, CONF_THRESHOLD) 155 | patch_images = out['img'] 156 | coordinates = out['label'] 157 | output_class = classifier.predict(patch_images) 158 | swoon_coords = [] 159 | for coord, class_ in zip(coordinates, output_class): # 한 프레임 안에 쓰러진 사람 좌표 append 160 | if class_ == 1: # swoon case 161 | swoon_coords.append(coord) 162 | max_iou = -1 163 | match_coordinate = None 164 | for swoon_coord in swoon_coords: 165 | iou = cal_iou(swoon_coord, tracker.bboxes[0]) 166 | if max_iou < iou: 167 | max_iou = iou 168 | if len(index_list): 169 | if max_iou < IOU_THRESHOLD: 170 | return index_list[0] - 1 171 | else: 172 | return index_list[0] 173 | else: 174 | if max_iou < IOU_THRESHOLD: 175 | get_last_swoon(index_list[:pivot+1]) 176 | else: 177 | get_last_swoon(index_list[pivot+1:]) 178 | 179 | first_swoon_index = get_first_swoon(index_list) # 첫번째 쓰러진 위치를 가져옴. 180 | if first_swoon_index == 1: 181 | first_swoon_index = 0 182 | 183 | 184 | 185 | last_frame = swoon_index[-1] 186 | if last_frame > tracker.index_list[-3]: # 쓰러짐이 동영상 끝까지 지속될 경우 187 | tracker_list[idx].swoon_by_end = True 188 | last_swoon_index = len(image_list)-1 189 | else: 190 | last_next_frame = last_frame + GET_EVERY 191 | last_index_list = list(range(last_frame+1, last_next_frame)) 192 | last_swoon_index = get_last_swoon(last_index_list) 193 | 194 | 195 | swoon_section = list(range(first_swoon_index, last_swoon_index+1)) 196 | first_swoon_coord = tracker.bboxes[0] 197 | first_swoon_conf = tracker.swoon_confidence[0] 198 | if first_swoon_index != tracker.index_list[0]: 199 | out = detector.predict(cv2.imread(image_list[first_swoon_index]), IOU_NMS_THRESHOLD, CONF_THRESHOLD) 200 | coordinates = out['label'] 201 | confs = out['score'] 202 | max_iou = -1 203 | real_first_swoon_coord = tracker.bboxes[0] 204 | real_first_swoon_conf = 0 205 | for coordinate in coordinates: 206 | iou = cal_iou(coordinate, first_swoon_coord) 207 | if max_iou < iou: 208 | max_iou = iou 209 | real_first_swoon_conf = first_swoon_conf 210 | real_first_swoon_coord = coordinate 211 | else: 212 | real_first_swoon_coord = first_swoon_coord 213 | real_first_swoon_conf = first_swoon_conf 214 | 215 | swoon_index_list = tracker.index_list[:end_index+1] 216 | swoon_boxes = tracker.bboxes[:end_index+1] 217 | swoon_confs = tracker.swoon_confidence[:end_index+1] 218 | i = 0 219 | ccount = 1 220 | if swoon_section[0] != swoon_index_list[0]: 221 | ccount = swoon_index_list[0] - swoon_section[0] + 1 222 | for idx_1, swoon_sec in enumerate(swoon_section): 223 | #print(swoon_sec, swoon_index_list[i], swoon_index_list[0], swoon_index_list[-1]) 224 | if swoon_index_list[0] > swoon_sec: 225 | remain_count = swoon_index_list[0] - swoon_sec # 5 4 3 2 1 226 | diff_ = (np.array(first_swoon_coord, dtype=np.float) - np.array(real_first_swoon_coord, dtype=np.float)) 227 | output_result[swoon_sec].append(np.round(np.array(real_first_swoon_coord) + (((ccount - remain_count) / ccount) * diff_)).astype(np.int).tolist()) 228 | diff_swoon = first_swoon_conf - real_first_swoon_conf 229 | output_confidence[swoon_sec].append(real_first_swoon_conf + ((ccount - remain_count) / ccount) * diff_swoon) 230 | elif swoon_index_list[0] <= swoon_sec < swoon_index_list[-1]: 231 | first_box = swoon_boxes[i] 232 | next_box = swoon_boxes[i+1] 233 | first_conf = swoon_confs[i] 234 | next_conf = swoon_confs[i+1] 235 | # print(swoon_boxes, next_box) 236 | diff = (np.array(next_box, dtype=np.float) - np.array(first_box, dtype=np.float)) 237 | output_box = (np.round(np.array(first_box) + (((swoon_sec - swoon_index_list[i])/GET_EVERY) * diff))).astype(np.int).tolist() 238 | output_result[swoon_sec].append(output_box) 239 | 240 | diff_conf = next_conf - first_conf 241 | output_confidence[swoon_sec].append(first_conf + ((swoon_sec - swoon_index_list[i]) / GET_EVERY) * diff_conf) 242 | #print('hell ,,',swoon_sec, swoon_index_list[i], swoon_index_list[0], swoon_index_list[-1]) 243 | if swoon_sec == swoon_index_list[i+1]: 244 | i += 1 245 | else: 246 | output_result[swoon_sec].append(swoon_boxes[-1]) 247 | output_confidence[swoon_sec].append(swoon_confs[-1]) 248 | 249 | if swoon_section[0] > 10: # 쓰러진 시작점이 영상의 초반 부분이 아니면 아래 작업 수행 250 | for idx_1, swoon_sec in enumerate(swoon_section[:ELIMINATE_FRAMES]): # 처음 몇 프레임은 버리는 프레임 251 | output_result[swoon_sec].pop() 252 | output_confidence[swoon_sec].pop() 253 | 254 | 255 | return output_result, output_confidence 256 | 257 | def cal_iou(boxA, boxB): 258 | # determine the (x, y)-coordinates of the intersection rectangle 259 | xA = max(boxA[0], boxB[0]) 260 | yA = max(boxA[1], boxB[1]) 261 | xB = min(boxA[2], boxB[2]) 262 | yB = min(boxA[3], boxB[3]) 263 | # compute the area of intersection rectangle 264 | interArea = max(0, xB - xA + 1) * max(0, yB - yA + 1) 265 | # compute the area of both the prediction and ground-truth 266 | # rectangles 267 | boxAArea = (boxA[2] - boxA[0] + 1) * (boxA[3] - boxA[1] + 1) 268 | boxBArea = (boxB[2] - boxB[0] + 1) * (boxB[3] - boxB[1] + 1) 269 | # compute the intersection over union by taking the intersection 270 | # area and dividing it by the sum of prediction + ground-truth 271 | # areas - the interesection area 272 | iou = interArea / float(boxAArea + boxBArea - interArea) 273 | # return the intersection over union value 274 | return iou 275 | 276 | 277 | # To get more score in this competition, we used heuristic rules that classify whether some person is swwon or not by using witdh, height informations. 278 | class classifier_h: 279 | def __init__(self, ratio_threshold): 280 | self.ratio_threshold = ratio_threshold 281 | 282 | def predict(self, imgs): 283 | output_list = [] 284 | for img in imgs: 285 | h, w, _ = img.shape 286 | if w / h > self.ratio_threshold: 287 | output_list.append(1) 288 | else: 289 | output_list.append(0) 290 | return output_list 291 | 292 | def main(): 293 | global classifier, detector 294 | #global CONF_THRESHOLD, IOU_NMS_THRESHOLD 295 | # print("torch version:", torch.__version__) 296 | # print("usable cuda:", torch.cuda.is_available()) 297 | # # OUTPUT_FILENAME = './test_iitp_detector_nms_test/t1_res_nms055_high_iou_minus_conf.json' 298 | test_folder = sys.argv[1] 299 | #test_folder = '/media/data2/AGC/IITP_Track01_Sample' 300 | #test_folder = '/media/data2/AGC_system_test/validation_data' 301 | # print("test folder exist:", os.path.exists(test_folder)) 302 | assert os.path.exists(test_folder) 303 | # prin = pprint.PrettyPrinter(indent=3) 304 | # print("test folder len:", os.listdir(test_folder)) 305 | # print("The number of frames to infer:", sum([len(os.listdir(frames)) for frames in glob.glob(os.path.join(test_folder, "*"))])) 306 | 307 | #video_list = sorted(glob.glob(os.path.join(test_folder, "*"))) 308 | 309 | # if you guys wanna use original classifier to get exact result, use below classifier 310 | 311 | # classifier = classifier_( 312 | # model_path1="./classifier_ns_2/1114_1110/bg.pt", 313 | # model_path2="./classifier_ns_2/1114_1110/falldown.pt", 314 | # back_vs_person_padding=False, back_vs_person_normalize=False, normal_vs_falldown_padding=False, 315 | # normal_vs_falldown_normalize=True) 316 | 317 | video_list = sorted(glob.glob(os.path.join(test_folder, "*"))) 318 | basepath = os.path.dirname(os.path.realpath(__file__)) 319 | classifier_sub_ = classifier_sub( 320 | model_path1= os.path.join(basepath, "classifier_ns_2/20201118/efficientnetb0_ns_fit_background_vs_person_test_ES_stage1stage2_back_vs_person_6_97.71505376344086_False_True.pt"), 321 | model_path2= os.path.join(basepath, "classifier_ns_2/20201211/efficientnetb0_ns_fit_normal_vs_falldown_stage1stage2_normal_vs_falldown_beforeafterjangdae_4_99.27884615384616_False_True.pt"), 322 | back_vs_person_padding=False, back_vs_person_normalize=True, normal_vs_falldown_padding=False, 323 | normal_vs_falldown_normalize=True) 324 | 325 | # classifier = classifier_(model_path=os.path.join(basepath, "classifier_ensemble/bestweight_1118")) 326 | classifier = classifier_h(0.7) 327 | ''' 328 | We used heuristic classifier to get more score, and could get biggest score. 329 | but, if you wanna use deep-learning based model , you can revise our code. 330 | ''' 331 | 332 | 333 | detector = HumanDetector(os.path.join(basepath, 'yolov5_test/weight/last.pt')) 334 | OUTPUT_FILENAME = os.path.join(basepath, 't1_res_U0000000302.json') 335 | 336 | output_json = {'annotations':[]} 337 | 338 | for idx, video in enumerate(video_list): 339 | # image_list = sorted(glob.glob(os.path.join(video, "*.*"))) 340 | image_list = sorted(glob.glob(os.path.join(video, "*.*"))) 341 | tracker_list = [] 342 | start_video = time.time() 343 | print(video) 344 | folder_name = video.split("/")[-1] 345 | #folder_name = video_index[idx] 346 | flag = False 347 | for idx, image in enumerate(image_list): 348 | 349 | if idx == 0: continue 350 | #if idx % GET_EVERY != 0: continue 351 | if idx % GET_EVERY != 0: continue 352 | 353 | if not flag: 354 | sub_images = [image_list[len(image_list) // 2 - 15], image_list[len(image_list) // 2 - 7], image_list[len(image_list) // 2 ], image_list[len(image_list) // 2 + 7], image_list[len(image_list) // 2 + 15]] 355 | 356 | for sub in sub_images: 357 | sub_frame = cv2.imread(sub) 358 | sub_out = detector.predict(sub_frame, IOU_NMS_THRESHOLD, CONF_THRESHOLD) 359 | sub_patch_images = sub_out['img'] 360 | patch_images = [img[:, :, ::-1] for img in sub_patch_images] 361 | output_class = classifier_sub_.predict(patch_images) 362 | if sum(output_class) > 0: 363 | flag = True 364 | 365 | if not flag: 366 | break 367 | 368 | frame = cv2.imread(image) 369 | out = detector.predict(frame, IOU_NMS_THRESHOLD, CONF_THRESHOLD) 370 | patch_images = out['img'] 371 | coordinates = out['label'] # 확인 완료 372 | confidences = out['score'] 373 | patch_images = [img[:, :, ::-1] for img in patch_images] 374 | output_class = classifier.predict(patch_images) 375 | 376 | 377 | swoon_coords = [] 378 | swoon_confidence = [] 379 | for id, (coord, class_, conf) in enumerate(zip(coordinates, output_class, confidences)): # 한 프레임 안에 쓰러진 사람 좌표 append 380 | if class_ == 1: # swoon case 381 | swoon_coords.append(coord) 382 | swoon_confidence.append(conf) 383 | 384 | 385 | if len(tracker_list) == 0: # 동영상 속 쓰러진 사람(들)이 처음 발견되면, tracker 생성 386 | for swoon_coord, swoon_conf in zip(swoon_coords, swoon_confidence): 387 | tracker_list.append(SwoonTracker(swoon_coord, image, idx, swoon_conf)) 388 | else: 389 | if len(swoon_coords) == 0: #tracker가 있지만, 쓰러진 사람이 탐지되지 않을 경우 390 | for i, tracker in enumerate(tracker_list): 391 | tracker_list[i].append([], 0, idx, image, 0) # 탐지되지 않은 정보를 모든 tracker에 append 392 | else: # tracker 가 있고, 쓰러진 사람이 탐지될 경우 393 | swoon_matching = [False] * len(swoon_coords) # 쓰러짐 좌표가 매칭이 되면 True로 변경 394 | tracker_matching = [False] * len(tracker_list) 395 | for i, (swoon_coord, swoon_conf) in enumerate(zip(swoon_coords, swoon_confidence)): 396 | max_iou = -1 397 | max_index = -1 398 | for j, tracker in enumerate(tracker_list): 399 | # print(tracker.average_frame.tolist(), swoon_coord) 400 | get_iou = cal_iou(tracker.bboxes[0], swoon_coord) 401 | if max_iou < get_iou and not tracker_matching[j]: 402 | max_iou = get_iou 403 | max_index = j 404 | if max_iou > IOU_THRESHOLD: 405 | swoon_matching[i] = True 406 | tracker_list[max_index].append(swoon_coord, 1, idx, image, swoon_conf) 407 | tracker_matching[max_index] = True 408 | for z, track_bool in enumerate(tracker_matching): 409 | if not track_bool: 410 | tracker_list[z].append([], 0, idx, image, 0) 411 | for match_result, swoon, swoon_conf in zip(swoon_matching, swoon_coords, swoon_confidence): # 매칭되지 않은 쓰러진 좌표가 있다면, 그 좌표를 시작점으로 새로운 Tracker 생성 412 | if not match_result: 413 | tracker_list.append(SwoonTracker(swoon, image, idx, swoon_conf)) 414 | 415 | if flag: 416 | print('Detected') 417 | else: 418 | for image_name in image_list: 419 | file_dict = { 420 | 'file_name': image_name.split("/")[-1], 421 | 'box': [] 422 | } 423 | output_json['annotations'].append(file_dict) 424 | print('NotDetected') 425 | continue 426 | print("time to infer on one video:", time.time() - start_video) 427 | final_decision, final_decision_confidence = analysis_tracker(tracker_list, image_list) 428 | 429 | 430 | for img_out, tracker_out, conf_out in zip(image_list, final_decision, final_decision_confidence): 431 | if len(tracker_out) == 0: 432 | file_dict = { 433 | 'file_name': img_out.split("/")[-1], 434 | 'box': [] 435 | } 436 | else: 437 | file_dict = { 438 | 'file_name': img_out.split("/")[-1], 439 | 'box': [] 440 | } 441 | for cd, cf in zip(tracker_out, conf_out): 442 | box_dict = { 443 | 'position': cd, 444 | 'confidence_score': str(cf) 445 | } 446 | file_dict['box'].append(box_dict) 447 | output_json['annotations'].append(file_dict) 448 | 449 | with open(OUTPUT_FILENAME, 'w') as f: 450 | json.dump(output_json, f) 451 | 452 | if __name__ == "__main__": 453 | main() 454 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | torch 2 | torchvision 3 | opencv-python 4 | tqdm 5 | Pillow 6 | numpy 7 | matplotlib 8 | timm 9 | scipy 10 | pyyaml 11 | ensemble_boxes 12 | 13 | -------------------------------------------------------------------------------- /source/falldown.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DASH-Lab/SwoonDetector/7ba2a9fe9697847f1e9bf9611b2829837e5b4a94/source/falldown.png -------------------------------------------------------------------------------- /source/swoon_person.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/DASH-Lab/SwoonDetector/7ba2a9fe9697847f1e9bf9611b2829837e5b4a94/source/swoon_person.gif -------------------------------------------------------------------------------- /utils/README.txt: -------------------------------------------------------------------------------- 1 | split_each_frame: Each video split into each frame. 2 | xml_to_txt: txts that follow coco dataset rule is obtained by executing this script, xmls written by labelImg program 3 | txt_to_xml: If you guys have only txt data and wanna visualize that, you can get xml files which are needed to see in labelImg by using this script. 4 | -------------------------------------------------------------------------------- /utils/split_each_frame.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | import glob 3 | import os 4 | import imageio 5 | # from PIL import Image 6 | VIDEO_PATH = glob.glob('/media/data2/AGC_system_test/VIDEO_RAW_NIGHT/*.MP4') 7 | TO_SAVE_PATH = '/media/data2/AGC_system_test/validation_data_night' 8 | os.makedirs(TO_SAVE_PATH, exist_ok=True) 9 | for video in VIDEO_PATH: 10 | frames = imageio.mimread(video, memtest=False) 11 | filename_only = video.split("/")[-1].split(".")[0] 12 | ret = True 13 | print(video) 14 | count = 0 15 | save_dir = os.path.join(TO_SAVE_PATH, filename_only) 16 | os.makedirs(save_dir, exist_ok=True) 17 | 18 | for idx, frame in enumerate(frames): 19 | cv2.imwrite(os.path.join(save_dir, filename_only+" "+str(idx).zfill(3)+".jpg"), frame[:,:,::-1]) 20 | -------------------------------------------------------------------------------- /utils/txt_to_xml.py: -------------------------------------------------------------------------------- 1 | from imageio import mimread 2 | import numpy as np 3 | import cv2 4 | import glob 5 | import os 6 | import xml.etree.cElementTree as ET 7 | import lxml.etree as etree 8 | import json 9 | import matplotlib.pyplot as plt 10 | from YoloDetector import human_detector_2 11 | import shutil 12 | #type_lis = ['assault', 'burglary', 'datefight', 'drunken', 'dump', 'fight', 'kidnap', 'robbery', 'swoon', 'trespass', 'vandalism', 'wander'] 13 | 14 | def txt_to_xml(label, image_saved_path, xml_saved_path, img_filepath): 15 | 16 | f_list = label 17 | 18 | annotation = 'annotation' 19 | folder = 'folder' 20 | filename = 'filename' 21 | path = 'path' 22 | source = 'source' 23 | database = 'database' 24 | size = 'size' 25 | width = 'width' 26 | height = 'height' 27 | depth = 'depth' 28 | segmented = 'segmented' 29 | object = 'object' 30 | name = 'name' 31 | pose = 'pose' 32 | truncated = 'truncated' 33 | difficult = 'difficult' 34 | bndbox = 'bndbox' 35 | xmin = 'xmin' 36 | ymin = 'ymin' 37 | xmax = 'xmax' 38 | ymax = 'ymax' 39 | 40 | anomaly_type = "swoon" 41 | folder_name = "user_generate" 42 | 43 | root_ = ET.Element(annotation) 44 | folder_ = ET.SubElement(root_, folder) 45 | folder_.text = anomaly_type 46 | filename_ = ET.SubElement(root_, filename) 47 | filename_.text = img_filepath 48 | path_ = ET.SubElement(root_, path) 49 | saved_path = "T:" +image_saved_path.replace("/","\\") 50 | 51 | path_.text = saved_path + "\\" + img_filepath 52 | source_ = ET.SubElement(root_, source) 53 | database_ = ET.SubElement(source_, database) 54 | database_.text = 'Unknown' 55 | size_ = ET.SubElement(root_, size) 56 | segmented_ = ET.SubElement(root_, segmented) 57 | segmented_.text = '0' 58 | width_ = ET.SubElement(size_, width) 59 | width_.text = '1920' 60 | height_ = ET.SubElement(size_, height) 61 | height_.text = '1080' 62 | depth_ = ET.SubElement(size_, depth) 63 | depth_.text = '3' 64 | 65 | for line in f_list: 66 | line = line.replace('\n', '') 67 | label = line.split(' ') 68 | 69 | object_ = ET.SubElement(root_, object) 70 | name_ = ET.SubElement(object_, name) 71 | name_.text = '0' 72 | pose_ = ET.SubElement(object_, pose) 73 | pose_.text = 'Unspecified' 74 | truncated_ = ET.SubElement(object_, truncated) 75 | truncated_.text = '0' 76 | difficult_ = ET.SubElement(object_, difficult) 77 | difficult_.text = '0' 78 | bndbox_ = ET.SubElement(object_, bndbox) 79 | xmin_ = ET.SubElement(bndbox_, xmin) 80 | xmin_.text = str(round(((float(label[1]) - float(label[3]) / 2))*1920)) 81 | #xmin_.text = str(label[1]) 82 | ymin_ = ET.SubElement(bndbox_, ymin) 83 | ymin_.text = str(round(((float(label[2]) - float(label[4]) / 2))*1080)) 84 | #ymin_.text = str(label[2]) 85 | xmax_ = ET.SubElement(bndbox_, xmax) 86 | xmax_.text = str(round(((float(label[1]) + float(label[3]) / 2))*1920)) 87 | #xmax_.text = str(label[3]) 88 | ymax_ = ET.SubElement(bndbox_, ymax) 89 | ymax_.text = str(round(((float(label[2]) + float(label[4]) / 2))*1080)) 90 | #ymax_.text = str(label[4]) 91 | tree = ET.ElementTree(root_) 92 | 93 | tree.write(os.path.join(xml_saved_path, img_filepath.split(".")[0] + ".xml")) 94 | pretty = etree.parse(os.path.join(xml_saved_path, img_filepath.split(".")[0] + ".xml")) 95 | pret = etree.tostring(pretty, pretty_print=True) 96 | 97 | #print(pret) 98 | 99 | f = open(os.path.join(xml_saved_path, img_filepath.split(".")[0] + ".xml"), 'wb') 100 | # print(os.path.join(xml_saved_path, img_filepath.split(".")[0] + ".xml")) 101 | 102 | f.write(pret) 103 | f.close() 104 | 105 | def mp4_to_xml(mp4_path, human_detector, extract_every=10, save_path='./'): 106 | #../gopro_4.mp4 107 | assert os.path.exists(mp4_path), 'File Not Found Error' 108 | print('hello') 109 | base_filename = mp4_path.split("/")[-1].split(".")[0] 110 | image_path = os.path.join(save_path, 'image') 111 | label_path = os.path.join(save_path, 'label') 112 | os.makedirs(image_path, exist_ok=True) 113 | os.makedirs(label_path, exist_ok=True) 114 | cap = cv2.VideoCapture(mp4_path) 115 | length = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) 116 | frames = list(range(0, length, extract_every))[10:] 117 | 118 | for idx, frame_no in enumerate(frames): 119 | cap.set(cv2.CAP_PROP_POS_FRAMES, frame_no) 120 | ret, frame = cap.read() 121 | 122 | if not ret or frame is None: 123 | continue 124 | resized_frame = cv2.resize(frame, (1920, 1080)) 125 | ret = human_detector.predict(resized_frame) 126 | 127 | to_save_image_name = base_filename +"_"+str(frame_no).zfill(4) +".jpg" 128 | 129 | 130 | imgs = ret['img'] 131 | coords = ret['label'] 132 | print(coords) 133 | if len(imgs) == 0 or len(coords) == 0: 134 | continue 135 | 136 | cv2.imwrite(os.path.join(image_path, to_save_image_name), resized_frame) 137 | 138 | 139 | yolo_label = [] 140 | for coord in coords: 141 | crd = ["0"] 142 | lis = list(map(str, coord)) 143 | crd.extend(lis) 144 | st = " ".join(crd) 145 | yolo_label.append(st) 146 | print(os.path.join(image_path, to_save_image_name)) 147 | txt_to_xml(yolo_label, image_path, label_path, to_save_image_name) 148 | 149 | 150 | if __name__ == "__main__": 151 | # to_save_path = '/media/data2/AGC/iitp_validation' 152 | # os.makedirs(os.path.join(to_save_path, "images"), exist_ok=True) 153 | # os.makedirs(os.path.join(to_save_path, "labels"), exist_ok=True) 154 | # 155 | # video_frame_folder = '/media/data2/AGC/IITP_Track01_Sample' 156 | # video_label_folder = '/media/data2/AGC/Track1_Result_20201006' 157 | # 158 | # folders = os.listdir(video_frame_folder) 159 | # 160 | # for folder in folders: 161 | # video_paths = sorted(glob.glob(os.path.join(video_frame_folder, folder, "*.jpg"))) 162 | # label_paths = sorted(glob.glob(os.path.join(video_label_folder, folder, "*.json"))) 163 | # to_extract = np.arange(0, len(video_paths), len(video_paths)/10)[1:].astype(np.int) 164 | # video_paths = np.array(video_paths)[to_extract] 165 | # label_paths = np.array(label_paths)[to_extract] 166 | # 167 | # for frame, label in zip(video_paths, label_paths): 168 | # assert frame.split("/")[-1].split(".")[0] == label.split("/")[-1].split(".")[0] 169 | # file = open(label, 'rb') 170 | # json_file = json.load(file) 171 | # shutil.copy(frame, os.path.join(to_save_path, "images")) 172 | # try: 173 | # coord = json_file['box'] 174 | # print('hello') 175 | # coord.insert(0, 0) 176 | # coord = list(map(str, coord)) 177 | # str_coord = " ".join(coord) 178 | # coord = [str_coord] 179 | # txt_to_xml(coord, os.path.join(to_save_path, "images"), os.path.join(to_save_path, "labels"), frame.split("/")[-1].split(".")[0]+".jpg") 180 | # except KeyError: 181 | # continue 182 | 183 | txt_filepath = '/media/data2/AGC/NightData/NightDataCut/labels' 184 | xml_saved_folder = '/media/data2/AGC/NightData/NightDataCut/xmls_reconstruction' 185 | txts = glob.glob(os.path.join(txt_filepath, '*.txt')) 186 | for txt in txts: 187 | print(txt) 188 | f = open(txt, 'r') 189 | readlines = f.readlines() 190 | 191 | txt_to_xml(readlines, '/media/data2/AGC/NightData/NightDataCut/images', '/media/data2/AGC/NightData/NightDataCut/xmls_reconstruction', txt.split("/")[-1].split(".")[0]+".jpg") 192 | 193 | 194 | -------------------------------------------------------------------------------- /utils/xml_to_txt.py: -------------------------------------------------------------------------------- 1 | import glob 2 | import os 3 | import munch 4 | import xmltodict 5 | 6 | def make_txt(txt_save_path, xml_folder): 7 | paths = glob.glob(os.path.join(xml_folder, "*.xml")) 8 | #paths = [path for path in paths if "swoon" in path] 9 | for xml_path in paths: 10 | print(xml_path) 11 | txt_name = xml_path.split("/")[-1].split(".")[0]+".txt" 12 | # print('hello', mp4_name) 13 | file = open(xml_path) 14 | # if os.path.exists(os.path.join(to_save_path, txt_name)): 15 | # continue 16 | save_file = open(os.path.join(txt_save_path, txt_name), 'w') 17 | doc = xmltodict.parse(file.read()) 18 | doc = munch.munchify(doc) 19 | try: 20 | annot = doc.annotation.object 21 | except: 22 | continue 23 | if isinstance(annot, munch.Munch): 24 | annot = [annot] 25 | 26 | for anot in annot: 27 | coord = anot.bndbox 28 | label = anot.name 29 | center_x = ((float(coord.xmax) + float(coord.xmin)) / 2) / 1920.0 30 | center_y = ((float(coord.ymax) + float(coord.ymin)) / 2) / 1080.0 31 | width = (float(coord.xmax) - float(coord.xmin)) / 1920.0 32 | height = (float(coord.ymax) - float(coord.ymin)) / 1080.0 33 | save_file.write('0 {} {} {} {}\n'.format(center_x, center_y, width, height)) 34 | save_file.close() 35 | 36 | 37 | #os.makedirs('/media/data2/AGC/WorkDirectory/complete_data/txt_labels', exist_ok=True) 38 | make_txt('/media/data2/AGC/WorkDirectory2/train/zips/labels', '/media/data2/AGC/WorkDirectory2/train/zips/xmls') 39 | -------------------------------------------------------------------------------- /yolov5_test/AdamW.py: -------------------------------------------------------------------------------- 1 | import math 2 | import torch 3 | from torch.optim.optimizer import Optimizer 4 | class AdamW_GCC2(Optimizer): 5 | """Implements Adam algorithm. 6 | It has been proposed in `Adam: A Method for Stochastic Optimization`_. 7 | Arguments: 8 | params (iterable): iterable of parameters to optimize or dicts defining 9 | parameter groups 10 | lr (float, optional): learning rate (default: 1e-3) 11 | betas (Tuple[float, float], optional): coefficients used for computing 12 | running averages of gradient and its square (default: (0.9, 0.999)) 13 | eps (float, optional): term added to the denominator to improve 14 | numerical stability (default: 1e-8) 15 | weight_decay (float, optional): weight decay (L2 penalty) (default: 0) 16 | amsgrad (boolean, optional): whether to use the AMSGrad variant of this 17 | algorithm from the paper `On the Convergence of Adam and Beyond`_ 18 | .. _Adam\: A Method for Stochastic Optimization: 19 | https://arxiv.org/abs/1412.6980 20 | .. _On the Convergence of Adam and Beyond: 21 | https://openreview.net/forum?id=ryQu7f-RZ 22 | """ 23 | 24 | def __init__(self, params, lr=1e-3, betas=(0.9, 0.999), eps=1e-8, 25 | weight_decay=0, amsgrad=False): 26 | if not 0.0 <= lr: 27 | raise ValueError("Invalid learning rate: {}".format(lr)) 28 | if not 0.0 <= eps: 29 | raise ValueError("Invalid epsilon value: {}".format(eps)) 30 | if not 0.0 <= betas[0] < 1.0: 31 | raise ValueError("Invalid beta parameter at index 0: {}".format(betas[0])) 32 | if not 0.0 <= betas[1] < 1.0: 33 | raise ValueError("Invalid beta parameter at index 1: {}".format(betas[1])) 34 | defaults = dict(lr=lr, betas=betas, eps=eps, 35 | weight_decay=weight_decay, amsgrad=amsgrad) 36 | super(AdamW_GCC2, self).__init__(params, defaults) 37 | 38 | def __setstate__(self, state): 39 | super(AdamW_GCC2, self).__setstate__(state) 40 | for group in self.param_groups: 41 | group.setdefault('amsgrad', False) 42 | 43 | def step(self, closure=None): 44 | """Performs a single optimization step. 45 | Arguments: 46 | closure (callable, optional): A closure that reevaluates the model 47 | and returns the loss. 48 | """ 49 | loss = None 50 | if closure is not None: 51 | loss = closure() 52 | 53 | for group in self.param_groups: 54 | for p in group['params']: 55 | if p.grad is None: 56 | continue 57 | grad = p.grad.data 58 | if grad.is_sparse: 59 | raise RuntimeError('Adam does not support sparse gradients, please consider SparseAdam instead') 60 | amsgrad = group['amsgrad'] 61 | 62 | state = self.state[p] 63 | 64 | # State initialization 65 | if len(state) == 0: 66 | state['step'] = 0 67 | # Exponential moving average of gradient values 68 | state['exp_avg'] = torch.zeros_like(p.data) 69 | # Exponential moving average of squared gradient values 70 | state['exp_avg_sq'] = torch.zeros_like(p.data) 71 | if amsgrad: 72 | # Maintains max of all exp. moving avg. of sq. grad. values 73 | state['max_exp_avg_sq'] = torch.zeros_like(p.data) 74 | 75 | exp_avg, exp_avg_sq = state['exp_avg'], state['exp_avg_sq'] 76 | if amsgrad: 77 | max_exp_avg_sq = state['max_exp_avg_sq'] 78 | beta1, beta2 = group['betas'] 79 | 80 | # GC operation for Conv layers 81 | if len(list(grad.size())) > 3: 82 | weight_mean = p.data.mean(dim=tuple(range(1, len(list(grad.size())))), keepdim=True) 83 | grad.add_(-grad.mean(dim=tuple(range(1, len(list(grad.size())))), keepdim=True)) 84 | 85 | state['step'] += 1 86 | 87 | # if group['weight_decay'] != 0: 88 | # grad = grad.add(group['weight_decay'], p.data) 89 | 90 | # Decay the first and second moment running average coefficient 91 | exp_avg.mul_(beta1).add_(1 - beta1, grad) 92 | exp_avg_sq.mul_(beta2).addcmul_(1 - beta2, grad, grad) 93 | if amsgrad: 94 | # Maintains the maximum of all 2nd moment running avg. till now 95 | torch.max(max_exp_avg_sq, exp_avg_sq, out=max_exp_avg_sq) 96 | # Use the max. for normalizing running avg. of gradient 97 | denom = max_exp_avg_sq.sqrt().add_(group['eps']) 98 | else: 99 | denom = exp_avg_sq.sqrt().add_(group['eps']) 100 | 101 | bias_correction1 = 1 - beta1 ** state['step'] 102 | bias_correction2 = 1 - beta2 ** state['step'] 103 | step_size = group['lr'] * math.sqrt(bias_correction2) / bias_correction1 104 | 105 | # GC operation for Conv layers 106 | if len(list(grad.size())) > 3: 107 | delta = (step_size * torch.mul(p.data, group['weight_decay']).addcdiv_(1, exp_avg, denom)).clone() 108 | delta.add_(-delta.mean(dim=tuple(range(1, len(list(grad.size())))), keepdim=True)) 109 | p.data.add_(-delta) 110 | else: 111 | p.data.add_(-step_size, torch.mul(p.data, group['weight_decay']).addcdiv_(1, exp_avg, denom)) 112 | 113 | return loss -------------------------------------------------------------------------------- /yolov5_test/Dockerfile: -------------------------------------------------------------------------------- 1 | # Start FROM Nvidia PyTorch image https://ngc.nvidia.com/catalog/containers/nvidia:pytorch 2 | FROM nvcr.io/nvidia/pytorch:20.10-py3 3 | 4 | # Install dependencies 5 | RUN pip install --upgrade pip 6 | # COPY requirements.txt . 7 | # RUN pip install -r requirements.txt 8 | RUN pip install gsutil 9 | 10 | # Create working directory 11 | RUN mkdir -p /usr/src/app 12 | WORKDIR /usr/src/app 13 | 14 | # Copy contents 15 | COPY . /usr/src/app 16 | 17 | # Copy weights 18 | #RUN python3 -c "from models import *; \ 19 | #attempt_download('weights/yolov5s.pt'); \ 20 | #attempt_download('weights/yolov5m.pt'); \ 21 | #attempt_download('weights/yolov5l.pt')" 22 | 23 | 24 | # --------------------------------------------------- Extras Below --------------------------------------------------- 25 | 26 | # Build and Push 27 | # t=ultralytics/yolov5:latest && sudo docker build -t $t . && sudo docker push $t 28 | # for v in {300..303}; do t=ultralytics/coco:v$v && sudo docker build -t $t . && sudo docker push $t; done 29 | 30 | # Pull and Run 31 | # t=ultralytics/yolov5:latest && sudo docker pull $t && sudo docker run -it --ipc=host $t 32 | 33 | # Pull and Run with local directory access 34 | # t=ultralytics/yolov5:latest && sudo docker pull $t && sudo docker run -it --ipc=host --gpus all -v "$(pwd)"/coco:/usr/src/coco $t 35 | 36 | # Kill all 37 | # sudo docker kill $(sudo docker ps -q) 38 | 39 | # Kill all image-based 40 | # sudo docker kill $(sudo docker ps -a -q --filter ancestor=ultralytics/yolov5:latest) 41 | 42 | # Bash into running container 43 | # sudo docker container exec -it ba65811811ab bash 44 | 45 | # Bash into stopped container 46 | # sudo docker commit 092b16b25c5b usr/resume && sudo docker run -it --gpus all --ipc=host -v "$(pwd)"/coco:/usr/src/coco --entrypoint=sh usr/resume 47 | 48 | # Send weights to GCP 49 | # python -c "from utils.general import *; strip_optimizer('runs/exp0_*/weights/best.pt', 'tmp.pt')" && gsutil cp tmp.pt gs://*.pt 50 | 51 | # Clean up 52 | # docker system prune -a --volumes 53 | -------------------------------------------------------------------------------- /yolov5_test/LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | Preamble 9 | 10 | The GNU General Public License is a free, copyleft license for 11 | software and other kinds of works. 12 | 13 | The licenses for most software and other practical works are designed 14 | to take away your freedom to share and change the works. By contrast, 15 | the GNU General Public License is intended to guarantee your freedom to 16 | share and change all versions of a program--to make sure it remains free 17 | software for all its users. We, the Free Software Foundation, use the 18 | GNU General Public License for most of our software; it applies also to 19 | any other work released this way by its authors. You can apply it to 20 | your programs, too. 21 | 22 | When we speak of free software, we are referring to freedom, not 23 | price. Our General Public Licenses are designed to make sure that you 24 | have the freedom to distribute copies of free software (and charge for 25 | them if you wish), that you receive source code or can get it if you 26 | want it, that you can change the software or use pieces of it in new 27 | free programs, and that you know you can do these things. 28 | 29 | To protect your rights, we need to prevent others from denying you 30 | these rights or asking you to surrender the rights. Therefore, you have 31 | certain responsibilities if you distribute copies of the software, or if 32 | you modify it: responsibilities to respect the freedom of others. 33 | 34 | For example, if you distribute copies of such a program, whether 35 | gratis or for a fee, you must pass on to the recipients the same 36 | freedoms that you received. You must make sure that they, too, receive 37 | or can get the source code. And you must show them these terms so they 38 | know their rights. 39 | 40 | Developers that use the GNU GPL protect your rights with two steps: 41 | (1) assert copyright on the software, and (2) offer you this License 42 | giving you legal permission to copy, distribute and/or modify it. 43 | 44 | For the developers' and authors' protection, the GPL clearly explains 45 | that there is no warranty for this free software. For both users' and 46 | authors' sake, the GPL requires that modified versions be marked as 47 | changed, so that their problems will not be attributed erroneously to 48 | authors of previous versions. 49 | 50 | Some devices are designed to deny users access to install or run 51 | modified versions of the software inside them, although the manufacturer 52 | can do so. This is fundamentally incompatible with the aim of 53 | protecting users' freedom to change the software. The systematic 54 | pattern of such abuse occurs in the area of products for individuals to 55 | use, which is precisely where it is most unacceptable. Therefore, we 56 | have designed this version of the GPL to prohibit the practice for those 57 | products. If such problems arise substantially in other domains, we 58 | stand ready to extend this provision to those domains in future versions 59 | of the GPL, as needed to protect the freedom of users. 60 | 61 | Finally, every program is threatened constantly by software patents. 62 | States should not allow patents to restrict development and use of 63 | software on general-purpose computers, but in those that do, we wish to 64 | avoid the special danger that patents applied to a free program could 65 | make it effectively proprietary. To prevent this, the GPL assures that 66 | patents cannot be used to render the program non-free. 67 | 68 | The precise terms and conditions for copying, distribution and 69 | modification follow. 70 | 71 | TERMS AND CONDITIONS 72 | 73 | 0. Definitions. 74 | 75 | "This License" refers to version 3 of the GNU General Public License. 76 | 77 | "Copyright" also means copyright-like laws that apply to other kinds of 78 | works, such as semiconductor masks. 79 | 80 | "The Program" refers to any copyrightable work licensed under this 81 | License. Each licensee is addressed as "you". "Licensees" and 82 | "recipients" may be individuals or organizations. 83 | 84 | To "modify" a work means to copy from or adapt all or part of the work 85 | in a fashion requiring copyright permission, other than the making of an 86 | exact copy. The resulting work is called a "modified version" of the 87 | earlier work or a work "based on" the earlier work. 88 | 89 | A "covered work" means either the unmodified Program or a work based 90 | on the Program. 91 | 92 | To "propagate" a work means to do anything with it that, without 93 | permission, would make you directly or secondarily liable for 94 | infringement under applicable copyright law, except executing it on a 95 | computer or modifying a private copy. Propagation includes copying, 96 | distribution (with or without modification), making available to the 97 | public, and in some countries other activities as well. 98 | 99 | To "convey" a work means any kind of propagation that enables other 100 | parties to make or receive copies. Mere interaction with a user through 101 | a computer network, with no transfer of a copy, is not conveying. 102 | 103 | An interactive user interface displays "Appropriate Legal Notices" 104 | to the extent that it includes a convenient and prominently visible 105 | feature that (1) displays an appropriate copyright notice, and (2) 106 | tells the user that there is no warranty for the work (except to the 107 | extent that warranties are provided), that licensees may convey the 108 | work under this License, and how to view a copy of this License. If 109 | the interface presents a list of user commands or options, such as a 110 | menu, a prominent item in the list meets this criterion. 111 | 112 | 1. Source Code. 113 | 114 | The "source code" for a work means the preferred form of the work 115 | for making modifications to it. "Object code" means any non-source 116 | form of a work. 117 | 118 | A "Standard Interface" means an interface that either is an official 119 | standard defined by a recognized standards body, or, in the case of 120 | interfaces specified for a particular programming language, one that 121 | is widely used among developers working in that language. 122 | 123 | The "System Libraries" of an executable work include anything, other 124 | than the work as a whole, that (a) is included in the normal form of 125 | packaging a Major Component, but which is not part of that Major 126 | Component, and (b) serves only to enable use of the work with that 127 | Major Component, or to implement a Standard Interface for which an 128 | implementation is available to the public in source code form. A 129 | "Major Component", in this context, means a major essential component 130 | (kernel, window system, and so on) of the specific operating system 131 | (if any) on which the executable work runs, or a compiler used to 132 | produce the work, or an object code interpreter used to run it. 133 | 134 | The "Corresponding Source" for a work in object code form means all 135 | the source code needed to generate, install, and (for an executable 136 | work) run the object code and to modify the work, including scripts to 137 | control those activities. However, it does not include the work's 138 | System Libraries, or general-purpose tools or generally available free 139 | programs which are used unmodified in performing those activities but 140 | which are not part of the work. For example, Corresponding Source 141 | includes interface definition files associated with source files for 142 | the work, and the source code for shared libraries and dynamically 143 | linked subprograms that the work is specifically designed to require, 144 | such as by intimate data communication or control flow between those 145 | subprograms and other parts of the work. 146 | 147 | The Corresponding Source need not include anything that users 148 | can regenerate automatically from other parts of the Corresponding 149 | Source. 150 | 151 | The Corresponding Source for a work in source code form is that 152 | same work. 153 | 154 | 2. Basic Permissions. 155 | 156 | All rights granted under this License are granted for the term of 157 | copyright on the Program, and are irrevocable provided the stated 158 | conditions are met. This License explicitly affirms your unlimited 159 | permission to run the unmodified Program. The output from running a 160 | covered work is covered by this License only if the output, given its 161 | content, constitutes a covered work. This License acknowledges your 162 | rights of fair use or other equivalent, as provided by copyright law. 163 | 164 | You may make, run and propagate covered works that you do not 165 | convey, without conditions so long as your license otherwise remains 166 | in force. You may convey covered works to others for the sole purpose 167 | of having them make modifications exclusively for you, or provide you 168 | with facilities for running those works, provided that you comply with 169 | the terms of this License in conveying all material for which you do 170 | not control copyright. Those thus making or running the covered works 171 | for you must do so exclusively on your behalf, under your direction 172 | and control, on terms that prohibit them from making any copies of 173 | your copyrighted material outside their relationship with you. 174 | 175 | Conveying under any other circumstances is permitted solely under 176 | the conditions stated below. Sublicensing is not allowed; section 10 177 | makes it unnecessary. 178 | 179 | 3. Protecting Users' Legal Rights From Anti-Circumvention Law. 180 | 181 | No covered work shall be deemed part of an effective technological 182 | measure under any applicable law fulfilling obligations under article 183 | 11 of the WIPO copyright treaty adopted on 20 December 1996, or 184 | similar laws prohibiting or restricting circumvention of such 185 | measures. 186 | 187 | When you convey a covered work, you waive any legal power to forbid 188 | circumvention of technological measures to the extent such circumvention 189 | is effected by exercising rights under this License with respect to 190 | the covered work, and you disclaim any intention to limit operation or 191 | modification of the work as a means of enforcing, against the work's 192 | users, your or third parties' legal rights to forbid circumvention of 193 | technological measures. 194 | 195 | 4. Conveying Verbatim Copies. 196 | 197 | You may convey verbatim copies of the Program's source code as you 198 | receive it, in any medium, provided that you conspicuously and 199 | appropriately publish on each copy an appropriate copyright notice; 200 | keep intact all notices stating that this License and any 201 | non-permissive terms added in accord with section 7 apply to the code; 202 | keep intact all notices of the absence of any warranty; and give all 203 | recipients a copy of this License along with the Program. 204 | 205 | You may charge any price or no price for each copy that you convey, 206 | and you may offer support or warranty protection for a fee. 207 | 208 | 5. Conveying Modified Source Versions. 209 | 210 | You may convey a work based on the Program, or the modifications to 211 | produce it from the Program, in the form of source code under the 212 | terms of section 4, provided that you also meet all of these conditions: 213 | 214 | a) The work must carry prominent notices stating that you modified 215 | it, and giving a relevant date. 216 | 217 | b) The work must carry prominent notices stating that it is 218 | released under this License and any conditions added under section 219 | 7. This requirement modifies the requirement in section 4 to 220 | "keep intact all notices". 221 | 222 | c) You must license the entire work, as a whole, under this 223 | License to anyone who comes into possession of a copy. This 224 | License will therefore apply, along with any applicable section 7 225 | additional terms, to the whole of the work, and all its parts, 226 | regardless of how they are packaged. This License gives no 227 | permission to license the work in any other way, but it does not 228 | invalidate such permission if you have separately received it. 229 | 230 | d) If the work has interactive user interfaces, each must display 231 | Appropriate Legal Notices; however, if the Program has interactive 232 | interfaces that do not display Appropriate Legal Notices, your 233 | work need not make them do so. 234 | 235 | A compilation of a covered work with other separate and independent 236 | works, which are not by their nature extensions of the covered work, 237 | and which are not combined with it such as to form a larger program, 238 | in or on a volume of a storage or distribution medium, is called an 239 | "aggregate" if the compilation and its resulting copyright are not 240 | used to limit the access or legal rights of the compilation's users 241 | beyond what the individual works permit. Inclusion of a covered work 242 | in an aggregate does not cause this License to apply to the other 243 | parts of the aggregate. 244 | 245 | 6. Conveying Non-Source Forms. 246 | 247 | You may convey a covered work in object code form under the terms 248 | of sections 4 and 5, provided that you also convey the 249 | machine-readable Corresponding Source under the terms of this License, 250 | in one of these ways: 251 | 252 | a) Convey the object code in, or embodied in, a physical product 253 | (including a physical distribution medium), accompanied by the 254 | Corresponding Source fixed on a durable physical medium 255 | customarily used for software interchange. 256 | 257 | b) Convey the object code in, or embodied in, a physical product 258 | (including a physical distribution medium), accompanied by a 259 | written offer, valid for at least three years and valid for as 260 | long as you offer spare parts or customer support for that product 261 | model, to give anyone who possesses the object code either (1) a 262 | copy of the Corresponding Source for all the software in the 263 | product that is covered by this License, on a durable physical 264 | medium customarily used for software interchange, for a price no 265 | more than your reasonable cost of physically performing this 266 | conveying of source, or (2) access to copy the 267 | Corresponding Source from a network server at no charge. 268 | 269 | c) Convey individual copies of the object code with a copy of the 270 | written offer to provide the Corresponding Source. This 271 | alternative is allowed only occasionally and noncommercially, and 272 | only if you received the object code with such an offer, in accord 273 | with subsection 6b. 274 | 275 | d) Convey the object code by offering access from a designated 276 | place (gratis or for a charge), and offer equivalent access to the 277 | Corresponding Source in the same way through the same place at no 278 | further charge. You need not require recipients to copy the 279 | Corresponding Source along with the object code. If the place to 280 | copy the object code is a network server, the Corresponding Source 281 | may be on a different server (operated by you or a third party) 282 | that supports equivalent copying facilities, provided you maintain 283 | clear directions next to the object code saying where to find the 284 | Corresponding Source. Regardless of what server hosts the 285 | Corresponding Source, you remain obligated to ensure that it is 286 | available for as long as needed to satisfy these requirements. 287 | 288 | e) Convey the object code using peer-to-peer transmission, provided 289 | you inform other peers where the object code and Corresponding 290 | Source of the work are being offered to the general public at no 291 | charge under subsection 6d. 292 | 293 | A separable portion of the object code, whose source code is excluded 294 | from the Corresponding Source as a System Library, need not be 295 | included in conveying the object code work. 296 | 297 | A "User Product" is either (1) a "consumer product", which means any 298 | tangible personal property which is normally used for personal, family, 299 | or household purposes, or (2) anything designed or sold for incorporation 300 | into a dwelling. In determining whether a product is a consumer product, 301 | doubtful cases shall be resolved in favor of coverage. For a particular 302 | product received by a particular user, "normally used" refers to a 303 | typical or common use of that class of product, regardless of the status 304 | of the particular user or of the way in which the particular user 305 | actually uses, or expects or is expected to use, the product. A product 306 | is a consumer product regardless of whether the product has substantial 307 | commercial, industrial or non-consumer uses, unless such uses represent 308 | the only significant mode of use of the product. 309 | 310 | "Installation Information" for a User Product means any methods, 311 | procedures, authorization keys, or other information required to install 312 | and execute modified versions of a covered work in that User Product from 313 | a modified version of its Corresponding Source. The information must 314 | suffice to ensure that the continued functioning of the modified object 315 | code is in no case prevented or interfered with solely because 316 | modification has been made. 317 | 318 | If you convey an object code work under this section in, or with, or 319 | specifically for use in, a User Product, and the conveying occurs as 320 | part of a transaction in which the right of possession and use of the 321 | User Product is transferred to the recipient in perpetuity or for a 322 | fixed term (regardless of how the transaction is characterized), the 323 | Corresponding Source conveyed under this section must be accompanied 324 | by the Installation Information. But this requirement does not apply 325 | if neither you nor any third party retains the ability to install 326 | modified object code on the User Product (for example, the work has 327 | been installed in ROM). 328 | 329 | The requirement to provide Installation Information does not include a 330 | requirement to continue to provide support service, warranty, or updates 331 | for a work that has been modified or installed by the recipient, or for 332 | the User Product in which it has been modified or installed. Access to a 333 | network may be denied when the modification itself materially and 334 | adversely affects the operation of the network or violates the rules and 335 | protocols for communication across the network. 336 | 337 | Corresponding Source conveyed, and Installation Information provided, 338 | in accord with this section must be in a format that is publicly 339 | documented (and with an implementation available to the public in 340 | source code form), and must require no special password or key for 341 | unpacking, reading or copying. 342 | 343 | 7. Additional Terms. 344 | 345 | "Additional permissions" are terms that supplement the terms of this 346 | License by making exceptions from one or more of its conditions. 347 | Additional permissions that are applicable to the entire Program shall 348 | be treated as though they were included in this License, to the extent 349 | that they are valid under applicable law. If additional permissions 350 | apply only to part of the Program, that part may be used separately 351 | under those permissions, but the entire Program remains governed by 352 | this License without regard to the additional permissions. 353 | 354 | When you convey a copy of a covered work, you may at your option 355 | remove any additional permissions from that copy, or from any part of 356 | it. (Additional permissions may be written to require their own 357 | removal in certain cases when you modify the work.) You may place 358 | additional permissions on material, added by you to a covered work, 359 | for which you have or can give appropriate copyright permission. 360 | 361 | Notwithstanding any other provision of this License, for material you 362 | add to a covered work, you may (if authorized by the copyright holders of 363 | that material) supplement the terms of this License with terms: 364 | 365 | a) Disclaiming warranty or limiting liability differently from the 366 | terms of sections 15 and 16 of this License; or 367 | 368 | b) Requiring preservation of specified reasonable legal notices or 369 | author attributions in that material or in the Appropriate Legal 370 | Notices displayed by works containing it; or 371 | 372 | c) Prohibiting misrepresentation of the origin of that material, or 373 | requiring that modified versions of such material be marked in 374 | reasonable ways as different from the original version; or 375 | 376 | d) Limiting the use for publicity purposes of names of licensors or 377 | authors of the material; or 378 | 379 | e) Declining to grant rights under trademark law for use of some 380 | trade names, trademarks, or service marks; or 381 | 382 | f) Requiring indemnification of licensors and authors of that 383 | material by anyone who conveys the material (or modified versions of 384 | it) with contractual assumptions of liability to the recipient, for 385 | any liability that these contractual assumptions directly impose on 386 | those licensors and authors. 387 | 388 | All other non-permissive additional terms are considered "further 389 | restrictions" within the meaning of section 10. If the Program as you 390 | received it, or any part of it, contains a notice stating that it is 391 | governed by this License along with a term that is a further 392 | restriction, you may remove that term. If a license document contains 393 | a further restriction but permits relicensing or conveying under this 394 | License, you may add to a covered work material governed by the terms 395 | of that license document, provided that the further restriction does 396 | not survive such relicensing or conveying. 397 | 398 | If you add terms to a covered work in accord with this section, you 399 | must place, in the relevant source files, a statement of the 400 | additional terms that apply to those files, or a notice indicating 401 | where to find the applicable terms. 402 | 403 | Additional terms, permissive or non-permissive, may be stated in the 404 | form of a separately written license, or stated as exceptions; 405 | the above requirements apply either way. 406 | 407 | 8. Termination. 408 | 409 | You may not propagate or modify a covered work except as expressly 410 | provided under this License. Any attempt otherwise to propagate or 411 | modify it is void, and will automatically terminate your rights under 412 | this License (including any patent licenses granted under the third 413 | paragraph of section 11). 414 | 415 | However, if you cease all violation of this License, then your 416 | license from a particular copyright holder is reinstated (a) 417 | provisionally, unless and until the copyright holder explicitly and 418 | finally terminates your license, and (b) permanently, if the copyright 419 | holder fails to notify you of the violation by some reasonable means 420 | prior to 60 days after the cessation. 421 | 422 | Moreover, your license from a particular copyright holder is 423 | reinstated permanently if the copyright holder notifies you of the 424 | violation by some reasonable means, this is the first time you have 425 | received notice of violation of this License (for any work) from that 426 | copyright holder, and you cure the violation prior to 30 days after 427 | your receipt of the notice. 428 | 429 | Termination of your rights under this section does not terminate the 430 | licenses of parties who have received copies or rights from you under 431 | this License. If your rights have been terminated and not permanently 432 | reinstated, you do not qualify to receive new licenses for the same 433 | material under section 10. 434 | 435 | 9. Acceptance Not Required for Having Copies. 436 | 437 | You are not required to accept this License in order to receive or 438 | run a copy of the Program. Ancillary propagation of a covered work 439 | occurring solely as a consequence of using peer-to-peer transmission 440 | to receive a copy likewise does not require acceptance. However, 441 | nothing other than this License grants you permission to propagate or 442 | modify any covered work. These actions infringe copyright if you do 443 | not accept this License. Therefore, by modifying or propagating a 444 | covered work, you indicate your acceptance of this License to do so. 445 | 446 | 10. Automatic Licensing of Downstream Recipients. 447 | 448 | Each time you convey a covered work, the recipient automatically 449 | receives a license from the original licensors, to run, modify and 450 | propagate that work, subject to this License. You are not responsible 451 | for enforcing compliance by third parties with this License. 452 | 453 | An "entity transaction" is a transaction transferring control of an 454 | organization, or substantially all assets of one, or subdividing an 455 | organization, or merging organizations. If propagation of a covered 456 | work results from an entity transaction, each party to that 457 | transaction who receives a copy of the work also receives whatever 458 | licenses to the work the party's predecessor in interest had or could 459 | give under the previous paragraph, plus a right to possession of the 460 | Corresponding Source of the work from the predecessor in interest, if 461 | the predecessor has it or can get it with reasonable efforts. 462 | 463 | You may not impose any further restrictions on the exercise of the 464 | rights granted or affirmed under this License. For example, you may 465 | not impose a license fee, royalty, or other charge for exercise of 466 | rights granted under this License, and you may not initiate litigation 467 | (including a cross-claim or counterclaim in a lawsuit) alleging that 468 | any patent claim is infringed by making, using, selling, offering for 469 | sale, or importing the Program or any portion of it. 470 | 471 | 11. Patents. 472 | 473 | A "contributor" is a copyright holder who authorizes use under this 474 | License of the Program or a work on which the Program is based. The 475 | work thus licensed is called the contributor's "contributor version". 476 | 477 | A contributor's "essential patent claims" are all patent claims 478 | owned or controlled by the contributor, whether already acquired or 479 | hereafter acquired, that would be infringed by some manner, permitted 480 | by this License, of making, using, or selling its contributor version, 481 | but do not include claims that would be infringed only as a 482 | consequence of further modification of the contributor version. For 483 | purposes of this definition, "control" includes the right to grant 484 | patent sublicenses in a manner consistent with the requirements of 485 | this License. 486 | 487 | Each contributor grants you a non-exclusive, worldwide, royalty-free 488 | patent license under the contributor's essential patent claims, to 489 | make, use, sell, offer for sale, import and otherwise run, modify and 490 | propagate the contents of its contributor version. 491 | 492 | In the following three paragraphs, a "patent license" is any express 493 | agreement or commitment, however denominated, not to enforce a patent 494 | (such as an express permission to practice a patent or covenant not to 495 | sue for patent infringement). To "grant" such a patent license to a 496 | party means to make such an agreement or commitment not to enforce a 497 | patent against the party. 498 | 499 | If you convey a covered work, knowingly relying on a patent license, 500 | and the Corresponding Source of the work is not available for anyone 501 | to copy, free of charge and under the terms of this License, through a 502 | publicly available network server or other readily accessible means, 503 | then you must either (1) cause the Corresponding Source to be so 504 | available, or (2) arrange to deprive yourself of the benefit of the 505 | patent license for this particular work, or (3) arrange, in a manner 506 | consistent with the requirements of this License, to extend the patent 507 | license to downstream recipients. "Knowingly relying" means you have 508 | actual knowledge that, but for the patent license, your conveying the 509 | covered work in a country, or your recipient's use of the covered work 510 | in a country, would infringe one or more identifiable patents in that 511 | country that you have reason to believe are valid. 512 | 513 | If, pursuant to or in connection with a single transaction or 514 | arrangement, you convey, or propagate by procuring conveyance of, a 515 | covered work, and grant a patent license to some of the parties 516 | receiving the covered work authorizing them to use, propagate, modify 517 | or convey a specific copy of the covered work, then the patent license 518 | you grant is automatically extended to all recipients of the covered 519 | work and works based on it. 520 | 521 | A patent license is "discriminatory" if it does not include within 522 | the scope of its coverage, prohibits the exercise of, or is 523 | conditioned on the non-exercise of one or more of the rights that are 524 | specifically granted under this License. You may not convey a covered 525 | work if you are a party to an arrangement with a third party that is 526 | in the business of distributing software, under which you make payment 527 | to the third party based on the extent of your activity of conveying 528 | the work, and under which the third party grants, to any of the 529 | parties who would receive the covered work from you, a discriminatory 530 | patent license (a) in connection with copies of the covered work 531 | conveyed by you (or copies made from those copies), or (b) primarily 532 | for and in connection with specific products or compilations that 533 | contain the covered work, unless you entered into that arrangement, 534 | or that patent license was granted, prior to 28 March 2007. 535 | 536 | Nothing in this License shall be construed as excluding or limiting 537 | any implied license or other defenses to infringement that may 538 | otherwise be available to you under applicable patent law. 539 | 540 | 12. No Surrender of Others' Freedom. 541 | 542 | If conditions are imposed on you (whether by court order, agreement or 543 | otherwise) that contradict the conditions of this License, they do not 544 | excuse you from the conditions of this License. If you cannot convey a 545 | covered work so as to satisfy simultaneously your obligations under this 546 | License and any other pertinent obligations, then as a consequence you may 547 | not convey it at all. For example, if you agree to terms that obligate you 548 | to collect a royalty for further conveying from those to whom you convey 549 | the Program, the only way you could satisfy both those terms and this 550 | License would be to refrain entirely from conveying the Program. 551 | 552 | 13. Use with the GNU Affero General Public License. 553 | 554 | Notwithstanding any other provision of this License, you have 555 | permission to link or combine any covered work with a work licensed 556 | under version 3 of the GNU Affero General Public License into a single 557 | combined work, and to convey the resulting work. The terms of this 558 | License will continue to apply to the part which is the covered work, 559 | but the special requirements of the GNU Affero General Public License, 560 | section 13, concerning interaction through a network will apply to the 561 | combination as such. 562 | 563 | 14. Revised Versions of this License. 564 | 565 | The Free Software Foundation may publish revised and/or new versions of 566 | the GNU General Public License from time to time. Such new versions will 567 | be similar in spirit to the present version, but may differ in detail to 568 | address new problems or concerns. 569 | 570 | Each version is given a distinguishing version number. If the 571 | Program specifies that a certain numbered version of the GNU General 572 | Public License "or any later version" applies to it, you have the 573 | option of following the terms and conditions either of that numbered 574 | version or of any later version published by the Free Software 575 | Foundation. If the Program does not specify a version number of the 576 | GNU General Public License, you may choose any version ever published 577 | by the Free Software Foundation. 578 | 579 | If the Program specifies that a proxy can decide which future 580 | versions of the GNU General Public License can be used, that proxy's 581 | public statement of acceptance of a version permanently authorizes you 582 | to choose that version for the Program. 583 | 584 | Later license versions may give you additional or different 585 | permissions. However, no additional obligations are imposed on any 586 | author or copyright holder as a result of your choosing to follow a 587 | later version. 588 | 589 | 15. Disclaimer of Warranty. 590 | 591 | THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY 592 | APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT 593 | HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY 594 | OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, 595 | THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 596 | PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM 597 | IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF 598 | ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 599 | 600 | 16. Limitation of Liability. 601 | 602 | IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 603 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS 604 | THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY 605 | GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE 606 | USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF 607 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD 608 | PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), 609 | EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF 610 | SUCH DAMAGES. 611 | 612 | 17. Interpretation of Sections 15 and 16. 613 | 614 | If the disclaimer of warranty and limitation of liability provided 615 | above cannot be given local legal effect according to their terms, 616 | reviewing courts shall apply local law that most closely approximates 617 | an absolute waiver of all civil liability in connection with the 618 | Program, unless a warranty or assumption of liability accompanies a 619 | copy of the Program in return for a fee. 620 | 621 | END OF TERMS AND CONDITIONS 622 | 623 | How to Apply These Terms to Your New Programs 624 | 625 | If you develop a new program, and you want it to be of the greatest 626 | possible use to the public, the best way to achieve this is to make it 627 | free software which everyone can redistribute and change under these terms. 628 | 629 | To do so, attach the following notices to the program. It is safest 630 | to attach them to the start of each source file to most effectively 631 | state the exclusion of warranty; and each file should have at least 632 | the "copyright" line and a pointer to where the full notice is found. 633 | 634 | 635 | Copyright (C) 636 | 637 | This program is free software: you can redistribute it and/or modify 638 | it under the terms of the GNU General Public License as published by 639 | the Free Software Foundation, either version 3 of the License, or 640 | (at your option) any later version. 641 | 642 | This program is distributed in the hope that it will be useful, 643 | but WITHOUT ANY WARRANTY; without even the implied warranty of 644 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 645 | GNU General Public License for more details. 646 | 647 | You should have received a copy of the GNU General Public License 648 | along with this program. If not, see . 649 | 650 | Also add information on how to contact you by electronic and paper mail. 651 | 652 | If the program does terminal interaction, make it output a short 653 | notice like this when it starts in an interactive mode: 654 | 655 | Copyright (C) 656 | This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 657 | This is free software, and you are welcome to redistribute it 658 | under certain conditions; type `show c' for details. 659 | 660 | The hypothetical commands `show w' and `show c' should show the appropriate 661 | parts of the General Public License. Of course, your program's commands 662 | might be different; for a GUI interface, you would use an "about box". 663 | 664 | You should also get your employer (if you work as a programmer) or school, 665 | if any, to sign a "copyright disclaimer" for the program, if necessary. 666 | For more information on this, and how to apply and follow the GNU GPL, see 667 | . 668 | 669 | The GNU General Public License does not permit incorporating your program 670 | into proprietary programs. If your program is a subroutine library, you 671 | may consider it more useful to permit linking proprietary applications with 672 | the library. If this is what you want to do, use the GNU Lesser General 673 | Public License instead of this License. But first, please read 674 | . -------------------------------------------------------------------------------- /yolov5_test/data/AGC.yaml: -------------------------------------------------------------------------------- 1 | train: /media/data2/AGC/generated_frames/kang/train/images 2 | # val: /media/data2/AGC/generated_frames/kang/val/images 3 | val: /media/data2/AGC/generated_frames/kang/val_iitp/images 4 | # test: /media/data2/AGC/generated_frames/kang/val/images 5 | test: /media/data2/AGC/generated_frames/kang/val_iitp/images 6 | 7 | 8 | # train: /media/data2/dataset/Arirang/256_128/train_tr_final/images 9 | # val: /media/data2/dataset/Arirang/256_128/train_val_split/images 10 | # test: /media/data2/dataset/Arirang/256_128/test_split 11 | 12 | nc: 1 13 | names: ['person'] -------------------------------------------------------------------------------- /yolov5_test/data/coco.yaml: -------------------------------------------------------------------------------- 1 | # COCO 2017 dataset http://cocodataset.org 2 | # Train command: python train.py --data coco.yaml 3 | # Default dataset location is next to /yolov5: 4 | # /parent_folder 5 | # /coco 6 | # /yolov5 7 | 8 | 9 | # download command/URL (optional) 10 | download: bash data/scripts/get_coco.sh 11 | 12 | # train and val data as 1) directory: path/images/, 2) file: path/images.txt, or 3) list: [path1/images/, path2/images/] 13 | train: ../coco/train2017.txt # 118287 images 14 | val: ../coco/val2017.txt # 5000 images 15 | test: ../coco/test-dev2017.txt # 20288 of 40670 images, submit to https://competitions.codalab.org/competitions/20794 16 | 17 | # number of classes 18 | nc: 80 19 | 20 | # class names 21 | names: ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light', 22 | 'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 23 | 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 24 | 'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard', 25 | 'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 26 | 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch', 27 | 'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone', 28 | 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear', 29 | 'hair drier', 'toothbrush'] 30 | 31 | # Print classes 32 | # with open('data/coco.yaml') as f: 33 | # d = yaml.load(f, Loader=yaml.FullLoader) # dict 34 | # for i, x in enumerate(d['names']): 35 | # print(i, x) 36 | -------------------------------------------------------------------------------- /yolov5_test/data/coco128.yaml: -------------------------------------------------------------------------------- 1 | # COCO 2017 dataset http://cocodataset.org - first 128 training images 2 | # Train command: python train.py --data coco128.yaml 3 | # Default dataset location is next to /yolov5: 4 | # /parent_folder 5 | # /coco128 6 | # /yolov5 7 | 8 | 9 | # download command/URL (optional) 10 | download: https://github.com/ultralytics/yolov5/releases/download/v1.0/coco128.zip 11 | 12 | # train and val data as 1) directory: path/images/, 2) file: path/images.txt, or 3) list: [path1/images/, path2/images/] 13 | train: ../coco128/images/train2017/ # 128 images 14 | val: ../coco128/images/train2017/ # 128 images 15 | 16 | # number of classes 17 | nc: 80 18 | 19 | # class names 20 | names: ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus', 'train', 'truck', 'boat', 'traffic light', 21 | 'fire hydrant', 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog', 'horse', 'sheep', 'cow', 22 | 'elephant', 'bear', 'zebra', 'giraffe', 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee', 23 | 'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat', 'baseball glove', 'skateboard', 'surfboard', 24 | 'tennis racket', 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl', 'banana', 'apple', 25 | 'sandwich', 'orange', 'broccoli', 'carrot', 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch', 26 | 'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop', 'mouse', 'remote', 'keyboard', 'cell phone', 27 | 'microwave', 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock', 'vase', 'scissors', 'teddy bear', 28 | 'hair drier', 'toothbrush'] 29 | -------------------------------------------------------------------------------- /yolov5_test/data/hyp.finetune.yaml: -------------------------------------------------------------------------------- 1 | # Hyperparameters for VOC finetuning 2 | # python train.py --batch 64 --weights yolov5m.pt --data voc.yaml --img 512 --epochs 50 3 | # See tutorials for hyperparameter evolution https://github.com/ultralytics/yolov5#tutorials 4 | 5 | 6 | # Hyperparameter Evolution Results 7 | # Generations: 306 8 | # P R mAP.5 mAP.5:.95 box obj cls 9 | # Metrics: 0.6 0.936 0.896 0.684 0.0115 0.00805 0.00146 10 | 11 | lr0: 0.0032 12 | lrf: 0.12 13 | momentum: 0.843 14 | weight_decay: 0.00036 15 | warmup_epochs: 2.0 16 | warmup_momentum: 0.5 17 | warmup_bias_lr: 0.05 18 | box: 0.0296 19 | cls: 0.243 20 | cls_pw: 0.631 21 | obj: 0.301 22 | obj_pw: 0.911 23 | iou_t: 0.2 24 | anchor_t: 2.91 25 | # anchors: 3.63 26 | fl_gamma: 0.0 27 | hsv_h: 0.0138 28 | hsv_s: 0.664 29 | hsv_v: 0.464 30 | degrees: 0.373 31 | translate: 0.245 32 | scale: 0.898 33 | shear: 0.602 34 | perspective: 0.0 35 | flipud: 0.00856 36 | fliplr: 0.5 37 | mosaic: 1.0 38 | mixup: 0.243 39 | -------------------------------------------------------------------------------- /yolov5_test/data/hyp.scratch.yaml: -------------------------------------------------------------------------------- 1 | # Hyperparameters for COCO training from scratch 2 | # python train.py --batch 40 --cfg yolov5m.yaml --weights '' --data coco.yaml --img 640 --epochs 300 3 | # See tutorials for hyperparameter evolution https://github.com/ultralytics/yolov5#tutorials 4 | 5 | 6 | lr0: 0.001 # initial learning rate (SGD=1E-2, Adam=1E-3) 7 | lrf: 0.2 # final OneCycleLR learning rate (lr0 * lrf) 8 | momentum: 0.937 # SGD momentum/Adam beta1 9 | weight_decay: 0.0005 # optimizer weight decay 5e-4 10 | warmup_epochs: 3.0 # warmup epochs (fractions ok) 11 | warmup_momentum: 0.8 # warmup initial momentum 12 | warmup_bias_lr: 0.1 # warmup initial bias lr 13 | box: 0.05 # box loss gain 14 | cls: 0.5 # cls loss gain 15 | cls_pw: 1.0 # cls BCELoss positive_weight 16 | obj: 1.0 # obj loss gain (scale with pixels) 17 | obj_pw: 1.0 # obj BCELoss positive_weight 18 | iou_t: 0.20 # IoU training threshold 19 | anchor_t: 4.0 # anchor-multiple threshold 20 | # anchors: 0 # anchors per output grid (0 to ignore) 21 | fl_gamma: 0.0 # focal loss gamma (efficientDet default gamma=1.5) 22 | hsv_h: 0.015 # image HSV-Hue augmentation (fraction) 23 | #hsv_h: 0.0 # image HSV-Hue augmentation (fraction) 24 | hsv_s: 0.7 # image HSV-Saturation augmentation (fraction) 25 | #hsv_s: 0.0 # image HSV-Saturation augmentation (fraction) 26 | hsv_v: 0.4 # image HSV-Value augmentation (fraction) 27 | #hsv_v: 0.0 # image HSV-Value augmentation (fraction) 28 | degrees: 0.0 # image rotation (+/- deg) 29 | translate: 0.1 # image translation (+/- fraction) 30 | #translate: 0.0 # image translation (+/- fraction) 31 | scale: 0.5 # image scale (+/- gain) 32 | #scale: 0.0 # image scale (+/- gain) 33 | shear: 0.0 # image shear (+/- deg) 34 | perspective: 0.0 # image perspective (+/- fraction), range 0-0.001 35 | flipud: 0.0 # image flip up-down (probability) 36 | fliplr: 0.5 # image flip left-right (probability) 37 | mosaic: 1.0 # image mosaic (probability) 38 | #mosaic: 0.0 # image mosaic (probability) 39 | mixup: 0.0 # image mixup (probability) 40 | -------------------------------------------------------------------------------- /yolov5_test/data/scripts/get_coco.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # COCO 2017 dataset http://cocodataset.org 3 | # Download command: bash data/scripts/get_coco.sh 4 | # Train command: python train.py --data coco.yaml 5 | # Default dataset location is next to /yolov5: 6 | # /parent_folder 7 | # /coco 8 | # /yolov5 9 | 10 | # Download/unzip labels 11 | d='../' # unzip directory 12 | url=https://github.com/ultralytics/yolov5/releases/download/v1.0/ 13 | f='coco2017labels.zip' # 68 MB 14 | echo 'Downloading' $url$f ' ...' && curl -L $url$f -o $f && unzip -q $f -d $d && rm $f # download, unzip, remove 15 | 16 | # Download/unzip images 17 | d='../coco/images' # unzip directory 18 | url=http://images.cocodataset.org/zips/ 19 | f1='train2017.zip' # 19G, 118k images 20 | f2='val2017.zip' # 1G, 5k images 21 | f3='test2017.zip' # 7G, 41k images (optional) 22 | for f in $f1 $f2; do 23 | echo 'Downloading' $url$f ' ...' && curl -L $url$f -o $f && unzip -q $f -d $d && rm $f # download, unzip, remove 24 | done 25 | -------------------------------------------------------------------------------- /yolov5_test/data/scripts/get_voc.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # PASCAL VOC dataset http://host.robots.ox.ac.uk/pascal/VOC/ 3 | # Download command: bash data/scripts/get_voc.sh 4 | # Train command: python train.py --data voc.yaml 5 | # Default dataset location is next to /yolov5: 6 | # /parent_folder 7 | # /VOC 8 | # /yolov5 9 | 10 | start=$(date +%s) 11 | mkdir -p ../tmp 12 | cd ../tmp/ 13 | 14 | # Download/unzip images and labels 15 | d='.' # unzip directory 16 | url=https://github.com/ultralytics/yolov5/releases/download/v1.0/ 17 | f1=VOCtrainval_06-Nov-2007.zip # 446MB, 5012 images 18 | f2=VOCtest_06-Nov-2007.zip # 438MB, 4953 images 19 | f3=VOCtrainval_11-May-2012.zip # 1.95GB, 17126 images 20 | for f in $f1 $f2 $f3; do 21 | echo 'Downloading' $url$f ' ...' && curl -L $url$f -o $f && unzip -q $f -d $d && rm $f # download, unzip, remove 22 | done 23 | 24 | end=$(date +%s) 25 | runtime=$((end - start)) 26 | echo "Completed in" $runtime "seconds" 27 | 28 | echo "Spliting dataset..." 29 | python3 - "$@" <train.txt 89 | cat 2007_train.txt 2007_val.txt 2007_test.txt 2012_train.txt 2012_val.txt >train.all.txt 90 | 91 | python3 - "$@" <= 1 86 | p, s, im0 = path[i], '%g: ' % i, im0s[i].copy() 87 | else: 88 | p, s, im0 = path, '', im0s 89 | 90 | save_path = str(Path(out) / Path(p).name) 91 | txt_path = str(Path(out) / Path(p).stem) + ('_%g' % dataset.frame if dataset.mode == 'video' else '') 92 | s += '%gx%g ' % img.shape[2:] # print string 93 | gn = torch.tensor(im0.shape)[[1, 0, 1, 0]] # normalization gain whwh 94 | if det is not None and len(det): 95 | # Rescale boxes from img_size to im0 size 96 | det[:, :4] = scale_coords(img.shape[2:], det[:, :4], im0.shape).round() 97 | 98 | # Print results 99 | for c in det[:, -1].unique(): 100 | n = (det[:, -1] == c).sum() # detections per class 101 | s += '%g %ss, ' % (n, names[int(c)]) # add to string 102 | 103 | # Write results 104 | for *xyxy, conf, cls in reversed(det): 105 | if save_txt: # Write to file 106 | xywh = (xyxy2xywh(torch.tensor(xyxy).view(1, 4)) / gn).view(-1).tolist() # normalized xywh 107 | line = (cls, conf, *xywh) if opt.save_conf else (cls, *xywh) # label format 108 | with open(txt_path + '.txt', 'a') as f: 109 | f.write(('%g ' * len(line) + '\n') % line) 110 | 111 | if save_img or view_img: # Add bbox to image 112 | label = '%s %.2f' % (names[int(cls)], conf) 113 | plot_one_box(xyxy, im0, label=label, color=colors[int(cls)], line_thickness=3) 114 | 115 | # Print time (inference + NMS) 116 | print('%sDone. (%.3fs)' % (s, t2 - t1)) 117 | 118 | # Stream results 119 | if view_img: 120 | cv2.imshow(p, im0) 121 | if cv2.waitKey(1) == ord('q'): # q to quit 122 | raise StopIteration 123 | 124 | # Save results (image with detections) 125 | if save_img: 126 | if dataset.mode == 'images': 127 | cv2.imwrite(save_path, im0) 128 | else: 129 | if vid_path != save_path: # new video 130 | vid_path = save_path 131 | if isinstance(vid_writer, cv2.VideoWriter): 132 | vid_writer.release() # release previous video writer 133 | 134 | fourcc = 'mp4v' # output video codec 135 | fps = vid_cap.get(cv2.CAP_PROP_FPS) 136 | w = int(vid_cap.get(cv2.CAP_PROP_FRAME_WIDTH)) 137 | h = int(vid_cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) 138 | vid_writer = cv2.VideoWriter(save_path, cv2.VideoWriter_fourcc(*fourcc), fps, (w, h)) 139 | vid_writer.write(im0) 140 | 141 | if save_txt or save_img: 142 | print('Results saved to %s' % Path(out)) 143 | 144 | print('Done. (%.3fs)' % (time.time() - t0)) 145 | 146 | 147 | if __name__ == '__main__': 148 | parser = argparse.ArgumentParser() 149 | parser.add_argument('--weights', nargs='+', type=str, default='yolov5s.pt', help='model.pt path(s)') 150 | parser.add_argument('--source', type=str, default='inference/images', help='source') # file/folder, 0 for webcam 151 | parser.add_argument('--img-size', type=int, default=640, help='inference size (pixels)') 152 | parser.add_argument('--conf-thres', type=float, default=0.01, help='object confidence threshold') 153 | parser.add_argument('--iou-thres', type=float, default=0.45, help='IOU threshold for NMS') 154 | parser.add_argument('--device', default='', help='cuda device, i.e. 0 or 0,1,2,3 or cpu') 155 | parser.add_argument('--view-img', action='store_true', help='display results') 156 | parser.add_argument('--save-txt', action='store_true', help='save results to *.txt') 157 | parser.add_argument('--save-conf', action='store_true', help='save confidences in --save-txt labels') 158 | parser.add_argument('--save-dir', type=str, default='inference/output', help='directory to save results') 159 | parser.add_argument('--classes', nargs='+', type=int, help='filter by class: --class 0, or --class 0 2 3') 160 | parser.add_argument('--agnostic-nms', action='store_true', help='class-agnostic NMS') 161 | parser.add_argument('--augment', action='store_true', help='augmented inference') 162 | parser.add_argument('--update', action='store_true', help='update all models') 163 | opt = parser.parse_args() 164 | print(opt) 165 | 166 | with torch.no_grad(): 167 | if opt.update: # update all models (to fix SourceChangeWarning) 168 | for opt.weights in ['yolov5s.pt', 'yolov5m.pt', 'yolov5l.pt', 'yolov5x.pt']: 169 | detect() 170 | strip_optimizer(opt.weights) 171 | else: 172 | detect() 173 | -------------------------------------------------------------------------------- /yolov5_test/detection_module.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import sys 3 | sys.path.insert(0, './yolov5_test') 4 | import os 5 | 6 | import shutil 7 | import time 8 | from pathlib import Path 9 | 10 | import cv2 11 | import math 12 | import numpy as np 13 | import torch 14 | from PIL import Image, ExifTags 15 | from torch.utils.data import Dataset 16 | from tqdm import tqdm 17 | from yolov5_test.utils.general import xywh2xyxy, pre_WBF 18 | print(torch.__version__) 19 | print(torch.cuda.is_available()) 20 | import torch.backends.cudnn as cudnn 21 | from numpy import random 22 | 23 | from yolov5_test.models.experimental import attempt_load 24 | from yolov5_test.utils.datasets import LoadStreams, LoadImages 25 | from yolov5_test.utils.general import ( 26 | check_img_size, non_max_suppression, apply_classifier, scale_coords, 27 | xyxy2xywh, plot_one_box, strip_optimizer, set_logging) 28 | from yolov5_test.utils.torch_utils import select_device, load_classifier, time_synchronized 29 | from ensemble_boxes import * 30 | def clip_coords(boxes, img_shape): 31 | # Clip bounding xyxy bounding boxes to image shape (height, width) 32 | boxes[:, 0] = boxes[:, 0].clamp(0, img_shape[1]) # x1 33 | boxes[:, 1] = boxes[:, 1].clamp(0, img_shape[0]) 34 | boxes[:, 2] = boxes[:, 2].clamp(0, img_shape[1]) 35 | boxes[:, 3] = boxes[:, 3].clamp(0, img_shape[0]) 36 | return boxes 37 | 38 | def letterbox(img, new_shape=(640, 640), color=(114, 114, 114), auto=True, scaleFill=False, scaleup=True): 39 | # Resize image to a 32-pixel-multiple rectangle https://github.com/ultralytics/yolov3/issues/232 40 | shape = img.shape[:2] # current shape [height, width] 41 | if isinstance(new_shape, int): 42 | new_shape = (new_shape, new_shape) 43 | 44 | # Scale ratio (new / old) 45 | r = min(new_shape[0] / shape[0], new_shape[1] / shape[1]) 46 | if not scaleup: # only scale down, do not scale up (for better test mAP) 47 | r = min(r, 1.0) 48 | 49 | # Compute padding 50 | ratio = r, r # width, height ratios 51 | new_unpad = int(round(shape[1] * r)), int(round(shape[0] * r)) 52 | dw, dh = new_shape[1] - new_unpad[0], new_shape[0] - new_unpad[1] # wh padding 53 | if auto: # minimum rectangle 54 | dw, dh = np.mod(dw, 32), np.mod(dh, 32) # wh padding 55 | elif scaleFill: # stretch 56 | dw, dh = 0.0, 0.0 57 | new_unpad = (new_shape[1], new_shape[0]) 58 | ratio = new_shape[1] / shape[1], new_shape[0] / shape[0] # width, height ratios 59 | 60 | dw /= 2 # divide padding into 2 sides 61 | dh /= 2 62 | 63 | if shape[::-1] != new_unpad: # resize 64 | img = cv2.resize(img, new_unpad, interpolation=cv2.INTER_LINEAR) 65 | top, bottom = int(round(dh - 0.1)), int(round(dh + 0.1)) 66 | left, right = int(round(dw - 0.1)), int(round(dw + 0.1)) 67 | img = cv2.copyMakeBorder(img, top, bottom, left, right, cv2.BORDER_CONSTANT, value=color) # add border 68 | return img, ratio, (dw, dh) 69 | 70 | class HumanDetector: 71 | def __init__(self, weight_path): 72 | self.confidence_thresh = 0.01 73 | self.iou_thresh = 0.55 74 | self.device = select_device('cuda') 75 | self.half = self.device.type != 'cpu' 76 | self.model = attempt_load(weight_path, map_location=self.device) 77 | # self.imgsz = check_img_size(1280, s=self.model.stride.max()) 78 | if self.half: 79 | self.model.half() 80 | 81 | def predict(self, image, iou, conf): 82 | h0, w0 = image.shape[:2] 83 | r = 1280 / max(h0, w0) # resize image to img_size 84 | interp = cv2.INTER_AREA 85 | img = cv2.resize(image, (int(w0 * r), int(h0 * r)), interpolation=interp) 86 | (h, w) = img.shape[:2] 87 | img, ratio, pad = letterbox(img, new_shape=(1280, 1280)) 88 | shapes = (h0, w0), ((h / h0, w / w0), pad) 89 | img = img[:, :, ::-1].transpose(2, 0, 1) 90 | img = np.ascontiguousarray(img) 91 | img = torch.from_numpy(img).to(self.device) 92 | img = img.half() if self.half else img.float() 93 | img /= 255.0 94 | if img.ndimension() == 3: 95 | img = img.unsqueeze(0) 96 | pred = self.model(img, False)[0] 97 | output = non_max_suppression(pred, conf_thres=conf, iou_thres=iou, merge=False) 98 | try: 99 | nms_result = output[0] 100 | coords = nms_result 101 | coords = clip_coords(coords, (1080, 1920)) 102 | box = coords[:, :4].clone() 103 | confidence_score = coords[:, 4].clone() 104 | coords = scale_coords(img.shape[1:], box, shapes[0], shapes[1]) 105 | coords = coords.cpu().detach().numpy() 106 | 107 | result_coord = [np.round(coord).astype(np.int).tolist() for coord in coords] 108 | confidence_score = [score.cpu().detach().numpy() for score in confidence_score] 109 | except: 110 | return {'img': [], 'label': [], 'score': []} 111 | 112 | return {'img': [image[x[1]:x[3], x[0]:x[2], :] for x in result_coord], 113 | 'label': result_coord 114 | , 'score': confidence_score} 115 | 116 | 117 | 118 | if __name__ == "__main__": 119 | a = Human_detector('./weight/best_.pt') 120 | cv = cv2.imread('/media/data2/AGC/IITP_Track01_Sample/01/GH010171 484.jpg') 121 | import matplotlib.pyplot as plt 122 | aa = a.predict(cv)['img'] 123 | print(aa) 124 | for i in aa: 125 | plt.imshow(i) 126 | plt.show() 127 | 128 | -------------------------------------------------------------------------------- /yolov5_test/hubconf.py: -------------------------------------------------------------------------------- 1 | """File for accessing YOLOv5 via PyTorch Hub https://pytorch.org/hub/ 2 | 3 | Usage: 4 | import torch 5 | model = torch.hub.load('ultralytics/yolov5', 'yolov5s', pretrained=True, channels=3, classes=80) 6 | """ 7 | 8 | dependencies = ['torch', 'yaml'] 9 | import os 10 | 11 | import torch 12 | 13 | from models.yolo import Model 14 | from utils.general import set_logging 15 | from utils.google_utils import attempt_download 16 | 17 | set_logging() 18 | 19 | 20 | def create(name, pretrained, channels, classes): 21 | """Creates a specified YOLOv5 model 22 | 23 | Arguments: 24 | name (str): name of model, i.e. 'yolov5s' 25 | pretrained (bool): load pretrained weights into the model 26 | channels (int): number of input channels 27 | classes (int): number of model classes 28 | 29 | Returns: 30 | pytorch model 31 | """ 32 | config = os.path.join(os.path.dirname(__file__), 'models', f'{name}.yaml') # model.yaml path 33 | try: 34 | model = Model(config, channels, classes) 35 | if pretrained: 36 | fname = f'{name}.pt' # checkpoint filename 37 | attempt_download(fname) # download if not found locally 38 | ckpt = torch.load(fname, map_location=torch.device('cpu')) # load 39 | state_dict = ckpt['model'].float().state_dict() # to FP32 40 | state_dict = {k: v for k, v in state_dict.items() if model.state_dict()[k].shape == v.shape} # filter 41 | model.load_state_dict(state_dict, strict=False) # load 42 | if len(ckpt['model'].names) == classes: 43 | model.names = ckpt['model'].names # set class names attribute 44 | # model = model.autoshape() # for autoshaping of PIL/cv2/np inputs and NMS 45 | return model 46 | 47 | except Exception as e: 48 | help_url = 'https://github.com/ultralytics/yolov5/issues/36' 49 | s = 'Cache maybe be out of date, try force_reload=True. See %s for help.' % help_url 50 | raise Exception(s) from e 51 | 52 | 53 | def yolov5s(pretrained=False, channels=3, classes=80): 54 | """YOLOv5-small model from https://github.com/ultralytics/yolov5 55 | 56 | Arguments: 57 | pretrained (bool): load pretrained weights into the model, default=False 58 | channels (int): number of input channels, default=3 59 | classes (int): number of model classes, default=80 60 | 61 | Returns: 62 | pytorch model 63 | """ 64 | return create('yolov5s', pretrained, channels, classes) 65 | 66 | 67 | def yolov5m(pretrained=False, channels=3, classes=80): 68 | """YOLOv5-medium model from https://github.com/ultralytics/yolov5 69 | 70 | Arguments: 71 | pretrained (bool): load pretrained weights into the model, default=False 72 | channels (int): number of input channels, default=3 73 | classes (int): number of model classes, default=80 74 | 75 | Returns: 76 | pytorch model 77 | """ 78 | return create('yolov5m', pretrained, channels, classes) 79 | 80 | 81 | def yolov5l(pretrained=False, channels=3, classes=80): 82 | """YOLOv5-large model from https://github.com/ultralytics/yolov5 83 | 84 | Arguments: 85 | pretrained (bool): load pretrained weights into the model, default=False 86 | channels (int): number of input channels, default=3 87 | classes (int): number of model classes, default=80 88 | 89 | Returns: 90 | pytorch model 91 | """ 92 | return create('yolov5l', pretrained, channels, classes) 93 | 94 | 95 | def yolov5x(pretrained=False, channels=3, classes=80): 96 | """YOLOv5-xlarge model from https://github.com/ultralytics/yolov5 97 | 98 | Arguments: 99 | pretrained (bool): load pretrained weights into the model, default=False 100 | channels (int): number of input channels, default=3 101 | classes (int): number of model classes, default=80 102 | 103 | Returns: 104 | pytorch model 105 | """ 106 | return create('yolov5x', pretrained, channels, classes) 107 | 108 | 109 | if __name__ == '__main__': 110 | model = create(name='yolov5s', pretrained=True, channels=3, classes=80) # example 111 | model = model.fuse().eval().autoshape() # for autoshaping of PIL/cv2/np inputs and NMS 112 | 113 | # Verify inference 114 | from PIL import Image 115 | 116 | img = Image.open('inference/images/zidane.jpg') 117 | y = model(img) 118 | print(y[0].shape) 119 | -------------------------------------------------------------------------------- /yolov5_test/models/common.py: -------------------------------------------------------------------------------- 1 | # This file contains modules common to various models 2 | 3 | import math 4 | import numpy as np 5 | import torch 6 | import torch.nn as nn 7 | 8 | from yolov5_test.utils.datasets import letterbox 9 | from yolov5_test.utils.general import non_max_suppression, make_divisible, scale_coords 10 | 11 | 12 | def autopad(k, p=None): # kernel, padding 13 | # Pad to 'same' 14 | if p is None: 15 | p = k // 2 if isinstance(k, int) else [x // 2 for x in k] # auto-pad 16 | return p 17 | 18 | 19 | def DWConv(c1, c2, k=1, s=1, act=True): 20 | # Depthwise convolution 21 | return Conv(c1, c2, k, s, g=math.gcd(c1, c2), act=act) 22 | 23 | 24 | class Conv(nn.Module): 25 | # Standard convolution 26 | def __init__(self, c1, c2, k=1, s=1, p=None, g=1, act=True): # ch_in, ch_out, kernel, stride, padding, groups 27 | super(Conv, self).__init__() 28 | self.conv = nn.Conv2d(c1, c2, k, s, autopad(k, p), groups=g, bias=False) 29 | self.bn = nn.BatchNorm2d(c2) 30 | self.act = nn.Hardswish() if act else nn.Identity() 31 | 32 | def forward(self, x): 33 | return self.act(self.bn(self.conv(x))) 34 | 35 | def fuseforward(self, x): 36 | return self.act(self.conv(x)) 37 | 38 | 39 | class Bottleneck(nn.Module): 40 | # Standard bottleneck 41 | def __init__(self, c1, c2, shortcut=True, g=1, e=0.5): # ch_in, ch_out, shortcut, groups, expansion 42 | super(Bottleneck, self).__init__() 43 | c_ = int(c2 * e) # hidden channels 44 | self.cv1 = Conv(c1, c_, 1, 1) 45 | self.cv2 = Conv(c_, c2, 3, 1, g=g) 46 | self.add = shortcut and c1 == c2 47 | 48 | def forward(self, x): 49 | return x + self.cv2(self.cv1(x)) if self.add else self.cv2(self.cv1(x)) 50 | 51 | 52 | class BottleneckCSP(nn.Module): 53 | # CSP Bottleneck https://github.com/WongKinYiu/CrossStagePartialNetworks 54 | def __init__(self, c1, c2, n=1, shortcut=True, g=1, e=0.5): # ch_in, ch_out, number, shortcut, groups, expansion 55 | super(BottleneckCSP, self).__init__() 56 | c_ = int(c2 * e) # hidden channels 57 | self.cv1 = Conv(c1, c_, 1, 1) 58 | self.cv2 = nn.Conv2d(c1, c_, 1, 1, bias=False) 59 | self.cv3 = nn.Conv2d(c_, c_, 1, 1, bias=False) 60 | self.cv4 = Conv(2 * c_, c2, 1, 1) 61 | self.bn = nn.BatchNorm2d(2 * c_) # applied to cat(cv2, cv3) 62 | self.act = nn.LeakyReLU(0.1, inplace=True) 63 | self.m = nn.Sequential(*[Bottleneck(c_, c_, shortcut, g, e=1.0) for _ in range(n)]) 64 | 65 | def forward(self, x): 66 | y1 = self.cv3(self.m(self.cv1(x))) 67 | y2 = self.cv2(x) 68 | return self.cv4(self.act(self.bn(torch.cat((y1, y2), dim=1)))) 69 | 70 | 71 | class SPP(nn.Module): 72 | # Spatial pyramid pooling layer used in YOLOv3-SPP 73 | def __init__(self, c1, c2, k=(5, 9, 13)): 74 | super(SPP, self).__init__() 75 | c_ = c1 // 2 # hidden channels 76 | self.cv1 = Conv(c1, c_, 1, 1) 77 | self.cv2 = Conv(c_ * (len(k) + 1), c2, 1, 1) 78 | self.m = nn.ModuleList([nn.MaxPool2d(kernel_size=x, stride=1, padding=x // 2) for x in k]) 79 | 80 | def forward(self, x): 81 | x = self.cv1(x) 82 | return self.cv2(torch.cat([x] + [m(x) for m in self.m], 1)) 83 | 84 | 85 | class Focus(nn.Module): 86 | # Focus wh information into c-space 87 | def __init__(self, c1, c2, k=1, s=1, p=None, g=1, act=True): # ch_in, ch_out, kernel, stride, padding, groups 88 | super(Focus, self).__init__() 89 | self.conv = Conv(c1 * 4, c2, k, s, p, g, act) 90 | 91 | def forward(self, x): # x(b,c,w,h) -> y(b,4c,w/2,h/2) 92 | return self.conv(torch.cat([x[..., ::2, ::2], x[..., 1::2, ::2], x[..., ::2, 1::2], x[..., 1::2, 1::2]], 1)) 93 | 94 | 95 | class Concat(nn.Module): 96 | # Concatenate a list of tensors along dimension 97 | def __init__(self, dimension=1): 98 | super(Concat, self).__init__() 99 | self.d = dimension 100 | 101 | def forward(self, x): 102 | return torch.cat(x, self.d) 103 | 104 | 105 | class NMS(nn.Module): 106 | # Non-Maximum Suppression (NMS) module 107 | conf = 0.25 # confidence threshold 108 | iou = 0.45 # IoU threshold 109 | classes = None # (optional list) filter by class 110 | 111 | def __init__(self): 112 | super(NMS, self).__init__() 113 | 114 | def forward(self, x): 115 | return non_max_suppression(x[0], conf_thres=self.conf, iou_thres=self.iou, classes=self.classes) 116 | 117 | 118 | class autoShape(nn.Module): 119 | # input-robust model wrapper for passing cv2/np/PIL/torch inputs. Includes preprocessing, inference and NMS 120 | img_size = 640 # inference size (pixels) 121 | conf = 0.25 # NMS confidence threshold 122 | iou = 0.45 # NMS IoU threshold 123 | classes = None # (optional list) filter by class 124 | 125 | def __init__(self, model): 126 | super(autoShape, self).__init__() 127 | self.model = model 128 | 129 | def forward(self, x, size=640, augment=False, profile=False): 130 | # supports inference from various sources. For height=720, width=1280, RGB images example inputs are: 131 | # opencv: x = cv2.imread('image.jpg')[:,:,::-1] # HWC BGR to RGB x(720,1280,3) 132 | # PIL: x = Image.open('image.jpg') # HWC x(720,1280,3) 133 | # numpy: x = np.zeros((720,1280,3)) # HWC 134 | # torch: x = torch.zeros(16,3,720,1280) # BCHW 135 | # multiple: x = [Image.open('image1.jpg'), Image.open('image2.jpg'), ...] # list of images 136 | 137 | p = next(self.model.parameters()) # for device and type 138 | if isinstance(x, torch.Tensor): # torch 139 | return self.model(x.to(p.device).type_as(p), augment, profile) # inference 140 | 141 | # Pre-process 142 | if not isinstance(x, list): 143 | x = [x] 144 | shape0, shape1 = [], [] # image and inference shapes 145 | batch = range(len(x)) # batch size 146 | for i in batch: 147 | x[i] = np.array(x[i])[:, :, :3] # up to 3 channels if png 148 | s = x[i].shape[:2] # HWC 149 | shape0.append(s) # image shape 150 | g = (size / max(s)) # gain 151 | shape1.append([y * g for y in s]) 152 | shape1 = [make_divisible(x, int(self.stride.max())) for x in np.stack(shape1, 0).max(0)] # inference shape 153 | x = [letterbox(x[i], new_shape=shape1, auto=False)[0] for i in batch] # pad 154 | x = np.stack(x, 0) if batch[-1] else x[0][None] # stack 155 | x = np.ascontiguousarray(x.transpose((0, 3, 1, 2))) # BHWC to BCHW 156 | x = torch.from_numpy(x).to(p.device).type_as(p) / 255. # uint8 to fp16/32 157 | 158 | # Inference 159 | x = self.model(x, augment, profile) # forward 160 | x = non_max_suppression(x[0], conf_thres=self.conf, iou_thres=self.iou, classes=self.classes) # NMS 161 | 162 | # Post-process 163 | for i in batch: 164 | if x[i] is not None: 165 | x[i][:, :4] = scale_coords(shape1, x[i][:, :4], shape0[i]) 166 | return x 167 | 168 | 169 | class Flatten(nn.Module): 170 | # Use after nn.AdaptiveAvgPool2d(1) to remove last 2 dimensions 171 | @staticmethod 172 | def forward(x): 173 | return x.view(x.size(0), -1) 174 | 175 | 176 | class Classify(nn.Module): 177 | # Classification head, i.e. x(b,c1,20,20) to x(b,c2) 178 | def __init__(self, c1, c2, k=1, s=1, p=None, g=1): # ch_in, ch_out, kernel, stride, padding, groups 179 | super(Classify, self).__init__() 180 | self.aap = nn.AdaptiveAvgPool2d(1) # to x(b,c1,1,1) 181 | self.conv = nn.Conv2d(c1, c2, k, s, autopad(k, p), groups=g, bias=False) # to x(b,c2,1,1) 182 | self.flat = Flatten() 183 | 184 | def forward(self, x): 185 | z = torch.cat([self.aap(y) for y in (x if isinstance(x, list) else [x])], 1) # cat if list 186 | return self.flat(self.conv(z)) # flatten to x(b,c2) 187 | -------------------------------------------------------------------------------- /yolov5_test/models/experimental.py: -------------------------------------------------------------------------------- 1 | # This file contains experimental modules 2 | 3 | import numpy as np 4 | import torch 5 | import torch.nn as nn 6 | 7 | from yolov5_test.models.common import Conv, DWConv 8 | from yolov5_test.utils.google_utils import attempt_download 9 | 10 | 11 | class CrossConv(nn.Module): 12 | # Cross Convolution Downsample 13 | def __init__(self, c1, c2, k=3, s=1, g=1, e=1.0, shortcut=False): 14 | # ch_in, ch_out, kernel, stride, groups, expansion, shortcut 15 | super(CrossConv, self).__init__() 16 | c_ = int(c2 * e) # hidden channels 17 | self.cv1 = Conv(c1, c_, (1, k), (1, s)) 18 | self.cv2 = Conv(c_, c2, (k, 1), (s, 1), g=g) 19 | self.add = shortcut and c1 == c2 20 | 21 | def forward(self, x): 22 | return x + self.cv2(self.cv1(x)) if self.add else self.cv2(self.cv1(x)) 23 | 24 | 25 | class C3(nn.Module): 26 | # Cross Convolution CSP 27 | def __init__(self, c1, c2, n=1, shortcut=True, g=1, e=0.5): # ch_in, ch_out, number, shortcut, groups, expansion 28 | super(C3, self).__init__() 29 | c_ = int(c2 * e) # hidden channels 30 | self.cv1 = Conv(c1, c_, 1, 1) 31 | self.cv2 = nn.Conv2d(c1, c_, 1, 1, bias=False) 32 | self.cv3 = nn.Conv2d(c_, c_, 1, 1, bias=False) 33 | self.cv4 = Conv(2 * c_, c2, 1, 1) 34 | self.bn = nn.BatchNorm2d(2 * c_) # applied to cat(cv2, cv3) 35 | self.act = nn.LeakyReLU(0.1, inplace=True) 36 | self.m = nn.Sequential(*[CrossConv(c_, c_, 3, 1, g, 1.0, shortcut) for _ in range(n)]) 37 | 38 | def forward(self, x): 39 | y1 = self.cv3(self.m(self.cv1(x))) 40 | y2 = self.cv2(x) 41 | return self.cv4(self.act(self.bn(torch.cat((y1, y2), dim=1)))) 42 | 43 | 44 | class Sum(nn.Module): 45 | # Weighted sum of 2 or more layers https://arxiv.org/abs/1911.09070 46 | def __init__(self, n, weight=False): # n: number of inputs 47 | super(Sum, self).__init__() 48 | self.weight = weight # apply weights boolean 49 | self.iter = range(n - 1) # iter object 50 | if weight: 51 | self.w = nn.Parameter(-torch.arange(1., n) / 2, requires_grad=True) # layer weights 52 | 53 | def forward(self, x): 54 | y = x[0] # no weight 55 | if self.weight: 56 | w = torch.sigmoid(self.w) * 2 57 | for i in self.iter: 58 | y = y + x[i + 1] * w[i] 59 | else: 60 | for i in self.iter: 61 | y = y + x[i + 1] 62 | return y 63 | 64 | 65 | class GhostConv(nn.Module): 66 | # Ghost Convolution https://github.com/huawei-noah/ghostnet 67 | def __init__(self, c1, c2, k=1, s=1, g=1, act=True): # ch_in, ch_out, kernel, stride, groups 68 | super(GhostConv, self).__init__() 69 | c_ = c2 // 2 # hidden channels 70 | self.cv1 = Conv(c1, c_, k, s, None, g, act) 71 | self.cv2 = Conv(c_, c_, 5, 1, None, c_, act) 72 | 73 | def forward(self, x): 74 | y = self.cv1(x) 75 | return torch.cat([y, self.cv2(y)], 1) 76 | 77 | 78 | class GhostBottleneck(nn.Module): 79 | # Ghost Bottleneck https://github.com/huawei-noah/ghostnet 80 | def __init__(self, c1, c2, k, s): 81 | super(GhostBottleneck, self).__init__() 82 | c_ = c2 // 2 83 | self.conv = nn.Sequential(GhostConv(c1, c_, 1, 1), # pw 84 | DWConv(c_, c_, k, s, act=False) if s == 2 else nn.Identity(), # dw 85 | GhostConv(c_, c2, 1, 1, act=False)) # pw-linear 86 | self.shortcut = nn.Sequential(DWConv(c1, c1, k, s, act=False), 87 | Conv(c1, c2, 1, 1, act=False)) if s == 2 else nn.Identity() 88 | 89 | def forward(self, x): 90 | return self.conv(x) + self.shortcut(x) 91 | 92 | 93 | class MixConv2d(nn.Module): 94 | # Mixed Depthwise Conv https://arxiv.org/abs/1907.09595 95 | def __init__(self, c1, c2, k=(1, 3), s=1, equal_ch=True): 96 | super(MixConv2d, self).__init__() 97 | groups = len(k) 98 | if equal_ch: # equal c_ per group 99 | i = torch.linspace(0, groups - 1E-6, c2).floor() # c2 indices 100 | c_ = [(i == g).sum() for g in range(groups)] # intermediate channels 101 | else: # equal weight.numel() per group 102 | b = [c2] + [0] * groups 103 | a = np.eye(groups + 1, groups, k=-1) 104 | a -= np.roll(a, 1, axis=1) 105 | a *= np.array(k) ** 2 106 | a[0] = 1 107 | c_ = np.linalg.lstsq(a, b, rcond=None)[0].round() # solve for equal weight indices, ax = b 108 | 109 | self.m = nn.ModuleList([nn.Conv2d(c1, int(c_[g]), k[g], s, k[g] // 2, bias=False) for g in range(groups)]) 110 | self.bn = nn.BatchNorm2d(c2) 111 | self.act = nn.LeakyReLU(0.1, inplace=True) 112 | 113 | def forward(self, x): 114 | return x + self.act(self.bn(torch.cat([m(x) for m in self.m], 1))) 115 | 116 | 117 | class Ensemble(nn.ModuleList): 118 | # Ensemble of models 119 | def __init__(self): 120 | super(Ensemble, self).__init__() 121 | 122 | def forward(self, x, augment=False): 123 | y = [] 124 | for module in self: 125 | y.append(module(x, augment)[0]) 126 | # y = torch.stack(y).max(0)[0] # max ensemble 127 | # y = torch.cat(y, 1) # nms ensemble 128 | y = torch.stack(y).mean(0) # mean ensemble 129 | return y, None # inference, train output 130 | 131 | 132 | def attempt_load(weights, map_location=None): 133 | # Loads an ensemble of models weights=[a,b,c] or a single model weights=[a] or weights=a 134 | model = Ensemble() 135 | for w in weights if isinstance(weights, list) else [weights]: 136 | attempt_download(w) 137 | model.append(torch.load(w, map_location=map_location)['model'].float().fuse().eval()) # load FP32 model 138 | 139 | # Compatibility updates 140 | for m in model.modules(): 141 | if type(m) in [nn.Hardswish, nn.LeakyReLU, nn.ReLU, nn.ReLU6]: 142 | m.inplace = True # pytorch 1.7.0 compatibility 143 | elif type(m) is Conv: 144 | m._non_persistent_buffers_set = set() # pytorch 1.6.0 compatibility 145 | 146 | if len(model) == 1: 147 | return model[-1] # return model 148 | else: 149 | print('Ensemble created with %s\n' % weights) 150 | for k in ['names', 'stride']: 151 | setattr(model, k, getattr(model[-1], k)) 152 | return model # return ensemble 153 | -------------------------------------------------------------------------------- /yolov5_test/models/export.py: -------------------------------------------------------------------------------- 1 | """Exports a YOLOv5 *.pt model to ONNX and TorchScript formats 2 | 3 | Usage: 4 | $ export PYTHONPATH="$PWD" && python models/export.py --weights ./weights/yolov5s.pt --img 640 --batch 1 5 | """ 6 | 7 | import argparse 8 | import sys 9 | import time 10 | 11 | sys.path.append('./') # to run '$ python *.py' files in subdirectories 12 | 13 | import torch 14 | import torch.nn as nn 15 | 16 | import yolov5_test.models 17 | from yolov5_test.models.experimental import attempt_load 18 | from yolov5_test.utils.activations import Hardswish 19 | from yolov5_test.utils.general import set_logging, check_img_size 20 | 21 | if __name__ == '__main__': 22 | parser = argparse.ArgumentParser() 23 | parser.add_argument('--weights', type=str, default='./yolov5s.pt', help='weights path') # from yolov5/models/ 24 | parser.add_argument('--img-size', nargs='+', type=int, default=[640, 640], help='image size') # height, width 25 | parser.add_argument('--batch-size', type=int, default=1, help='batch size') 26 | opt = parser.parse_args() 27 | opt.img_size *= 2 if len(opt.img_size) == 1 else 1 # expand 28 | print(opt) 29 | set_logging() 30 | t = time.time() 31 | 32 | # Load PyTorch model 33 | model = attempt_load(opt.weights, map_location=torch.device('cpu')) # load FP32 model 34 | labels = model.names 35 | 36 | # Checks 37 | gs = int(max(model.stride)) # grid size (max stride) 38 | opt.img_size = [check_img_size(x, gs) for x in opt.img_size] # verify img_size are gs-multiples 39 | 40 | # Input 41 | img = torch.zeros(opt.batch_size, 3, *opt.img_size) # image size(1,3,320,192) iDetection 42 | 43 | # Update model 44 | for k, m in model.named_modules(): 45 | m._non_persistent_buffers_set = set() # pytorch 1.6.0 compatibility 46 | if isinstance(m, models.common.Conv) and isinstance(m.act, nn.Hardswish): 47 | m.act = Hardswish() # assign activation 48 | # if isinstance(m, models.yolo.Detect): 49 | # m.forward = m.forward_export # assign forward (optional) 50 | model.model[-1].export = True # set Detect() layer export=True 51 | y = model(img) # dry run 52 | 53 | # TorchScript export 54 | try: 55 | print('\nStarting TorchScript export with torch %s...' % torch.__version__) 56 | f = opt.weights.replace('.pt', '.torchscript.pt') # filename 57 | ts = torch.jit.trace(model, img) 58 | ts.save(f) 59 | print('TorchScript export success, saved as %s' % f) 60 | except Exception as e: 61 | print('TorchScript export failure: %s' % e) 62 | 63 | # ONNX export 64 | try: 65 | import onnx 66 | 67 | print('\nStarting ONNX export with onnx %s...' % onnx.__version__) 68 | f = opt.weights.replace('.pt', '.onnx') # filename 69 | torch.onnx.export(model, img, f, verbose=False, opset_version=12, input_names=['images'], 70 | output_names=['classes', 'boxes'] if y is None else ['output']) 71 | 72 | # Checks 73 | onnx_model = onnx.load(f) # load onnx model 74 | onnx.checker.check_model(onnx_model) # check onnx model 75 | # print(onnx.helper.printable_graph(onnx_model.graph)) # print a human readable model 76 | print('ONNX export success, saved as %s' % f) 77 | except Exception as e: 78 | print('ONNX export failure: %s' % e) 79 | 80 | # CoreML export 81 | try: 82 | import coremltools as ct 83 | 84 | print('\nStarting CoreML export with coremltools %s...' % ct.__version__) 85 | # convert model from torchscript and apply pixel scaling as per detect.py 86 | model = ct.convert(ts, inputs=[ct.ImageType(name='image', shape=img.shape, scale=1 / 255.0, bias=[0, 0, 0])]) 87 | f = opt.weights.replace('.pt', '.mlmodel') # filename 88 | model.save(f) 89 | print('CoreML export success, saved as %s' % f) 90 | except Exception as e: 91 | print('CoreML export failure: %s' % e) 92 | 93 | # Finish 94 | print('\nExport complete (%.2fs). Visualize with https://github.com/lutzroeder/netron.' % (time.time() - t)) 95 | -------------------------------------------------------------------------------- /yolov5_test/models/hub/yolov3-spp.yaml: -------------------------------------------------------------------------------- 1 | # parameters 2 | nc: 80 # number of classes 3 | depth_multiple: 1.0 # model depth multiple 4 | width_multiple: 1.0 # layer channel multiple 5 | 6 | # anchors 7 | anchors: 8 | - [10,13, 16,30, 33,23] # P3/8 9 | - [30,61, 62,45, 59,119] # P4/16 10 | - [116,90, 156,198, 373,326] # P5/32 11 | 12 | # darknet53 backbone 13 | backbone: 14 | # [from, number, module, args] 15 | [[-1, 1, Conv, [32, 3, 1]], # 0 16 | [-1, 1, Conv, [64, 3, 2]], # 1-P1/2 17 | [-1, 1, Bottleneck, [64]], 18 | [-1, 1, Conv, [128, 3, 2]], # 3-P2/4 19 | [-1, 2, Bottleneck, [128]], 20 | [-1, 1, Conv, [256, 3, 2]], # 5-P3/8 21 | [-1, 8, Bottleneck, [256]], 22 | [-1, 1, Conv, [512, 3, 2]], # 7-P4/16 23 | [-1, 8, Bottleneck, [512]], 24 | [-1, 1, Conv, [1024, 3, 2]], # 9-P5/32 25 | [-1, 4, Bottleneck, [1024]], # 10 26 | ] 27 | 28 | # YOLOv3-SPP head 29 | head: 30 | [[-1, 1, Bottleneck, [1024, False]], 31 | [-1, 1, SPP, [512, [5, 9, 13]]], 32 | [-1, 1, Conv, [1024, 3, 1]], 33 | [-1, 1, Conv, [512, 1, 1]], 34 | [-1, 1, Conv, [1024, 3, 1]], # 15 (P5/32-large) 35 | 36 | [-2, 1, Conv, [256, 1, 1]], 37 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 38 | [[-1, 8], 1, Concat, [1]], # cat backbone P4 39 | [-1, 1, Bottleneck, [512, False]], 40 | [-1, 1, Bottleneck, [512, False]], 41 | [-1, 1, Conv, [256, 1, 1]], 42 | [-1, 1, Conv, [512, 3, 1]], # 22 (P4/16-medium) 43 | 44 | [-2, 1, Conv, [128, 1, 1]], 45 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 46 | [[-1, 6], 1, Concat, [1]], # cat backbone P3 47 | [-1, 1, Bottleneck, [256, False]], 48 | [-1, 2, Bottleneck, [256, False]], # 27 (P3/8-small) 49 | 50 | [[27, 22, 15], 1, Detect, [nc, anchors]], # Detect(P3, P4, P5) 51 | ] 52 | -------------------------------------------------------------------------------- /yolov5_test/models/hub/yolov5-fpn.yaml: -------------------------------------------------------------------------------- 1 | # parameters 2 | nc: 80 # number of classes 3 | depth_multiple: 1.0 # model depth multiple 4 | width_multiple: 1.0 # layer channel multiple 5 | 6 | # anchors 7 | anchors: 8 | - [10,13, 16,30, 33,23] # P3/8 9 | - [30,61, 62,45, 59,119] # P4/16 10 | - [116,90, 156,198, 373,326] # P5/32 11 | 12 | # YOLOv5 backbone 13 | backbone: 14 | # [from, number, module, args] 15 | [[-1, 1, Focus, [64, 3]], # 0-P1/2 16 | [-1, 1, Conv, [128, 3, 2]], # 1-P2/4 17 | [-1, 3, Bottleneck, [128]], 18 | [-1, 1, Conv, [256, 3, 2]], # 3-P3/8 19 | [-1, 9, BottleneckCSP, [256]], 20 | [-1, 1, Conv, [512, 3, 2]], # 5-P4/16 21 | [-1, 9, BottleneckCSP, [512]], 22 | [-1, 1, Conv, [1024, 3, 2]], # 7-P5/32 23 | [-1, 1, SPP, [1024, [5, 9, 13]]], 24 | [-1, 6, BottleneckCSP, [1024]], # 9 25 | ] 26 | 27 | # YOLOv5 FPN head 28 | head: 29 | [[-1, 3, BottleneckCSP, [1024, False]], # 10 (P5/32-large) 30 | 31 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 32 | [[-1, 6], 1, Concat, [1]], # cat backbone P4 33 | [-1, 1, Conv, [512, 1, 1]], 34 | [-1, 3, BottleneckCSP, [512, False]], # 14 (P4/16-medium) 35 | 36 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 37 | [[-1, 4], 1, Concat, [1]], # cat backbone P3 38 | [-1, 1, Conv, [256, 1, 1]], 39 | [-1, 3, BottleneckCSP, [256, False]], # 18 (P3/8-small) 40 | 41 | [[18, 14, 10], 1, Detect, [nc, anchors]], # Detect(P3, P4, P5) 42 | ] 43 | -------------------------------------------------------------------------------- /yolov5_test/models/hub/yolov5-panet.yaml: -------------------------------------------------------------------------------- 1 | # parameters 2 | nc: 80 # number of classes 3 | depth_multiple: 1.0 # model depth multiple 4 | width_multiple: 1.0 # layer channel multiple 5 | 6 | # anchors 7 | anchors: 8 | - [116,90, 156,198, 373,326] # P5/32 9 | - [30,61, 62,45, 59,119] # P4/16 10 | - [10,13, 16,30, 33,23] # P3/8 11 | 12 | # YOLOv5 backbone 13 | backbone: 14 | # [from, number, module, args] 15 | [[-1, 1, Focus, [64, 3]], # 0-P1/2 16 | [-1, 1, Conv, [128, 3, 2]], # 1-P2/4 17 | [-1, 3, BottleneckCSP, [128]], 18 | [-1, 1, Conv, [256, 3, 2]], # 3-P3/8 19 | [-1, 9, BottleneckCSP, [256]], 20 | [-1, 1, Conv, [512, 3, 2]], # 5-P4/16 21 | [-1, 9, BottleneckCSP, [512]], 22 | [-1, 1, Conv, [1024, 3, 2]], # 7-P5/32 23 | [-1, 1, SPP, [1024, [5, 9, 13]]], 24 | [-1, 3, BottleneckCSP, [1024, False]], # 9 25 | ] 26 | 27 | # YOLOv5 PANet head 28 | head: 29 | [[-1, 1, Conv, [512, 1, 1]], 30 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 31 | [[-1, 6], 1, Concat, [1]], # cat backbone P4 32 | [-1, 3, BottleneckCSP, [512, False]], # 13 33 | 34 | [-1, 1, Conv, [256, 1, 1]], 35 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 36 | [[-1, 4], 1, Concat, [1]], # cat backbone P3 37 | [-1, 3, BottleneckCSP, [256, False]], # 17 (P3/8-small) 38 | 39 | [-1, 1, Conv, [256, 3, 2]], 40 | [[-1, 14], 1, Concat, [1]], # cat head P4 41 | [-1, 3, BottleneckCSP, [512, False]], # 20 (P4/16-medium) 42 | 43 | [-1, 1, Conv, [512, 3, 2]], 44 | [[-1, 10], 1, Concat, [1]], # cat head P5 45 | [-1, 3, BottleneckCSP, [1024, False]], # 23 (P5/32-large) 46 | 47 | [[17, 20, 23], 1, Detect, [nc, anchors]], # Detect(P5, P4, P3) 48 | ] 49 | -------------------------------------------------------------------------------- /yolov5_test/models/yolo.py: -------------------------------------------------------------------------------- 1 | import argparse 2 | import logging 3 | import sys 4 | from copy import deepcopy 5 | from pathlib import Path 6 | 7 | import math 8 | 9 | sys.path.append('./') # to run '$ python *.py' files in subdirectories 10 | logger = logging.getLogger(__name__) 11 | 12 | import torch 13 | import torch.nn as nn 14 | 15 | from yolov5_test.models.common import Conv, Bottleneck, SPP, DWConv, Focus, BottleneckCSP, Concat, NMS, autoShape 16 | from yolov5_test.models.experimental import MixConv2d, CrossConv, C3 17 | from yolov5_test.utils.general import check_anchor_order, make_divisible, check_file, set_logging 18 | from yolov5_test.utils.torch_utils import time_synchronized, fuse_conv_and_bn, model_info, scale_img, initialize_weights, \ 19 | select_device, copy_attr 20 | 21 | 22 | class Detect(nn.Module): 23 | stride = None # strides computed during build 24 | export = False # onnx export 25 | 26 | def __init__(self, nc=80, anchors=(), ch=()): # detection layer 27 | super(Detect, self).__init__() 28 | self.nc = nc # number of classes 29 | self.no = nc + 5 # number of outputs per anchor 30 | self.nl = len(anchors) # number of detection layers 31 | self.na = len(anchors[0]) // 2 # number of anchors 32 | self.grid = [torch.zeros(1)] * self.nl # init grid 33 | a = torch.tensor(anchors).float().view(self.nl, -1, 2) 34 | self.register_buffer('anchors', a) # shape(nl,na,2) 35 | self.register_buffer('anchor_grid', a.clone().view(self.nl, 1, -1, 1, 1, 2)) # shape(nl,1,na,1,1,2) 36 | self.m = nn.ModuleList(nn.Conv2d(x, self.no * self.na, 1) for x in ch) # output conv 37 | 38 | def forward(self, x): 39 | # x = x.copy() # for profiling 40 | z = [] # inference output 41 | self.training |= self.export 42 | for i in range(self.nl): 43 | x[i] = self.m[i](x[i]) # conv 44 | bs, _, ny, nx = x[i].shape # x(bs,255,20,20) to x(bs,3,20,20,85) 45 | x[i] = x[i].view(bs, self.na, self.no, ny, nx).permute(0, 1, 3, 4, 2).contiguous() 46 | 47 | if not self.training: # inference 48 | if self.grid[i].shape[2:4] != x[i].shape[2:4]: 49 | self.grid[i] = self._make_grid(nx, ny).to(x[i].device) 50 | 51 | y = x[i].sigmoid() 52 | y[..., 0:2] = (y[..., 0:2] * 2. - 0.5 + self.grid[i].to(x[i].device)) * self.stride[i] # xy 53 | y[..., 2:4] = (y[..., 2:4] * 2) ** 2 * self.anchor_grid[i] # wh 54 | z.append(y.view(bs, -1, self.no)) 55 | 56 | return x if self.training else (torch.cat(z, 1), x) 57 | 58 | @staticmethod 59 | def _make_grid(nx=20, ny=20): 60 | yv, xv = torch.meshgrid([torch.arange(ny), torch.arange(nx)]) 61 | return torch.stack((xv, yv), 2).view((1, 1, ny, nx, 2)).float() 62 | 63 | 64 | class Model(nn.Module): 65 | def __init__(self, cfg='yolov5s.yaml', ch=3, nc=None): # model, input channels, number of classes 66 | super(Model, self).__init__() 67 | if isinstance(cfg, dict): 68 | self.yaml = cfg # model dict 69 | else: # is *.yaml 70 | import yaml # for torch hub 71 | self.yaml_file = Path(cfg).name 72 | with open(cfg) as f: 73 | self.yaml = yaml.load(f, Loader=yaml.FullLoader) # model dict 74 | 75 | # Define model 76 | if nc and nc != self.yaml['nc']: 77 | print('Overriding model.yaml nc=%g with nc=%g' % (self.yaml['nc'], nc)) 78 | self.yaml['nc'] = nc # override yaml value 79 | self.model, self.save = parse_model(deepcopy(self.yaml), ch=[ch]) # model, savelist, ch_out 80 | # print([x.shape for x in self.forward(torch.zeros(1, ch, 64, 64))]) 81 | 82 | # Build strides, anchors 83 | m = self.model[-1] # Detect() 84 | if isinstance(m, Detect): 85 | s = 128 # 2x min stride 86 | m.stride = torch.tensor([s / x.shape[-2] for x in self.forward(torch.zeros(1, ch, s, s))]) # forward 87 | m.anchors /= m.stride.view(-1, 1, 1) 88 | check_anchor_order(m) 89 | self.stride = m.stride 90 | self._initialize_biases() # only run once 91 | # print('Strides: %s' % m.stride.tolist()) 92 | 93 | # Init weights, biases 94 | initialize_weights(self) 95 | self.info() 96 | print('') 97 | 98 | def forward(self, x, augment=False, profile=False): 99 | if augment: 100 | img_size = x.shape[-2:] # height, width 101 | s = [1, 0.83, 0.67] # scales 102 | f = [None, 3, None] # flips (2-ud, 3-lr) 103 | y = [] # outputs 104 | for si, fi in zip(s, f): 105 | xi = scale_img(x.flip(fi) if fi else x, si) 106 | yi = self.forward_once(xi)[0] # forward 107 | # cv2.imwrite('img%g.jpg' % s, 255 * xi[0].numpy().transpose((1, 2, 0))[:, :, ::-1]) # save 108 | yi[..., :4] /= si # de-scale 109 | if fi == 2: 110 | yi[..., 1] = img_size[0] - yi[..., 1] # de-flip ud 111 | elif fi == 3: 112 | yi[..., 0] = img_size[1] - yi[..., 0] # de-flip lr 113 | y.append(yi) 114 | return torch.cat(y, 1), None # augmented inference, train 115 | else: 116 | return self.forward_once(x, profile) # single-scale inference, train 117 | 118 | def forward_once(self, x, profile=False): 119 | y, dt = [], [] # outputs 120 | for m in self.model: 121 | if m.f != -1: # if not from previous layer 122 | x = y[m.f] if isinstance(m.f, int) else [x if j == -1 else y[j] for j in m.f] # from earlier layers 123 | 124 | if profile: 125 | try: 126 | import thop 127 | o = thop.profile(m, inputs=(x,), verbose=False)[0] / 1E9 * 2 # FLOPS 128 | except: 129 | o = 0 130 | t = time_synchronized() 131 | for _ in range(10): 132 | _ = m(x) 133 | dt.append((time_synchronized() - t) * 100) 134 | print('%10.1f%10.0f%10.1fms %-40s' % (o, m.np, dt[-1], m.type)) 135 | 136 | x = m(x) # run 137 | y.append(x if m.i in self.save else None) # save output 138 | 139 | if profile: 140 | print('%.1fms total' % sum(dt)) 141 | return x 142 | 143 | def _initialize_biases(self, cf=None): # initialize biases into Detect(), cf is class frequency 144 | # https://arxiv.org/abs/1708.02002 section 3.3 145 | # cf = torch.bincount(torch.tensor(np.concatenate(dataset.labels, 0)[:, 0]).long(), minlength=nc) + 1. 146 | m = self.model[-1] # Detect() module 147 | for mi, s in zip(m.m, m.stride): # from 148 | b = mi.bias.view(m.na, -1) # conv.bias(255) to (3,85) 149 | b[:, 4] += math.log(8 / (640 / s) ** 2) # obj (8 objects per 640 image) 150 | b[:, 5:] += math.log(0.6 / (m.nc - 0.99)) if cf is None else torch.log(cf / cf.sum()) # cls 151 | mi.bias = torch.nn.Parameter(b.view(-1), requires_grad=True) 152 | 153 | def _print_biases(self): 154 | m = self.model[-1] # Detect() module 155 | for mi in m.m: # from 156 | b = mi.bias.detach().view(m.na, -1).T # conv.bias(255) to (3,85) 157 | print(('%6g Conv2d.bias:' + '%10.3g' * 6) % (mi.weight.shape[1], *b[:5].mean(1).tolist(), b[5:].mean())) 158 | 159 | # def _print_weights(self): 160 | # for m in self.model.modules(): 161 | # if type(m) is Bottleneck: 162 | # print('%10.3g' % (m.w.detach().sigmoid() * 2)) # shortcut weights 163 | 164 | def fuse(self): # fuse model Conv2d() + BatchNorm2d() layers 165 | print('Fusing layers... ') 166 | for m in self.model.modules(): 167 | if type(m) is Conv and hasattr(m, 'bn'): 168 | m.conv = fuse_conv_and_bn(m.conv, m.bn) # update conv 169 | delattr(m, 'bn') # remove batchnorm 170 | m.forward = m.fuseforward # update forward 171 | self.info() 172 | return self 173 | 174 | def nms(self, mode=True): # add or remove NMS module 175 | present = type(self.model[-1]) is NMS # last layer is NMS 176 | if mode and not present: 177 | print('Adding NMS... ') 178 | m = NMS() # module 179 | m.f = -1 # from 180 | m.i = self.model[-1].i + 1 # index 181 | self.model.add_module(name='%s' % m.i, module=m) # add 182 | self.eval() 183 | elif not mode and present: 184 | print('Removing NMS... ') 185 | self.model = self.model[:-1] # remove 186 | return self 187 | 188 | def autoshape(self): # add autoShape module 189 | print('Adding autoShape... ') 190 | m = autoShape(self) # wrap model 191 | copy_attr(m, self, include=('yaml', 'nc', 'hyp', 'names', 'stride'), exclude=()) # copy attributes 192 | return m 193 | 194 | def info(self, verbose=False): # print model information 195 | model_info(self, verbose) 196 | 197 | 198 | def parse_model(d, ch): # model_dict, input_channels(3) 199 | logger.info('\n%3s%18s%3s%10s %-40s%-30s' % ('', 'from', 'n', 'params', 'module', 'arguments')) 200 | anchors, nc, gd, gw = d['anchors'], d['nc'], d['depth_multiple'], d['width_multiple'] 201 | na = (len(anchors[0]) // 2) if isinstance(anchors, list) else anchors # number of anchors 202 | no = na * (nc + 5) # number of outputs = anchors * (classes + 5) 203 | 204 | layers, save, c2 = [], [], ch[-1] # layers, savelist, ch out 205 | for i, (f, n, m, args) in enumerate(d['backbone'] + d['head']): # from, number, module, args 206 | m = eval(m) if isinstance(m, str) else m # eval strings 207 | for j, a in enumerate(args): 208 | try: 209 | args[j] = eval(a) if isinstance(a, str) else a # eval strings 210 | except: 211 | pass 212 | 213 | n = max(round(n * gd), 1) if n > 1 else n # depth gain 214 | if m in [Conv, Bottleneck, SPP, DWConv, MixConv2d, Focus, CrossConv, BottleneckCSP, C3]: 215 | c1, c2 = ch[f], args[0] 216 | 217 | # Normal 218 | # if i > 0 and args[0] != no: # channel expansion factor 219 | # ex = 1.75 # exponential (default 2.0) 220 | # e = math.log(c2 / ch[1]) / math.log(2) 221 | # c2 = int(ch[1] * ex ** e) 222 | # if m != Focus: 223 | 224 | c2 = make_divisible(c2 * gw, 8) if c2 != no else c2 225 | 226 | # Experimental 227 | # if i > 0 and args[0] != no: # channel expansion factor 228 | # ex = 1 + gw # exponential (default 2.0) 229 | # ch1 = 32 # ch[1] 230 | # e = math.log(c2 / ch1) / math.log(2) # level 1-n 231 | # c2 = int(ch1 * ex ** e) 232 | # if m != Focus: 233 | # c2 = make_divisible(c2, 8) if c2 != no else c2 234 | 235 | args = [c1, c2, *args[1:]] 236 | if m in [BottleneckCSP, C3]: 237 | args.insert(2, n) 238 | n = 1 239 | elif m is nn.BatchNorm2d: 240 | args = [ch[f]] 241 | elif m is Concat: 242 | c2 = sum([ch[-1 if x == -1 else x + 1] for x in f]) 243 | elif m is Detect: 244 | args.append([ch[x + 1] for x in f]) 245 | if isinstance(args[1], int): # number of anchors 246 | args[1] = [list(range(args[1] * 2))] * len(f) 247 | else: 248 | c2 = ch[f] 249 | 250 | m_ = nn.Sequential(*[m(*args) for _ in range(n)]) if n > 1 else m(*args) # module 251 | t = str(m)[8:-2].replace('__main__.', '') # module type 252 | np = sum([x.numel() for x in m_.parameters()]) # number params 253 | m_.i, m_.f, m_.type, m_.np = i, f, t, np # attach index, 'from' index, type, number params 254 | logger.info('%3s%18s%3s%10.0f %-40s%-30s' % (i, f, n, np, t, args)) # print 255 | save.extend(x % i for x in ([f] if isinstance(f, int) else f) if x != -1) # append to savelist 256 | layers.append(m_) 257 | ch.append(c2) 258 | return nn.Sequential(*layers), sorted(save) 259 | 260 | 261 | if __name__ == '__main__': 262 | parser = argparse.ArgumentParser() 263 | parser.add_argument('--cfg', type=str, default='yolov5s.yaml', help='model.yaml') 264 | parser.add_argument('--device', default='', help='cuda device, i.e. 0 or 0,1,2,3 or cpu') 265 | opt = parser.parse_args() 266 | opt.cfg = check_file(opt.cfg) # check file 267 | set_logging() 268 | device = select_device(opt.device) 269 | 270 | # Create model 271 | model = Model(opt.cfg).to(device) 272 | model.train() 273 | 274 | # Profile 275 | # img = torch.rand(8 if torch.cuda.is_available() else 1, 3, 640, 640).to(device) 276 | # y = model(img, profile=True) 277 | 278 | # Tensorboard 279 | # from torch.utils.tensorboard import SummaryWriter 280 | # tb_writer = SummaryWriter() 281 | # print("Run 'tensorboard --logdir=models/runs' to view tensorboard at http://localhost:6006/") 282 | # tb_writer.add_graph(model.model, img) # add model to tensorboard 283 | # tb_writer.add_image('test', img[0], dataformats='CWH') # add model to tensorboard 284 | -------------------------------------------------------------------------------- /yolov5_test/models/yolov5l.yaml: -------------------------------------------------------------------------------- 1 | # parameters 2 | nc: 80 # number of classes 3 | depth_multiple: 1.0 # model depth multiple 4 | width_multiple: 1.0 # layer channel multiple 5 | 6 | # anchors 7 | anchors: 8 | - [10,13, 16,30, 33,23] # P3/8 9 | - [30,61, 62,45, 59,119] # P4/16 10 | - [116,90, 156,198, 373,326] # P5/32 11 | 12 | # YOLOv5 backbone 13 | backbone: 14 | # [from, number, module, args] 15 | [[-1, 1, Focus, [64, 3]], # 0-P1/2 16 | [-1, 1, Conv, [128, 3, 2]], # 1-P2/4 17 | [-1, 3, BottleneckCSP, [128]], 18 | [-1, 1, Conv, [256, 3, 2]], # 3-P3/8 19 | [-1, 9, BottleneckCSP, [256]], 20 | [-1, 1, Conv, [512, 3, 2]], # 5-P4/16 21 | [-1, 9, BottleneckCSP, [512]], 22 | [-1, 1, Conv, [1024, 3, 2]], # 7-P5/32 23 | [-1, 1, SPP, [1024, [5, 9, 13]]], 24 | [-1, 3, BottleneckCSP, [1024, False]], # 9 25 | ] 26 | 27 | # YOLOv5 head 28 | head: 29 | [[-1, 1, Conv, [512, 1, 1]], 30 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 31 | [[-1, 6], 1, Concat, [1]], # cat backbone P4 32 | [-1, 3, BottleneckCSP, [512, False]], # 13 33 | 34 | [-1, 1, Conv, [256, 1, 1]], 35 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 36 | [[-1, 4], 1, Concat, [1]], # cat backbone P3 37 | [-1, 3, BottleneckCSP, [256, False]], # 17 (P3/8-small) 38 | 39 | [-1, 1, Conv, [256, 3, 2]], 40 | [[-1, 14], 1, Concat, [1]], # cat head P4 41 | [-1, 3, BottleneckCSP, [512, False]], # 20 (P4/16-medium) 42 | 43 | [-1, 1, Conv, [512, 3, 2]], 44 | [[-1, 10], 1, Concat, [1]], # cat head P5 45 | [-1, 3, BottleneckCSP, [1024, False]], # 23 (P5/32-large) 46 | 47 | [[17, 20, 23], 1, Detect, [nc, anchors]], # Detect(P3, P4, P5) 48 | ] 49 | -------------------------------------------------------------------------------- /yolov5_test/models/yolov5m.yaml: -------------------------------------------------------------------------------- 1 | # parameters 2 | nc: 80 # number of classes 3 | depth_multiple: 0.67 # model depth multiple 4 | width_multiple: 0.75 # layer channel multiple 5 | 6 | # anchors 7 | anchors: 8 | - [10,13, 16,30, 33,23] # P3/8 9 | - [30,61, 62,45, 59,119] # P4/16 10 | - [116,90, 156,198, 373,326] # P5/32 11 | 12 | # YOLOv5 backbone 13 | backbone: 14 | # [from, number, module, args] 15 | [[-1, 1, Focus, [64, 3]], # 0-P1/2 16 | [-1, 1, Conv, [128, 3, 2]], # 1-P2/4 17 | [-1, 3, BottleneckCSP, [128]], 18 | [-1, 1, Conv, [256, 3, 2]], # 3-P3/8 19 | [-1, 9, BottleneckCSP, [256]], 20 | [-1, 1, Conv, [512, 3, 2]], # 5-P4/16 21 | [-1, 9, BottleneckCSP, [512]], 22 | [-1, 1, Conv, [1024, 3, 2]], # 7-P5/32 23 | [-1, 1, SPP, [1024, [5, 9, 13]]], 24 | [-1, 3, BottleneckCSP, [1024, False]], # 9 25 | ] 26 | 27 | # YOLOv5 head 28 | head: 29 | [[-1, 1, Conv, [512, 1, 1]], 30 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 31 | [[-1, 6], 1, Concat, [1]], # cat backbone P4 32 | [-1, 3, BottleneckCSP, [512, False]], # 13 33 | 34 | [-1, 1, Conv, [256, 1, 1]], 35 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 36 | [[-1, 4], 1, Concat, [1]], # cat backbone P3 37 | [-1, 3, BottleneckCSP, [256, False]], # 17 (P3/8-small) 38 | 39 | [-1, 1, Conv, [256, 3, 2]], 40 | [[-1, 14], 1, Concat, [1]], # cat head P4 41 | [-1, 3, BottleneckCSP, [512, False]], # 20 (P4/16-medium) 42 | 43 | [-1, 1, Conv, [512, 3, 2]], 44 | [[-1, 10], 1, Concat, [1]], # cat head P5 45 | [-1, 3, BottleneckCSP, [1024, False]], # 23 (P5/32-large) 46 | 47 | [[17, 20, 23], 1, Detect, [nc, anchors]], # Detect(P3, P4, P5) 48 | ] 49 | -------------------------------------------------------------------------------- /yolov5_test/models/yolov5s.yaml: -------------------------------------------------------------------------------- 1 | # parameters 2 | nc: 80 # number of classes 3 | depth_multiple: 0.33 # model depth multiple 4 | width_multiple: 0.50 # layer channel multiple 5 | 6 | # anchors 7 | anchors: 8 | - [10,13, 16,30, 33,23] # P3/8 9 | - [30,61, 62,45, 59,119] # P4/16 10 | - [116,90, 156,198, 373,326] # P5/32 11 | 12 | # YOLOv5 backbone 13 | backbone: 14 | # [from, number, module, args] 15 | [[-1, 1, Focus, [64, 3]], # 0-P1/2 16 | [-1, 1, Conv, [128, 3, 2]], # 1-P2/4 17 | [-1, 3, BottleneckCSP, [128]], 18 | [-1, 1, Conv, [256, 3, 2]], # 3-P3/8 19 | [-1, 9, BottleneckCSP, [256]], 20 | [-1, 1, Conv, [512, 3, 2]], # 5-P4/16 21 | [-1, 9, BottleneckCSP, [512]], 22 | [-1, 1, Conv, [1024, 3, 2]], # 7-P5/32 23 | [-1, 1, SPP, [1024, [5, 9, 13]]], 24 | [-1, 3, BottleneckCSP, [1024, False]], # 9 25 | ] 26 | 27 | # YOLOv5 head 28 | head: 29 | [[-1, 1, Conv, [512, 1, 1]], 30 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 31 | [[-1, 6], 1, Concat, [1]], # cat backbone P4 32 | [-1, 3, BottleneckCSP, [512, False]], # 13 33 | 34 | [-1, 1, Conv, [256, 1, 1]], 35 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 36 | [[-1, 4], 1, Concat, [1]], # cat backbone P3 37 | [-1, 3, BottleneckCSP, [256, False]], # 17 (P3/8-small) 38 | 39 | [-1, 1, Conv, [256, 3, 2]], 40 | [[-1, 14], 1, Concat, [1]], # cat head P4 41 | [-1, 3, BottleneckCSP, [512, False]], # 20 (P4/16-medium) 42 | 43 | [-1, 1, Conv, [512, 3, 2]], 44 | [[-1, 10], 1, Concat, [1]], # cat head P5 45 | [-1, 3, BottleneckCSP, [1024, False]], # 23 (P5/32-large) 46 | 47 | [[17, 20, 23], 1, Detect, [nc, anchors]], # Detect(P3, P4, P5) 48 | ] 49 | -------------------------------------------------------------------------------- /yolov5_test/models/yolov5x.yaml: -------------------------------------------------------------------------------- 1 | # parameters 2 | nc: 1 # number of classes 3 | depth_multiple: 1.33 # model depth multiple 4 | width_multiple: 1.25 # layer channel multiple 5 | 6 | # anchors 7 | anchors: 8 | - [10,13, 16,30, 33,23] # P3/8 9 | - [30,61, 62,45, 59,119] # P4/16 10 | - [116,90, 156,198, 373,326] # P5/32 11 | 12 | # YOLOv5 backbone 13 | backbone: 14 | # [from, number, module, args] 15 | [[-1, 1, Focus, [64, 3]], # 0-P1/2 16 | [-1, 1, Conv, [128, 3, 2]], # 1-P2/4 17 | [-1, 3, BottleneckCSP, [128]], 18 | [-1, 1, Conv, [256, 3, 2]], # 3-P3/8 19 | [-1, 9, BottleneckCSP, [256]], 20 | [-1, 1, Conv, [512, 3, 2]], # 5-P4/16 21 | [-1, 9, BottleneckCSP, [512]], 22 | [-1, 1, Conv, [1024, 3, 2]], # 7-P5/32 23 | [-1, 1, SPP, [1024, [5, 9, 13]]], 24 | [-1, 3, BottleneckCSP, [1024, False]], # 9 25 | ] 26 | 27 | # YOLOv5 head 28 | head: 29 | [[-1, 1, Conv, [512, 1, 1]], 30 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 31 | [[-1, 6], 1, Concat, [1]], # cat backbone P4 32 | [-1, 3, BottleneckCSP, [512, False]], # 13 33 | 34 | [-1, 1, Conv, [256, 1, 1]], 35 | [-1, 1, nn.Upsample, [None, 2, 'nearest']], 36 | [[-1, 4], 1, Concat, [1]], # cat backbone P3 37 | [-1, 3, BottleneckCSP, [256, False]], # 17 (P3/8-small) 38 | 39 | [-1, 1, Conv, [256, 3, 2]], 40 | [[-1, 14], 1, Concat, [1]], # cat head P4 41 | [-1, 3, BottleneckCSP, [512, False]], # 20 (P4/16-medium) 42 | 43 | [-1, 1, Conv, [512, 3, 2]], 44 | [[-1, 10], 1, Concat, [1]], # cat head P5 45 | [-1, 3, BottleneckCSP, [1024, False]], # 23 (P5/32-large) 46 | 47 | [[17, 20, 23], 1, Detect, [nc, anchors]], # Detect(P3, P4, P5) 48 | ] 49 | -------------------------------------------------------------------------------- /yolov5_test/readme: -------------------------------------------------------------------------------- 1 | this folder doesn't include weight file. 2 | So, you can generate own weight file from yolov5 training 3 | --- 4 | --------------------------------------------------------------------------------