├── LICENSE ├── LLNL-CODE-747688 ├── README.md ├── _config.yml ├── caffe_data ├── README.md ├── train_val_AlexNet-Custom_single.prototxt ├── train_val_AlexNet-Custom_triple.prototxt ├── train_val_CaffeNet_single.prototxt ├── train_val_CaffeNet_triple.prototxt ├── train_val_Inception_single.prototxt ├── train_val_Inception_triple.prototxt ├── train_val_ResCeption_single.prototxt └── train_val_ResCeption_triple.prototxt ├── figures └── README.md ├── moab_output └── README.md ├── py_proj └── README.md ├── python ├── LoadFunctions.py ├── LoadTrainImages.py ├── NumpyFileList.py ├── PrefetchTrain.py ├── README.md └── RunTripleContextTrain.py ├── scripts └── README.txt └── tests ├── CUB ├── dataset.txt ├── solver_AlexNet.prototxt ├── test_list.txt └── train_list.txt ├── CompCars ├── dataset.txt ├── solver_AlexNet.prototxt ├── test_list.txt └── train_list.txt └── README.md /LICENSE: -------------------------------------------------------------------------------- 1 | The GNU Lesser General Public License (Lesser GPU) Version 2.1, 2 | February 1999 3 | 4 | Copyright (c) 2018, Lawrence Livermore National Security, 5 | LLC. Produced at the Lawrence Livermore National Laboratory. Written 6 | by T. Nathan Mundhenk. CODE-747688. All rights 7 | reserved. This file is part of Self Supervised . For details, https://gdo-datasci.llnl.gov/selfsupervised/. 9 | Please also read this link - Our Notice and GNU Lesser General Public 10 | License. This program is free software; you can redistribute it 11 | and/or modify it under the terms of the GNU General Public License (as 12 | published by the Free Software Foundation) version 2.1 dated February 13 | 1999. This program is distributed in the hope that it will be useful, 14 | but WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF 15 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the terms and 16 | conditions of the GNU General Public License for more details. You 17 | should have received a copy of the GNU Lesser General Public License 18 | along with this program; if not, write to the Free Software 19 | Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 20 | USA OUR NOTICE AND TERMS AND CONDITIONS OF THE GNU GENERAL PUBLIC 21 | LICENSE Our Preamble Notice A. This notice is required to be provided 22 | under our contract with the U.S. Department of Energy (DOE). This work 23 | was produced at the Lawrence Livermore National Laboratory under 24 | Contract No. DE-AC52-07NA27344 with the DOE. B. Neither the United 25 | States Government nor Lawrence Livermore National Security, LLC nor 26 | any of their employees, makes any warranty, express or implied, or 27 | assumes any liability or responsibility for the accuracy, 28 | completeness, or usefulness of any information, apparatus, product, or 29 | process disclosed, or represents that its use would not infringe 30 | privately-owned rights. C. Also, reference herein to any specific 31 | commercial products, process, or services by trade name, trademark, 32 | manufacturer or otherwise does not necessarily constitute or imply its 33 | endorsement, recommendation, or favoring by the United States 34 | Government or Lawrence Livermore National Security, LLC. The views 35 | and opinions of authors expressed herein do not necessarily state or 36 | reflect those of the United States Government or Lawrence Livermore 37 | National Security, LLC, and shall not be used for advertising or 38 | product endorsement purposes. The precise terms and conditions for 39 | copying, distribution and modification follows. GNU Lesser GPL terms 40 | and Conditions for Copying, Distribution, and Modification 0. This 41 | License Agreement applies to any software library or other program 42 | which contains a notice placed by the copyright holder or other 43 | authorized party saying it may be distributed under the terms of this 44 | Lesser General Public License (also called "this License"). Each 45 | licensee is addressed as "you". A "library" means a collection of 46 | software functions and/or data prepared so as to be conveniently 47 | linked with application programs (which use some of those functions 48 | and data) to form executables. The "Library", below, refers to any 49 | such software library or work which has been distributed under these 50 | terms. A "work based on the Library" means either the Library or any 51 | derivative work under copyright law: that is to say, a work containing 52 | the Library or a portion of it, either verbatim or with modifications 53 | and/or translated straightforwardly into another 54 | language. (Hereinafter, translation is included without limitation in 55 | the term "modification".) "Source code" for a work means the 56 | preferred form of the work for making modifications to it. For a 57 | library, complete source code means all the source code for all 58 | modules it contains, plus any associated interface definition files, 59 | plus the scripts used to control compilation and installation of the 60 | library. Activities other than copying, distribution and modification 61 | are not covered by this License; they are outside its scope. The act 62 | of running a program using the Library is not restricted, and output 63 | from such a program is covered only if its contents constitute a work 64 | based on the Library (independent of the use of the Library in a tool 65 | for writing it). Whether that is true depends on what the Library does 66 | and what the program that uses the Library does. 0 1. You may copy 67 | and distribute verbatim copies of the Library's complete source code 68 | as you receive it, in any medium, provided that you conspicuously and 69 | appropriately publish on each copy an appropriate copyright notice and 70 | disclaimer of warranty; keep intact all the notices that refer to this 71 | License and to the absence of any warranty; and distribute a copy of 72 | this License along with the Library. 1 You may charge a fee for the 73 | physical act of transferring a copy, and you may at your option offer 74 | warranty protection in exchange for a fee. 2 You may modify your copy 75 | or copies of the Library or any portion of it, thus forming a work 76 | based on the Library, and copy and distribute such modifications or 77 | work under the terms of Section 1 above, provided that you also meet 78 | all of these conditions: 79 | 80 | a) The modified work must itself be a software library. b) You must 81 | cause the files modified to carry prominent notices stating that you 82 | changed the files and the date of any change. c) You must cause the 83 | whole of the work to be licensed at no charge to all third parties 84 | under the terms of this License. d) If a facility in the modified 85 | Library refers to a function or a table of data to be supplied by an 86 | application program that uses the facility, other than as an argument 87 | passed when the facility is invoked, then you must make a good faith 88 | effort to ensure that, in the event an application does not supply 89 | such function or table, the facility still operates, and performs 90 | whatever part of its purpose remains meaningful. (For example, a 91 | function in a library to compute square roots has a purpose that is 92 | entirely well-defined independent of the application. Therefore, 93 | Subsection 2d requires that any application-supplied function or table 94 | used by this function must be optional: if the application does not 95 | supply it, the square root function must still compute square roots.) 96 | These requirements apply to the modified work as a whole. If 97 | identifiable sections of that work are not derived from the Library, 98 | and can be reasonably considered independent and separate works in 99 | themselves, then this License, and its terms, do not apply to those 100 | sections when you distribute them as separate works. But when you 101 | distribute the same sections as part of a whole which is a work based 102 | on the Library, the distribution of the whole must be on the terms of 103 | this License, whose permissions for other licensees extend to the 104 | entire whole, and thus to each and every part regardless of who wrote 105 | it. Thus, it is not the intent of this section to claim rights or 106 | contest your rights to work written entirely by you; rather, the 107 | intent is to exercise the right to control the distribution of 108 | derivative or collective works based on the Library. In addition, 109 | mere aggregation of another work not based on the Library with the 110 | Library (or with a work based on the Library) on a volume of a storage 111 | or distribution medium does not bring the other work under the scope 112 | of this License. 3. You may opt to apply the terms of the ordinary 113 | GNU General Public License instead of this License to a given copy of 114 | the Library. To do this, you must alter all the notices that refer to 115 | this License, so that they refer to the ordinary GNU General Public 116 | License, version 2, instead of to this License. (If a newer version 117 | than version 2 of the ordinary GNU General Public License has 118 | appeared, then you can specify that version instead if you wish.) Do 119 | not make any other change in these notices. Once this change is made 120 | in a given copy, it is irreversible for that copy, so the ordinary GNU 121 | General Public License applies to all subsequent copies and derivative 122 | works made from that copy. This option is useful when you wish to 123 | copy part of the code of the Library into a program that is not a 124 | library. 0 4. You may copy and distribute the Library (or a portion 125 | or derivative of it, under Section 2) in object code or executable 126 | form under the terms of Sections 1 and 2 above provided that you 127 | accompany it with the complete corresponding machine-readable source 128 | code, which must be distributed under the terms of Sections 1 and 2 129 | above on a medium customarily used for software interchange. 1 If 130 | distribution of object code is made by offering access to copy from a 131 | designated place, then offering equivalent access to copy the source 132 | code from the same place satisfies the requirement to distribute the 133 | source code, even though third parties are not compelled to copy the 134 | source along with the object code. 2 A program that contains no 135 | derivative of any portion of the Library, but is designed to work with 136 | the Library by being compiled or linked with it, is called a "work 137 | that uses the Library". Such a work, in isolation, is not a 138 | derivative work of the Library, and therefore falls outside the scope 139 | of this License. 140 | 141 | However, linking a "work that uses the Library" with the Library 142 | creates an executable that is a derivative of the Library (because it 143 | contains portions of the Library), rather than a "work that uses the 144 | library". The executable is therefore covered by this License. 145 | Section 6 states terms for distribution of such executables. When a 146 | "work that uses the Library" uses material from a header file that is 147 | part of the Library, the object code for the work may be a derivative 148 | work of the Library even though the source code is not. Whether this 149 | is true is especially significant if the work can be linked without 150 | the Library, or if the work is itself a library. The threshold for 151 | this to be true is not precisely defined by law. If such an object 152 | file uses only numerical parameters, data structure layouts and 153 | accessors, and small macros and small inline functions (ten lines or 154 | less in length), then the use of the object file is unrestricted, 155 | regardless of whether it is legally a derivative work. (Executables 156 | containing this object code plus portions of the Library will still 157 | fall under section 6.) Otherwise, if the work is a derivative of the 158 | Library, you may distribute the object code for the work under the 159 | terms of Section 6. Any executables containing that work also fall 160 | under Section 6, whether or not they are linked directly with the 161 | Library itself. 6. As an exception to the Sections above, you may 162 | also combine or link a "work that uses the Library" with the Library 163 | to produce a work containing portions of the Library, and distribute 164 | that work under terms of your choice, provided that the terms permit 165 | modification of the work for the customer's own use and reverse 166 | engineering for debugging such modifications. You must give prominent 167 | notice with each copy of the work that the Library is used in it and 168 | that the Library and its use are covered by this License. You must 169 | supply a copy of this License. If the work during execution displays 170 | copyright notices, you must include the copyright notice for the 171 | Library among them, as well as a reference directing the user to the 172 | copy of this License. Also, you must do one of these things: a) 173 | Accompany the work with the complete corresponding machine-readable 174 | source code for the Library including whatever changes were used in 175 | the work (which must be distributed under Sections 1 and 2 above); 176 | and, if the work is an executable liked with the Library, with the 177 | complete machine-readable "work that uses the Library", as object code 178 | and/or source code, so that the user can modify the Library and then 179 | relink to produce a modified executable containing the modified 180 | Library. (It is understood that the user who changes the contents of 181 | definitions files in the Library will not necessarily be able to 182 | recompile the application to use the modified definitions.) b) Use a 183 | suitable shared library mechanism for linking with the Library. A 184 | suitable mechanism is one that (1) uses at run time a copy of the 185 | library already present on the user's computer system, rather than 186 | copying library functions into the executable, and (2) will operate 187 | properly with a modified version of the library, if the user installs 188 | one, as long as the modified version is interface-compatible with the 189 | version that the work was made with. c) Accompany the work with a 190 | written offer, valid for at least three years, to give the same user 191 | the materials specified in Subsection 6a, above, for a charge no more 192 | than the cost of performing this distribution. d) If distribution of 193 | the work is made by offering access to copy from a designated place, 194 | offer equivalent access to copy the above specified materials from the 195 | same place. e) Verify that the user has already received a copy of 196 | these materials or that you have already sent this user a copy. For 197 | an executable, the required form of the "work that uses the Library" 198 | must include any data and utility programs needed for reproducing the 199 | executable from it. However, as a special exception, the materials to 200 | be distributed need not include anything that is normally distributed 201 | (in either source or binary form) with the major components (compiler, 202 | kernel, and so on) of the operating system on which the executable 203 | runs, unless that component itself accompanies the executable. It may 204 | happen that this requirement contradicts the license restrictions of 205 | other propriety libraries that do not normally accompany the operating 206 | system. Such a contradiction means you cannot use both them and the 207 | Library together in an executable that you distribute. 7. You may 208 | place library facilities that are a work based on the Library 209 | side-by-side in a single library together with other library 210 | facilities not covered by this License, and distribute such a combined 211 | library, provided that the separate distribution of the work based on 212 | the Library and of the other library facilities is otherwise 213 | permitted, and provided that you do these two things: a) Accompany the 214 | combined library with a copy of the same work based on the Library, 215 | uncombined with any other library facilities. This must be distributed 216 | under the terms of the Sections above. b) Give prominent notice with 217 | the combined library of the fact that part of it is a work based on 218 | the Library, and explaining where to find the accompanying uncombined 219 | form of the same work. 1 You may not copy, modify, sublicense, link 220 | with, or distribute the Library except as expressly provided under 221 | this License. Any attempt otherwise to copy, modify, sublicense, link 222 | with, or distribute the Library is void, and will automatically 223 | terminate your rights under this License. However, parties who have 224 | received copies, or rights, from you under this License will not have 225 | their licenses terminated so long as such parties remain in full 226 | compliance. 2 You are not required to accept this License, since you 227 | have not signed it. However, nothing else grants you permission to 228 | modify or distribute the Library or its derivative works. These 229 | actions are prohibited by law if you do not accept this 230 | License. Therefore, by modifying or distributing the Library (or any 231 | work based on the Library), you indicate your acceptance of this 232 | License to do so, and all its terms and conditions for copying, 233 | distributing or modifying the Library or works based on it. 3 Each 234 | time you redistribute the Library (or any work based on the Library), 235 | the recipient automatically receives a license from the original 236 | licensor to copy, distribute, link with or modify the Library subject 237 | to these terms and conditions. You may not impose any further 238 | restrictions on the recipients' exercise of the rights granted herein. 239 | You are not responsible for enforcing compliance by third parties with 240 | this License. 4 If, as a consequence of a court judgment or 241 | allegation of patent infringement or for any other reason (not limited 242 | to patent issues), conditions are imposed on you (whether by court 243 | order, agreement or otherwise) that contradict the conditions of this 244 | License, they do not excuse you from the conditions of this License. 245 | If you cannot distribute so as to satisfy simultaneously your 246 | obligations under this License and any other pertinent obligations, 247 | then as a consequence you may not distribute the Library at all. For 248 | example, if a patent license would not permit royalty-free 249 | redistribution of the Library by all those who receive copies directly 250 | or indirectly through you, then the only way you could satisfy both it 251 | and this License would be to refrain entirely from distribution of the 252 | Library. 253 | 254 | If any portion of this section is held invalid or unenforceable under 255 | any particular circumstance, the balance of the section is intended to 256 | apply, and the section as a whole is intended to apply in other 257 | circumstances. It is not the purpose of this section to induce you to 258 | infringe any patents or other property right claims or to contest 259 | validity of any such claims; this section has the sole purpose of 260 | protecting the integrity of the free software distribution system 261 | which is implemented by public license practices. Many people have 262 | made generous contributions to the wide range of software distributed 263 | through that system in reliance on consistent application of that 264 | system; it is up to the author/donor to decide if he or she is willing 265 | to distribute software through any other system and a licensee cannot 266 | impose that choice. This section is intended to make thoroughly clear 267 | what is believed to be a consequence of the rest of this License. 1 268 | If the distribution and/or use of the Library is restricted in certain 269 | countries either by patents or by copyrighted interfaces, the original 270 | copyright holder who places the Library under this License may add an 271 | explicit geographical distribution limitation excluding those 272 | countries, so that distribution is permitted only in or among 273 | countries not thus excluded. In such case, this License incorporates 274 | the limitation as if written in the body of this License. 0 13. The 275 | Free Software Foundation may publish revised and/or new versions of 276 | the Lesser General Public License from time to time. Such new 277 | versions will be similar in spirit to the present version, but may 278 | differ in detail to address new problems or concerns. 1 Each version 279 | is given a distinguishing version number. If the Library specifies a 280 | version number of this License which applies to it and "any later 281 | version", you have the option of following the terms and conditions 282 | either of that version or of any later version published by the Free 283 | Software Foundation. If the Library does not specify a license version 284 | number, you may choose any version ever published by the Free Software 285 | Foundation. 2 If you wish to incorporate parts of the Library into 286 | other free programs whose distribution conditions are incompatible 287 | with these, 288 | 289 | write to the author to ask for permission. For software which is 290 | copyrighted by the Free Software Foundation, write to the Free 291 | Software Foundation; we sometimes make exceptions for this. Our 292 | decision will be guided by the two goals of preserving the free status 293 | of all derivatives of our free software and of promoting the sharing 294 | and reuse of software generally. NO WARRANTY 1 BECAUSE THE LIBRARY IS 295 | LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE LIBRARY, TO THE 296 | EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN 297 | WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE LIBRARY 298 | "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED 299 | INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 300 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK 301 | AS TO THE QUALITY AND PERFORMANCE OF THE LIBRARY IS WITH YOU. SHOULD 302 | THE LIBRARY PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY 303 | SERVICING, REPAIR OR CORRECTION. 2 IN NO EVENT UNLESS REQUIRED BY 304 | APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR 305 | ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE LIBRARY AS 306 | PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, 307 | SPECIAL, INCIDENTAL OR CONSQUENTIAL DAMAGES ARISING OUT OF THE USE OR 308 | INABILITY TO USE THE LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF 309 | DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR 310 | THIRD PARTIES OR A FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER 311 | SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 312 | POSSIBILITY OF SUCH DAMAGES. 313 | 314 | -------------------------------------------------------------------------------- /LLNL-CODE-747688: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/LLNL/selfsupervised/f5e05b62dc6cd9da07783a7578955f141c82bda4/LLNL-CODE-747688 -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Lawrence Livermore National Lab :sunrise_over_mountains: Self Supervised Learning Repo 2 | This is source related to [our self-supervised paper](https://arxiv.org/abs/1711.06379) appearing in CVPR 2018. For more information take a look at the **Github Wiki page**. Just click on the [Wiki tab](https://github.com/LLNL/selfsupervised/wiki). 3 | 4 | ### What you Need 5 | These scripts are a wrapper around Caffe. So, you need Caffe. Once you've installed it, you will probably have everything you need to run this source. These include Caffe itself, Python 2.x :snake: , OpenCV, Numpy and Pyplot. 6 | 7 | ### Get Thee Some Data 8 | The source code will do the self-supervised training. The patches have been preprocessed for you. You can take a peek in https://gdo-datasci.llnl.gov/selfsupervised/download/image_sets/patches/ . There are patches with and without chroma blurring. The training and testing lists are in Numpy/Pickle format to save space. The training and validation images should be extracted into a directory structure that mirrors the standard ImageNet ILSVRC2012 directory structure. If your image dir is *patches* each of the training directories nXXXXXXXX should be in *patches/train*. All validation images should just be shoved into one dir *patches/val* . 9 | 10 | ### How to Run 11 | :one: Edit **python/RunTripleContextTrain.py** and set paths in *project_home*, *path_prefix* and *gpus*. These are the only ones you must edit. Set *gpus* based on the number you want to run. In a single GPU environment, just set it to [1]. You can choose different network models that are in the *caffe_data* directory. Each one is named train_val_*network*_triple.caffenet. For comparison with our results in *tables 2,3 and 4* , using the CaffeNet model is suitable to test on VOC and Linear tests. You should run the AlexNet-Custom network to compare with our CUB and CompCars results from *table 1*. You can use ResCeption and Inception networks to compare with *table 5*. 12 | 13 | :two: Run **python/RunTripleContextTrain.py**. The first time you run it, it will create a copy in a project directory. If you are in a cluster environment, you can run the Slurm script instead. 14 | 15 | :three: Run the script just created in **py_proj**. If the paths are correct, things should run and start to train. Take as look at the Wiki. This has cool pictures of how it should look if it's training correctly. 16 | 17 | :four: The figures it creates should also be saved in the **figures** directory. 18 | 19 | :five: The trained caffe models will be saved in py_proj/*my_project*/caffe_model 20 | 21 | ### How to Test 22 | #### CUB and CompCars 23 | We have provided the Cub birds :hatched_chick: and Comp Cars :car: data processed as we used it. These are in: https://gdo-datasci.llnl.gov/selfsupervised/download/image_sets/. The network models are in *caffe_data* named train_val_*network*_single.caffenet. You don't need python to run these. You can just use the direct Caffe command line. 24 | 25 | #### VOC Tests 26 | You should be able to plug the trained model into *Detection* and *Classification* tests without too much difficulty. 27 | 28 | :one: Try classification with an initial learning rate of 0.000025 29 | 30 | :two: Try detection with an initial learning rate of 0.0005 31 | 32 | :three: Segmentation requires a few tweaks. We cannot transfer fc6 and fc7 via surgery. So we have to init them when we run segmentation. By default, the *fcn.berkeleyvision.org* would not init these layers. You can download a model that does this from https://gdo-datasci.llnl.gov/selfsupervised/download/models/caffenet/. Notice this also has a bunch of pretrained models you can try out. You still need to run surgery even though you are not transferring fc6 and fc7. 33 | 34 | #### Linear Tests 35 | This runs pretty straightforward. In https://gdo-datasci.llnl.gov/selfsupervised/download/models/caffenet/ is variant linear training network that uses CaffeNet. It also chops off calls to python graphing layers so it can be run in multi GPU mode. Other than that, it's the same as the default model. 36 | 37 | ### Design Concepts 38 | The source has several design choices which determine why certain things were done this way or that. 39 | 40 | :one: The Python source wraps around Caffe rather than using python layers. This is because Python layers in Caffe do not support multiple GPU execution. This allows us preprocess data and even use a multi process prefetcher to speed up image loading. 41 | 42 | :two: We are running in a cluster environment. This is why it creates a project copy when you run it. This also keeps experiments in their own tidy folder. You can look back to check how something actually ran. This is also why we load raw images rather than a lmdb. 43 | 44 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | theme: jekyll-theme-architect -------------------------------------------------------------------------------- /caffe_data/README.md: -------------------------------------------------------------------------------- 1 | Single and Triple caffe models. -------------------------------------------------------------------------------- /caffe_data/train_val_AlexNet-Custom_single.prototxt: -------------------------------------------------------------------------------- 1 | name: "AlexNet-Custom" 2 | 3 | # train data 4 | layer { 5 | top: "data" 6 | top: "label" 7 | name: "data" 8 | type: "ImageData" 9 | image_data_param { 10 | source: "train_list.txt" 11 | batch_size: 64 12 | shuffle: true 13 | } 14 | transform_param { 15 | mirror: true 16 | crop_size: 224 17 | mean_value: 104 18 | mean_value: 117 19 | mean_value: 123 20 | } 21 | include { 22 | phase: TRAIN 23 | } 24 | } 25 | 26 | # test data 27 | layer { 28 | top: "data" 29 | top: "label" 30 | name: "data" 31 | type: "ImageData" 32 | image_data_param { 33 | source: "test_list.txt" 34 | batch_size: 10 35 | } 36 | transform_param { 37 | mirror: false 38 | crop_size: 224 39 | mean_value: 104 40 | mean_value: 117 41 | mean_value: 123 42 | } 43 | include { 44 | phase: TEST 45 | } 46 | } 47 | 48 | # ================================================================================================ 49 | # ======== Layer 1 50 | # ================================================================================================ 51 | 52 | 53 | 54 | layer { 55 | name: "conv1" 56 | type: "Convolution" 57 | bottom: "data" 58 | top: "conv1" 59 | param { 60 | lr_mult: 1 61 | decay_mult: 1 62 | name: "conv1_w" 63 | } 64 | param { 65 | lr_mult: 2 66 | decay_mult: 0 67 | name: "conv1_b" 68 | } 69 | convolution_param { 70 | num_output: 96 71 | kernel_size: 11 72 | stride: 4 73 | weight_filler { 74 | type: "gaussian" 75 | std: 0.01 76 | } 77 | bias_filler { 78 | type: "constant" 79 | value: 0 80 | } 81 | } 82 | } 83 | 84 | layer { 85 | name: "bn1" 86 | type: "BatchNorm" 87 | bottom: "conv1" 88 | top: "bn1" 89 | param { 90 | lr_mult: 0 91 | } 92 | param { 93 | lr_mult: 0 94 | } 95 | param { 96 | lr_mult: 0 97 | } 98 | } 99 | 100 | layer { 101 | name: "relu1" 102 | type: "ReLU" 103 | bottom: "bn1" 104 | top: "relu1" 105 | } 106 | 107 | layer { 108 | name: "pool1" 109 | type: "Pooling" 110 | bottom: "relu1" 111 | top: "pool1" 112 | pooling_param { 113 | pool: MAX 114 | kernel_size: 3 115 | stride: 2 116 | } 117 | } 118 | layer { 119 | name: "conv2" 120 | type: "Convolution" 121 | bottom: "pool1" 122 | top: "conv2" 123 | param { 124 | name: "conv2_w" 125 | lr_mult: 1 126 | decay_mult: 1 127 | } 128 | param { 129 | name: "conv2_b" 130 | lr_mult: 2 131 | decay_mult: 0 132 | } 133 | convolution_param { 134 | num_output: 256 135 | pad: 2 136 | kernel_size: 5 137 | group: 2 138 | weight_filler { 139 | type: "gaussian" 140 | std: 0.01 141 | } 142 | bias_filler { 143 | type: "constant" 144 | value: 0.1 145 | } 146 | } 147 | } 148 | layer { 149 | name: "bn2" 150 | type: "BatchNorm" 151 | bottom: "conv2" 152 | top: "bn2" 153 | param { 154 | lr_mult: 0 155 | } 156 | param { 157 | lr_mult: 0 158 | } 159 | param { 160 | lr_mult: 0 161 | } 162 | } 163 | 164 | layer { 165 | name: "relu2" 166 | type: "ReLU" 167 | bottom: "bn2" 168 | top: "relu2" 169 | } 170 | 171 | layer { 172 | name: "pool2" 173 | type: "Pooling" 174 | bottom: "relu2" 175 | top: "pool2" 176 | pooling_param { 177 | pool: MAX 178 | kernel_size: 3 179 | stride: 2 180 | } 181 | } 182 | layer { 183 | name: "conv3" 184 | type: "Convolution" 185 | bottom: "pool2" 186 | top: "conv3" 187 | param { 188 | name: "conv3_w" 189 | lr_mult: 1 190 | decay_mult: 1 191 | } 192 | param { 193 | name: "conv3_b" 194 | lr_mult: 2 195 | decay_mult: 0 196 | } 197 | convolution_param { 198 | num_output: 384 199 | pad: 1 200 | kernel_size: 3 201 | weight_filler { 202 | type: "gaussian" 203 | std: 0.01 204 | } 205 | bias_filler { 206 | type: "constant" 207 | value: 0 208 | } 209 | } 210 | } 211 | layer { 212 | name: "bn3" 213 | type: "BatchNorm" 214 | bottom: "conv3" 215 | top: "bn3" 216 | param { 217 | lr_mult: 0 218 | } 219 | param { 220 | lr_mult: 0 221 | } 222 | param { 223 | lr_mult: 0 224 | } 225 | } 226 | 227 | layer { 228 | name: "relu3" 229 | type: "ReLU" 230 | bottom: "bn3" 231 | top: "relu3" 232 | } 233 | layer { 234 | name: "conv4" 235 | type: "Convolution" 236 | bottom: "relu3" 237 | top: "conv4" 238 | param { 239 | name: "conv4_w" 240 | lr_mult: 1 241 | decay_mult: 1 242 | } 243 | param { 244 | name: "conv4_b" 245 | lr_mult: 2 246 | decay_mult: 0 247 | } 248 | convolution_param { 249 | num_output: 384 250 | pad: 1 251 | kernel_size: 3 252 | group: 2 253 | weight_filler { 254 | type: "gaussian" 255 | std: 0.01 256 | } 257 | bias_filler { 258 | type: "constant" 259 | value: 0.1 260 | } 261 | } 262 | } 263 | layer { 264 | name: "bn4" 265 | type: "BatchNorm" 266 | bottom: "conv4" 267 | top: "bn4" 268 | param { 269 | lr_mult: 0 270 | } 271 | param { 272 | lr_mult: 0 273 | } 274 | param { 275 | lr_mult: 0 276 | } 277 | } 278 | 279 | layer { 280 | name: "relu4" 281 | type: "ReLU" 282 | bottom: "bn4" 283 | top: "relu4" 284 | } 285 | layer { 286 | name: "conv5" 287 | type: "Convolution" 288 | bottom: "relu4" 289 | top: "conv5" 290 | param { 291 | name: "conv5_w" 292 | lr_mult: 1 293 | decay_mult: 1 294 | } 295 | param { 296 | name: "conv5_b" 297 | lr_mult: 2 298 | decay_mult: 0 299 | } 300 | convolution_param { 301 | num_output: 256 302 | pad: 1 303 | kernel_size: 3 304 | group: 2 305 | weight_filler { 306 | type: "gaussian" 307 | std: 0.01 308 | } 309 | bias_filler { 310 | type: "constant" 311 | value: 0.1 312 | } 313 | } 314 | } 315 | layer { 316 | name: "bn5" 317 | type: "BatchNorm" 318 | bottom: "conv5" 319 | top: "bn5" 320 | param { 321 | lr_mult: 0 322 | } 323 | param { 324 | lr_mult: 0 325 | } 326 | param { 327 | lr_mult: 0 328 | } 329 | } 330 | 331 | layer { 332 | name: "relu5" 333 | type: "ReLU" 334 | bottom: "bn5" 335 | top: "relu5" 336 | } 337 | layer { 338 | name: "pool5" 339 | type: "Pooling" 340 | bottom: "relu5" 341 | top: "pool5" 342 | pooling_param { 343 | pool: MAX 344 | kernel_size: 3 345 | stride: 2 346 | } 347 | } 348 | 349 | 350 | # ================================================================================================ 351 | # ======== Doersch Layer 6 352 | # ================================================================================================ 353 | 354 | layer { 355 | name: "conv6" 356 | type: "Convolution" 357 | bottom: "pool5" 358 | top: "conv6" 359 | param { 360 | name: "conv6_w" 361 | lr_mult: 1 362 | decay_mult: 1 363 | } 364 | param { 365 | name: "conv6_b" 366 | lr_mult: 2 367 | decay_mult: 0 368 | } 369 | convolution_param { 370 | num_output: 4096 371 | pad: 1 372 | kernel_size: 3 373 | weight_filler { 374 | type: "gaussian" 375 | std: 0.01 376 | } 377 | bias_filler { 378 | type: "constant" 379 | value: 0.1 380 | } 381 | } 382 | } 383 | layer { 384 | name: "bn6a" 385 | type: "BatchNorm" 386 | bottom: "conv6" 387 | top: "bn6a" 388 | param { 389 | lr_mult: 0 390 | } 391 | param { 392 | lr_mult: 0 393 | } 394 | param { 395 | lr_mult: 0 396 | } 397 | } 398 | 399 | layer { 400 | name: "relu6a" 401 | type: "ReLU" 402 | bottom: "bn6a" 403 | top: "relu6a" 404 | } 405 | 406 | layer { 407 | name: "conv6a" 408 | type: "Convolution" 409 | bottom: "relu6a" 410 | top: "conv6a" 411 | param { 412 | name: "conv6a_w" 413 | lr_mult: 1 414 | decay_mult: 1 415 | } 416 | param { 417 | name: "conv6a_b" 418 | lr_mult: 2 419 | decay_mult: 0 420 | } 421 | convolution_param { 422 | num_output: 1024 423 | pad: 1 424 | kernel_size: 1 425 | weight_filler { 426 | type: "gaussian" 427 | std: 0.01 428 | } 429 | bias_filler { 430 | type: "constant" 431 | value: 0.1 432 | } 433 | } 434 | } 435 | layer { 436 | name: "bn6a-ss" 437 | type: "BatchNorm" 438 | bottom: "conv6a" 439 | top: "bn6a-ss" 440 | param { 441 | lr_mult: 0 442 | } 443 | param { 444 | lr_mult: 0 445 | } 446 | param { 447 | lr_mult: 0 448 | } 449 | } 450 | 451 | layer { 452 | name: "relu6a-ss" 453 | type: "ReLU" 454 | bottom: "bn6a-ss" 455 | top: "relu6a-ss" 456 | } 457 | 458 | layer { 459 | name: "pool6" 460 | type: "Pooling" 461 | bottom: "relu6a-ss" 462 | top: "pool6" 463 | pooling_param { 464 | pool: MAX 465 | kernel_size: 3 466 | stride: 2 467 | } 468 | } 469 | 470 | # ================================================================================================ 471 | # ======== END Doersch Layer 6 472 | # ================================================================================================ 473 | 474 | 475 | layer { 476 | name: "fc7-cc" 477 | type: "InnerProduct" 478 | bottom: "pool6" 479 | top: "fc7-cc" 480 | param { 481 | lr_mult: 1 482 | decay_mult: 1 483 | } 484 | param { 485 | lr_mult: 2 486 | decay_mult: 0 487 | } 488 | inner_product_param { 489 | num_output: 4096 490 | weight_filler { 491 | type: "gaussian" 492 | std: 0.005 493 | } 494 | bias_filler { 495 | type: "constant" 496 | value: 0.1 497 | } 498 | } 499 | } 500 | 501 | layer { 502 | name: "bn7-cc" 503 | type: "BatchNorm" 504 | bottom: "fc7-cc" 505 | top: "bn7-cc" 506 | param { 507 | lr_mult: 0 508 | } 509 | param { 510 | lr_mult: 0 511 | } 512 | param { 513 | lr_mult: 0 514 | } 515 | } 516 | 517 | layer { 518 | name: "relu7-cc" 519 | type: "ReLU" 520 | bottom: "bn7-cc" 521 | top: "relu7-cc" 522 | } 523 | 524 | layer { 525 | name: "fc8-cc" 526 | type: "InnerProduct" 527 | bottom: "relu7-cc" 528 | top: "fc8-cc" 529 | param { 530 | lr_mult: 1 531 | decay_mult: 1 532 | } 533 | param { 534 | lr_mult: 2 535 | decay_mult: 0 536 | } 537 | inner_product_param { 538 | ###NUM_OUTPUT### 539 | weight_filler { 540 | type: "gaussian" 541 | std: 0.01 542 | } 543 | bias_filler { 544 | type: "constant" 545 | value: 0 546 | } 547 | } 548 | } 549 | layer { 550 | name: "accuracy" 551 | type: "Accuracy" 552 | bottom: "fc8-cc" 553 | bottom: "label" 554 | top: "accuracy" 555 | include { 556 | phase: TEST 557 | } 558 | } 559 | layer { 560 | name: "accuracy_top5" 561 | type: "Accuracy" 562 | bottom: "fc8-cc" 563 | bottom: "label" 564 | top: "accuracy_top5" 565 | include { 566 | phase: TEST 567 | } 568 | accuracy_param { 569 | top_k: 5 570 | } 571 | } 572 | layer { 573 | name: "loss" 574 | type: "SoftmaxWithLoss" 575 | bottom: "fc8-cc" 576 | bottom: "label" 577 | top: "loss" 578 | } 579 | -------------------------------------------------------------------------------- /caffe_data/train_val_AlexNet-Custom_triple.prototxt: -------------------------------------------------------------------------------- 1 | name: "AlexNet-Custom" 2 | input: "data_1" 3 | input_dim: 5 4 | input_dim: 3 5 | input_dim: 96 6 | input_dim: 96 7 | input: "data_2" 8 | input_dim: 5 9 | input_dim: 3 10 | input_dim: 96 11 | input_dim: 96 12 | input: "data_3" 13 | input_dim: 5 14 | input_dim: 3 15 | input_dim: 96 16 | input_dim: 96 17 | input: "label" 18 | input_dim: 5 19 | input_dim: 1 20 | input_dim: 1 21 | input_dim: 1 22 | 23 | # ================================================================================================ 24 | # ================================================================================================ 25 | # ======== NETWORK 1 26 | # ================================================================================================ 27 | # ================================================================================================ 28 | 29 | layer { 30 | name: "conv1" 31 | type: "Convolution" 32 | bottom: "data_1" 33 | top: "conv1" 34 | param { 35 | lr_mult: 0.7 36 | decay_mult: 1 37 | name: "conv1_w" 38 | } 39 | param { 40 | lr_mult: 2 41 | decay_mult: 0 42 | name: "conv1_b" 43 | } 44 | convolution_param { 45 | num_output: 96 46 | kernel_size: 11 47 | stride: 4 48 | pad: 5 49 | weight_filler { 50 | type: "gaussian" 51 | std: 0.01 52 | } 53 | bias_filler { 54 | type: "constant" 55 | value: 0 56 | } 57 | } 58 | } 59 | 60 | layer { 61 | name: "bn1" 62 | type: "BatchNorm" 63 | bottom: "conv1" 64 | top: "bn1" 65 | param { 66 | lr_mult: 0 67 | } 68 | param { 69 | lr_mult: 0 70 | } 71 | param { 72 | lr_mult: 0 73 | } 74 | } 75 | 76 | layer { 77 | name: "relu1" 78 | type: "ReLU" 79 | bottom: "bn1" 80 | top: "relu1" 81 | } 82 | 83 | layer { 84 | name: "pool1" 85 | type: "Pooling" 86 | bottom: "relu1" 87 | top: "pool1" 88 | pooling_param { 89 | pool: MAX 90 | kernel_size: 3 91 | stride: 2 92 | } 93 | } 94 | layer { 95 | name: "conv2" 96 | type: "Convolution" 97 | bottom: "pool1" 98 | top: "conv2" 99 | param { 100 | name: "conv2_w" 101 | lr_mult: 0.8 102 | decay_mult: 1 103 | } 104 | param { 105 | name: "conv2_b" 106 | lr_mult: 2 107 | decay_mult: 0 108 | } 109 | convolution_param { 110 | num_output: 256 111 | pad: 2 112 | kernel_size: 5 113 | group: 2 114 | weight_filler { 115 | type: "gaussian" 116 | std: 0.01 117 | } 118 | bias_filler { 119 | type: "constant" 120 | value: 0.1 121 | } 122 | } 123 | } 124 | layer { 125 | name: "bn2" 126 | type: "BatchNorm" 127 | bottom: "conv2" 128 | top: "bn2" 129 | param { 130 | lr_mult: 0 131 | } 132 | param { 133 | lr_mult: 0 134 | } 135 | param { 136 | lr_mult: 0 137 | } 138 | } 139 | 140 | layer { 141 | name: "relu2" 142 | type: "ReLU" 143 | bottom: "bn2" 144 | top: "relu2" 145 | } 146 | 147 | layer { 148 | name: "pool2" 149 | type: "Pooling" 150 | bottom: "relu2" 151 | top: "pool2" 152 | pooling_param { 153 | pool: MAX 154 | kernel_size: 3 155 | stride: 2 156 | } 157 | } 158 | layer { 159 | name: "conv3" 160 | type: "Convolution" 161 | bottom: "pool2" 162 | top: "conv3" 163 | param { 164 | name: "conv3_w" 165 | lr_mult: 0.9 166 | decay_mult: 1 167 | } 168 | param { 169 | name: "conv3_b" 170 | lr_mult: 2 171 | decay_mult: 0 172 | } 173 | convolution_param { 174 | num_output: 384 175 | pad: 1 176 | kernel_size: 3 177 | weight_filler { 178 | type: "gaussian" 179 | std: 0.01 180 | } 181 | bias_filler { 182 | type: "constant" 183 | value: 0 184 | } 185 | } 186 | } 187 | layer { 188 | name: "bn3" 189 | type: "BatchNorm" 190 | bottom: "conv3" 191 | top: "bn3" 192 | param { 193 | lr_mult: 0 194 | } 195 | param { 196 | lr_mult: 0 197 | } 198 | param { 199 | lr_mult: 0 200 | } 201 | } 202 | 203 | layer { 204 | name: "relu3" 205 | type: "ReLU" 206 | bottom: "bn3" 207 | top: "relu3" 208 | } 209 | layer { 210 | name: "conv4" 211 | type: "Convolution" 212 | bottom: "relu3" 213 | top: "conv4" 214 | param { 215 | name: "conv4_w" 216 | lr_mult: 1 217 | decay_mult: 1 218 | } 219 | param { 220 | name: "conv4_b" 221 | lr_mult: 2 222 | decay_mult: 0 223 | } 224 | convolution_param { 225 | num_output: 384 226 | pad: 1 227 | kernel_size: 3 228 | group: 2 229 | weight_filler { 230 | type: "gaussian" 231 | std: 0.01 232 | } 233 | bias_filler { 234 | type: "constant" 235 | value: 0.1 236 | } 237 | } 238 | } 239 | layer { 240 | name: "bn4" 241 | type: "BatchNorm" 242 | bottom: "conv4" 243 | top: "bn4" 244 | param { 245 | lr_mult: 0 246 | } 247 | param { 248 | lr_mult: 0 249 | } 250 | param { 251 | lr_mult: 0 252 | } 253 | } 254 | 255 | layer { 256 | name: "relu4" 257 | type: "ReLU" 258 | bottom: "bn4" 259 | top: "relu4" 260 | } 261 | layer { 262 | name: "conv5" 263 | type: "Convolution" 264 | bottom: "relu4" 265 | top: "conv5" 266 | param { 267 | name: "conv5_w" 268 | lr_mult: 0.9 269 | decay_mult: 1 270 | } 271 | param { 272 | name: "conv5_b" 273 | lr_mult: 2 274 | decay_mult: 0 275 | } 276 | convolution_param { 277 | num_output: 256 278 | pad: 1 279 | kernel_size: 3 280 | group: 2 281 | weight_filler { 282 | type: "gaussian" 283 | std: 0.01 284 | } 285 | bias_filler { 286 | type: "constant" 287 | value: 0.1 288 | } 289 | } 290 | } 291 | layer { 292 | name: "bn5" 293 | type: "BatchNorm" 294 | bottom: "conv5" 295 | top: "bn5" 296 | param { 297 | lr_mult: 0 298 | } 299 | param { 300 | lr_mult: 0 301 | } 302 | param { 303 | lr_mult: 0 304 | } 305 | } 306 | 307 | layer { 308 | name: "relu5" 309 | type: "ReLU" 310 | bottom: "bn5" 311 | top: "relu5" 312 | } 313 | layer { 314 | name: "pool5" 315 | type: "Pooling" 316 | bottom: "relu5" 317 | top: "pool5" 318 | pooling_param { 319 | pool: MAX 320 | kernel_size: 3 321 | stride: 2 322 | } 323 | } 324 | layer { 325 | name: "fc6" 326 | type: "InnerProduct" 327 | bottom: "pool5" 328 | top: "fc6" 329 | param { 330 | name: "fc6_w" 331 | lr_mult: 0.8 332 | decay_mult: 1 333 | } 334 | param { 335 | name: "fc6_b" 336 | lr_mult: 2 337 | decay_mult: 0 338 | } 339 | inner_product_param { 340 | num_output: 4096 341 | weight_filler { 342 | type: "gaussian" 343 | std: 0.005 344 | } 345 | bias_filler { 346 | type: "constant" 347 | value: 0.1 348 | } 349 | } 350 | } 351 | layer { 352 | name: "bn6" 353 | type: "BatchNorm" 354 | bottom: "fc6" 355 | top: "bn6" 356 | param { 357 | lr_mult: 0 358 | } 359 | param { 360 | lr_mult: 0 361 | } 362 | param { 363 | lr_mult: 0 364 | } 365 | } 366 | 367 | layer { 368 | name: "relu6" 369 | type: "ReLU" 370 | bottom: "bn6" 371 | top: "relu6" 372 | } 373 | 374 | # ================================================================================================ 375 | # ================================================================================================ 376 | # ======== NETWORK 2 377 | # ================================================================================================ 378 | # ================================================================================================ 379 | 380 | layer { 381 | name: "conv1_s" 382 | type: "Convolution" 383 | bottom: "data_2" 384 | top: "conv1_s" 385 | param { 386 | lr_mult: 0.7 387 | decay_mult: 1 388 | name: "conv1_w" 389 | } 390 | param { 391 | lr_mult: 2 392 | decay_mult: 0 393 | name: "conv1_b" 394 | } 395 | convolution_param { 396 | num_output: 96 397 | kernel_size: 11 398 | stride: 4 399 | pad: 5 400 | weight_filler { 401 | type: "gaussian" 402 | std: 0.01 403 | } 404 | bias_filler { 405 | type: "constant" 406 | value: 0 407 | } 408 | } 409 | } 410 | 411 | layer { 412 | name: "bn1_s" 413 | type: "BatchNorm" 414 | bottom: "conv1_s" 415 | top: "bn1_s" 416 | param { 417 | lr_mult: 0 418 | } 419 | param { 420 | lr_mult: 0 421 | } 422 | param { 423 | lr_mult: 0 424 | } 425 | } 426 | 427 | layer { 428 | name: "relu1_s" 429 | type: "ReLU" 430 | bottom: "bn1_s" 431 | top: "relu1_s" 432 | } 433 | 434 | layer { 435 | name: "pool1_s" 436 | type: "Pooling" 437 | bottom: "relu1_s" 438 | top: "pool1_s" 439 | pooling_param { 440 | pool: MAX 441 | kernel_size: 3 442 | stride: 2 443 | } 444 | } 445 | layer { 446 | name: "conv2_s" 447 | type: "Convolution" 448 | bottom: "pool1_s" 449 | top: "conv2_s" 450 | param { 451 | name: "conv2_w" 452 | lr_mult: 0.8 453 | decay_mult: 1 454 | } 455 | param { 456 | name: "conv2_b" 457 | lr_mult: 2 458 | decay_mult: 0 459 | } 460 | convolution_param { 461 | num_output: 256 462 | pad: 2 463 | kernel_size: 5 464 | group: 2 465 | weight_filler { 466 | type: "gaussian" 467 | std: 0.01 468 | } 469 | bias_filler { 470 | type: "constant" 471 | value: 0.1 472 | } 473 | } 474 | } 475 | layer { 476 | name: "bn2_s" 477 | type: "BatchNorm" 478 | bottom: "conv2_s" 479 | top: "bn2_s" 480 | param { 481 | lr_mult: 0 482 | } 483 | param { 484 | lr_mult: 0 485 | } 486 | param { 487 | lr_mult: 0 488 | } 489 | } 490 | 491 | layer { 492 | name: "relu2_s" 493 | type: "ReLU" 494 | bottom: "bn2_s" 495 | top: "relu2_s" 496 | } 497 | 498 | layer { 499 | name: "pool2_s" 500 | type: "Pooling" 501 | bottom: "relu2_s" 502 | top: "pool2_s" 503 | pooling_param { 504 | pool: MAX 505 | kernel_size: 3 506 | stride: 2 507 | } 508 | } 509 | layer { 510 | name: "conv3_s" 511 | type: "Convolution" 512 | bottom: "pool2_s" 513 | top: "conv3_s" 514 | param { 515 | name: "conv3_w" 516 | lr_mult: 0.9 517 | decay_mult: 1 518 | } 519 | param { 520 | name: "conv3_b" 521 | lr_mult: 2 522 | decay_mult: 0 523 | } 524 | convolution_param { 525 | num_output: 384 526 | pad: 1 527 | kernel_size: 3 528 | weight_filler { 529 | type: "gaussian" 530 | std: 0.01 531 | } 532 | bias_filler { 533 | type: "constant" 534 | value: 0 535 | } 536 | } 537 | } 538 | layer { 539 | name: "bn3_s" 540 | type: "BatchNorm" 541 | bottom: "conv3_s" 542 | top: "bn3_s" 543 | param { 544 | lr_mult: 0 545 | } 546 | param { 547 | lr_mult: 0 548 | } 549 | param { 550 | lr_mult: 0 551 | } 552 | } 553 | 554 | layer { 555 | name: "relu3_s" 556 | type: "ReLU" 557 | bottom: "bn3_s" 558 | top: "relu3_s" 559 | } 560 | layer { 561 | name: "conv4_s" 562 | type: "Convolution" 563 | bottom: "relu3_s" 564 | top: "conv4_s" 565 | param { 566 | name: "conv4_w" 567 | lr_mult: 1 568 | decay_mult: 1 569 | } 570 | param { 571 | name: "conv4_b" 572 | lr_mult: 2 573 | decay_mult: 0 574 | } 575 | convolution_param { 576 | num_output: 384 577 | pad: 1 578 | kernel_size: 3 579 | group: 2 580 | weight_filler { 581 | type: "gaussian" 582 | std: 0.01 583 | } 584 | bias_filler { 585 | type: "constant" 586 | value: 0.1 587 | } 588 | } 589 | } 590 | layer { 591 | name: "bn4_s" 592 | type: "BatchNorm" 593 | bottom: "conv4_s" 594 | top: "bn4_s" 595 | param { 596 | lr_mult: 0 597 | } 598 | param { 599 | lr_mult: 0 600 | } 601 | param { 602 | lr_mult: 0 603 | } 604 | } 605 | 606 | layer { 607 | name: "relu4_s" 608 | type: "ReLU" 609 | bottom: "bn4_s" 610 | top: "relu4_s" 611 | } 612 | layer { 613 | name: "conv5_s" 614 | type: "Convolution" 615 | bottom: "relu4_s" 616 | top: "conv5_s" 617 | param { 618 | name: "conv5_w" 619 | lr_mult: 0.9 620 | decay_mult: 1 621 | } 622 | param { 623 | name: "conv5_b" 624 | lr_mult: 2 625 | decay_mult: 0 626 | } 627 | convolution_param { 628 | num_output: 256 629 | pad: 1 630 | kernel_size: 3 631 | group: 2 632 | weight_filler { 633 | type: "gaussian" 634 | std: 0.01 635 | } 636 | bias_filler { 637 | type: "constant" 638 | value: 0.1 639 | } 640 | } 641 | } 642 | layer { 643 | name: "bn5_s" 644 | type: "BatchNorm" 645 | bottom: "conv5_s" 646 | top: "bn5_s" 647 | param { 648 | lr_mult: 0 649 | } 650 | param { 651 | lr_mult: 0 652 | } 653 | param { 654 | lr_mult: 0 655 | } 656 | } 657 | 658 | layer { 659 | name: "relu5_s" 660 | type: "ReLU" 661 | bottom: "bn5_s" 662 | top: "relu5_s" 663 | } 664 | layer { 665 | name: "pool5_s" 666 | type: "Pooling" 667 | bottom: "relu5_s" 668 | top: "pool5_s" 669 | pooling_param { 670 | pool: MAX 671 | kernel_size: 3 672 | stride: 2 673 | } 674 | } 675 | layer { 676 | name: "fc6_s" 677 | type: "InnerProduct" 678 | bottom: "pool5_s" 679 | top: "fc6_s" 680 | param { 681 | name: "fc6_w" 682 | lr_mult: 0.8 683 | decay_mult: 1 684 | } 685 | param { 686 | name: "fc6_b" 687 | lr_mult: 2 688 | decay_mult: 0 689 | } 690 | inner_product_param { 691 | num_output: 4096 692 | weight_filler { 693 | type: "gaussian" 694 | std: 0.005 695 | } 696 | bias_filler { 697 | type: "constant" 698 | value: 0.1 699 | } 700 | } 701 | } 702 | layer { 703 | name: "bn6_s" 704 | type: "BatchNorm" 705 | bottom: "fc6_s" 706 | top: "bn6_s" 707 | param { 708 | lr_mult: 0 709 | } 710 | param { 711 | lr_mult: 0 712 | } 713 | param { 714 | lr_mult: 0 715 | } 716 | } 717 | 718 | layer { 719 | name: "relu6_s" 720 | type: "ReLU" 721 | bottom: "bn6_s" 722 | top: "relu6_s" 723 | } 724 | 725 | # ================================================================================================ 726 | # ================================================================================================ 727 | # ======== NETWORK 3 728 | # ================================================================================================ 729 | # ================================================================================================ 730 | 731 | layer { 732 | name: "conv1_t" 733 | type: "Convolution" 734 | bottom: "data_3" 735 | top: "conv1_t" 736 | param { 737 | lr_mult: 0.7 738 | decay_mult: 1 739 | name: "conv1_w" 740 | } 741 | param { 742 | lr_mult: 2 743 | decay_mult: 0 744 | name: "conv1_b" 745 | } 746 | convolution_param { 747 | num_output: 96 748 | kernel_size: 11 749 | stride: 4 750 | pad: 5 751 | weight_filler { 752 | type: "gaussian" 753 | std: 0.01 754 | } 755 | bias_filler { 756 | type: "constant" 757 | value: 0 758 | } 759 | } 760 | } 761 | 762 | layer { 763 | name: "bn1_t" 764 | type: "BatchNorm" 765 | bottom: "conv1_t" 766 | top: "bn1_t" 767 | param { 768 | lr_mult: 0 769 | } 770 | param { 771 | lr_mult: 0 772 | } 773 | param { 774 | lr_mult: 0 775 | } 776 | } 777 | 778 | layer { 779 | name: "relu1_t" 780 | type: "ReLU" 781 | bottom: "bn1_t" 782 | top: "relu1_t" 783 | } 784 | 785 | layer { 786 | name: "pool1_t" 787 | type: "Pooling" 788 | bottom: "relu1_t" 789 | top: "pool1_t" 790 | pooling_param { 791 | pool: MAX 792 | kernel_size: 3 793 | stride: 2 794 | } 795 | } 796 | layer { 797 | name: "conv2_t" 798 | type: "Convolution" 799 | bottom: "pool1_t" 800 | top: "conv2_t" 801 | param { 802 | name: "conv2_w" 803 | lr_mult: 0.8 804 | decay_mult: 1 805 | } 806 | param { 807 | name: "conv2_b" 808 | lr_mult: 2 809 | decay_mult: 0 810 | } 811 | convolution_param { 812 | num_output: 256 813 | pad: 2 814 | kernel_size: 5 815 | group: 2 816 | weight_filler { 817 | type: "gaussian" 818 | std: 0.01 819 | } 820 | bias_filler { 821 | type: "constant" 822 | value: 0.1 823 | } 824 | } 825 | } 826 | layer { 827 | name: "bn2_t" 828 | type: "BatchNorm" 829 | bottom: "conv2_t" 830 | top: "bn2_t" 831 | param { 832 | lr_mult: 0 833 | } 834 | param { 835 | lr_mult: 0 836 | } 837 | param { 838 | lr_mult: 0 839 | } 840 | } 841 | 842 | layer { 843 | name: "relu2_t" 844 | type: "ReLU" 845 | bottom: "bn2_t" 846 | top: "relu2_t" 847 | } 848 | 849 | layer { 850 | name: "pool2_t" 851 | type: "Pooling" 852 | bottom: "relu2_t" 853 | top: "pool2_t" 854 | pooling_param { 855 | pool: MAX 856 | kernel_size: 3 857 | stride: 2 858 | } 859 | } 860 | layer { 861 | name: "conv3_t" 862 | type: "Convolution" 863 | bottom: "pool2_t" 864 | top: "conv3_t" 865 | param { 866 | name: "conv3_w" 867 | lr_mult: 0.9 868 | decay_mult: 1 869 | } 870 | param { 871 | name: "conv3_b" 872 | lr_mult: 2 873 | decay_mult: 0 874 | } 875 | convolution_param { 876 | num_output: 384 877 | pad: 1 878 | kernel_size: 3 879 | weight_filler { 880 | type: "gaussian" 881 | std: 0.01 882 | } 883 | bias_filler { 884 | type: "constant" 885 | value: 0 886 | } 887 | } 888 | } 889 | layer { 890 | name: "bn3_t" 891 | type: "BatchNorm" 892 | bottom: "conv3_t" 893 | top: "bn3_t" 894 | param { 895 | lr_mult: 0 896 | } 897 | param { 898 | lr_mult: 0 899 | } 900 | param { 901 | lr_mult: 0 902 | } 903 | } 904 | 905 | layer { 906 | name: "relu3_t" 907 | type: "ReLU" 908 | bottom: "bn3_t" 909 | top: "relu3_t" 910 | } 911 | layer { 912 | name: "conv4_t" 913 | type: "Convolution" 914 | bottom: "relu3_t" 915 | top: "conv4_t" 916 | param { 917 | name: "conv4_w" 918 | lr_mult: 1 919 | decay_mult: 1 920 | } 921 | param { 922 | name: "conv4_b" 923 | lr_mult: 2 924 | decay_mult: 0 925 | } 926 | convolution_param { 927 | num_output: 384 928 | pad: 1 929 | kernel_size: 3 930 | group: 2 931 | weight_filler { 932 | type: "gaussian" 933 | std: 0.01 934 | } 935 | bias_filler { 936 | type: "constant" 937 | value: 0.1 938 | } 939 | } 940 | } 941 | layer { 942 | name: "bn4_t" 943 | type: "BatchNorm" 944 | bottom: "conv4_t" 945 | top: "bn4_t" 946 | param { 947 | lr_mult: 0 948 | } 949 | param { 950 | lr_mult: 0 951 | } 952 | param { 953 | lr_mult: 0 954 | } 955 | } 956 | 957 | layer { 958 | name: "relu4_t" 959 | type: "ReLU" 960 | bottom: "bn4_t" 961 | top: "relu4_t" 962 | } 963 | layer { 964 | name: "conv5_t" 965 | type: "Convolution" 966 | bottom: "relu4_t" 967 | top: "conv5_t" 968 | param { 969 | name: "conv5_w" 970 | lr_mult: 0.9 971 | decay_mult: 1 972 | } 973 | param { 974 | name: "conv5_b" 975 | lr_mult: 2 976 | decay_mult: 0 977 | } 978 | convolution_param { 979 | num_output: 256 980 | pad: 1 981 | kernel_size: 3 982 | group: 2 983 | weight_filler { 984 | type: "gaussian" 985 | std: 0.01 986 | } 987 | bias_filler { 988 | type: "constant" 989 | value: 0.1 990 | } 991 | } 992 | } 993 | layer { 994 | name: "bn5_t" 995 | type: "BatchNorm" 996 | bottom: "conv5_t" 997 | top: "bn5_t" 998 | param { 999 | lr_mult: 0 1000 | } 1001 | param { 1002 | lr_mult: 0 1003 | } 1004 | param { 1005 | lr_mult: 0 1006 | } 1007 | } 1008 | 1009 | layer { 1010 | name: "relu5_t" 1011 | type: "ReLU" 1012 | bottom: "bn5_t" 1013 | top: "relu5_t" 1014 | } 1015 | layer { 1016 | name: "pool5_t" 1017 | type: "Pooling" 1018 | bottom: "relu5_t" 1019 | top: "pool5_t" 1020 | pooling_param { 1021 | pool: MAX 1022 | kernel_size: 3 1023 | stride: 2 1024 | } 1025 | } 1026 | layer { 1027 | name: "fc6_t" 1028 | type: "InnerProduct" 1029 | bottom: "pool5_t" 1030 | top: "fc6_t" 1031 | param { 1032 | name: "fc6_w" 1033 | lr_mult: 0.8 1034 | decay_mult: 1 1035 | } 1036 | param { 1037 | name: "fc6_b" 1038 | lr_mult: 2 1039 | decay_mult: 0 1040 | } 1041 | inner_product_param { 1042 | num_output: 4096 1043 | weight_filler { 1044 | type: "gaussian" 1045 | std: 0.005 1046 | } 1047 | bias_filler { 1048 | type: "constant" 1049 | value: 0.1 1050 | } 1051 | } 1052 | } 1053 | layer { 1054 | name: "bn6_t" 1055 | type: "BatchNorm" 1056 | bottom: "fc6_t" 1057 | top: "bn6_t" 1058 | param { 1059 | lr_mult: 0 1060 | } 1061 | param { 1062 | lr_mult: 0 1063 | } 1064 | param { 1065 | lr_mult: 0 1066 | } 1067 | } 1068 | 1069 | layer { 1070 | name: "relu6_t" 1071 | type: "ReLU" 1072 | bottom: "bn6_t" 1073 | top: "relu6_t" 1074 | } 1075 | 1076 | # ================================================================================================ 1077 | # ================================================================================================ 1078 | # ======== JOIN NETWORK 1, 2 AND 3 1079 | # ================================================================================================ 1080 | # ================================================================================================ 1081 | 1082 | layer { 1083 | name: "joined_concat" 1084 | type: "Concat" 1085 | bottom: "relu6" 1086 | bottom: "relu6_s" 1087 | bottom: "relu6_t" 1088 | top: "joined_concat" 1089 | } 1090 | 1091 | layer { 1092 | name: "joined_fc1" 1093 | type: "InnerProduct" 1094 | bottom: "joined_concat" 1095 | top: "joined_fc1" 1096 | param { 1097 | lr_mult: 0.7 1098 | decay_mult: 1 1099 | } 1100 | param { 1101 | lr_mult: 2 1102 | decay_mult: 0 1103 | } 1104 | inner_product_param { 1105 | num_output: 4096 1106 | weight_filler { 1107 | type: "gaussian" 1108 | std: 0.01 1109 | } 1110 | bias_filler { 1111 | type: "constant" 1112 | value: 0 1113 | } 1114 | } 1115 | } 1116 | 1117 | layer { 1118 | name: "joined_bn1" 1119 | type: "BatchNorm" 1120 | bottom: "joined_fc1" 1121 | top: "joined_bn1" 1122 | param { 1123 | lr_mult: 0 1124 | } 1125 | param { 1126 | lr_mult: 0 1127 | } 1128 | param { 1129 | lr_mult: 0 1130 | } 1131 | } 1132 | 1133 | layer { 1134 | name: "joined_relu1" 1135 | type: "ReLU" 1136 | bottom: "joined_bn1" 1137 | top: "joined_relu1" 1138 | } 1139 | 1140 | layer { 1141 | name: "joined_fc2" 1142 | type: "InnerProduct" 1143 | bottom: "joined_relu1" 1144 | top: "joined_fc2" 1145 | param { 1146 | lr_mult: 0.6 1147 | decay_mult: 1 1148 | } 1149 | param { 1150 | lr_mult: 2 1151 | decay_mult: 0 1152 | } 1153 | inner_product_param { 1154 | num_output: 4096 1155 | weight_filler { 1156 | type: "gaussian" 1157 | std: 0.01 1158 | } 1159 | bias_filler { 1160 | type: "constant" 1161 | value: 0 1162 | } 1163 | } 1164 | } 1165 | 1166 | layer { 1167 | name: "joined_bn2" 1168 | type: "BatchNorm" 1169 | bottom: "joined_fc2" 1170 | top: "joined_bn2" 1171 | param { 1172 | lr_mult: 0 1173 | } 1174 | param { 1175 | lr_mult: 0 1176 | } 1177 | param { 1178 | lr_mult: 0 1179 | } 1180 | } 1181 | 1182 | layer { 1183 | name: "joined_relu2" 1184 | type: "ReLU" 1185 | bottom: "joined_bn2" 1186 | top: "joined_relu2" 1187 | } 1188 | 1189 | # ================================================================================================ 1190 | # ======== Loss 1191 | # ================================================================================================ 1192 | 1193 | layer { 1194 | name: "joined_fc3" 1195 | type: "InnerProduct" 1196 | bottom: "joined_relu2" 1197 | top: "joined_fc3" 1198 | param { 1199 | lr_mult: 0.5 1200 | decay_mult: 1 1201 | } 1202 | param { 1203 | lr_mult: 2 1204 | decay_mult: 0 1205 | } 1206 | inner_product_param { 1207 | num_output: 80 1208 | weight_filler { 1209 | type: "gaussian" 1210 | std: 0.01 1211 | } 1212 | bias_filler { 1213 | type: "constant" 1214 | value: 0 1215 | } 1216 | } 1217 | } 1218 | 1219 | layer { 1220 | name: "accuracy" 1221 | type: "Accuracy" 1222 | bottom: "joined_fc3" 1223 | bottom: "label" 1224 | top: "accuracy" 1225 | include { 1226 | phase: TEST 1227 | } 1228 | } 1229 | 1230 | layer { 1231 | name: "loss" 1232 | type: "SoftmaxWithLoss" 1233 | bottom: "joined_fc3" 1234 | bottom: "label" 1235 | top: "loss" 1236 | } 1237 | 1238 | layer { 1239 | name: "softmax_plain" 1240 | type: "Softmax" 1241 | bottom: "joined_fc3" 1242 | top: "softmax_plain" 1243 | loss_weight: 0 1244 | } 1245 | 1246 | layer { 1247 | name: "softmax_plain_silence" 1248 | type: "Silence" 1249 | bottom: "softmax_plain" 1250 | } 1251 | -------------------------------------------------------------------------------- /caffe_data/train_val_CaffeNet_single.prototxt: -------------------------------------------------------------------------------- 1 | # ================================================================================================ 2 | # ======== CaffeNet Model for use with training on CUB and Comp Cars dataset 3 | # ======== 4 | # ======== For training on VOC, Fast RCNN and Linear Models, you should use the CaffeNet 5 | # ======== model that comes with that package. 6 | # ================================================================================================ 7 | 8 | name: "CaffeNet" 9 | 10 | # train data 11 | layer { 12 | top: "data" 13 | top: "label" 14 | name: "data" 15 | type: "ImageData" 16 | image_data_param { 17 | source: "train_list.txt" 18 | batch_size: 64 19 | shuffle: true 20 | } 21 | transform_param { 22 | mirror: true 23 | crop_size: 224 24 | mean_value: 104 25 | mean_value: 117 26 | mean_value: 123 27 | } 28 | include { 29 | phase: TRAIN 30 | } 31 | } 32 | 33 | # test data 34 | layer { 35 | top: "data" 36 | top: "label" 37 | name: "data" 38 | type: "ImageData" 39 | image_data_param { 40 | source: "test_list.txt" 41 | batch_size: 10 42 | } 43 | transform_param { 44 | mirror: false 45 | crop_size: 224 46 | mean_value: 104 47 | mean_value: 117 48 | mean_value: 123 49 | } 50 | include { 51 | phase: TEST 52 | } 53 | } 54 | 55 | # ================================================================================================ 56 | # ======== Layer 1 57 | # ================================================================================================ 58 | 59 | layer { 60 | name: "conv1" 61 | type: "Convolution" 62 | bottom: "data" 63 | top: "conv1" 64 | param { 65 | lr_mult: 1 66 | decay_mult: 1 67 | } 68 | param { 69 | lr_mult: 2 70 | decay_mult: 0 71 | } 72 | convolution_param { 73 | num_output: 96 74 | kernel_size: 11 75 | stride: 4 76 | weight_filler { 77 | type: "gaussian" 78 | std: 0.01 79 | } 80 | bias_filler { 81 | type: "constant" 82 | value: 0 83 | } 84 | } 85 | } 86 | layer { 87 | name: "relu1" 88 | type: "ReLU" 89 | bottom: "conv1" 90 | top: "conv1" 91 | } 92 | layer { 93 | name: "pool1" 94 | type: "Pooling" 95 | bottom: "conv1" 96 | top: "pool1" 97 | pooling_param { 98 | pool: MAX 99 | kernel_size: 3 100 | stride: 2 101 | } 102 | } 103 | layer { 104 | name: "norm1" 105 | type: "LRN" 106 | bottom: "pool1" 107 | top: "norm1" 108 | lrn_param { 109 | local_size: 5 110 | alpha: 0.0001 111 | beta: 0.75 112 | } 113 | } 114 | layer { 115 | name: "conv2" 116 | type: "Convolution" 117 | bottom: "norm1" 118 | top: "conv2" 119 | param { 120 | lr_mult: 1 121 | decay_mult: 1 122 | } 123 | param { 124 | lr_mult: 2 125 | decay_mult: 0 126 | } 127 | convolution_param { 128 | num_output: 256 129 | pad: 2 130 | kernel_size: 5 131 | group: 2 132 | weight_filler { 133 | type: "gaussian" 134 | std: 0.01 135 | } 136 | bias_filler { 137 | type: "constant" 138 | value: 1 139 | } 140 | } 141 | } 142 | layer { 143 | name: "relu2" 144 | type: "ReLU" 145 | bottom: "conv2" 146 | top: "conv2" 147 | } 148 | layer { 149 | name: "pool2" 150 | type: "Pooling" 151 | bottom: "conv2" 152 | top: "pool2" 153 | pooling_param { 154 | pool: MAX 155 | kernel_size: 3 156 | stride: 2 157 | } 158 | } 159 | layer { 160 | name: "norm2" 161 | type: "LRN" 162 | bottom: "pool2" 163 | top: "norm2" 164 | lrn_param { 165 | local_size: 5 166 | alpha: 0.0001 167 | beta: 0.75 168 | } 169 | } 170 | layer { 171 | name: "conv3" 172 | type: "Convolution" 173 | bottom: "norm2" 174 | top: "conv3" 175 | param { 176 | lr_mult: 1 177 | decay_mult: 1 178 | } 179 | param { 180 | lr_mult: 2 181 | decay_mult: 0 182 | } 183 | convolution_param { 184 | num_output: 384 185 | pad: 1 186 | kernel_size: 3 187 | weight_filler { 188 | type: "gaussian" 189 | std: 0.01 190 | } 191 | bias_filler { 192 | type: "constant" 193 | value: 0 194 | } 195 | } 196 | } 197 | layer { 198 | name: "relu3" 199 | type: "ReLU" 200 | bottom: "conv3" 201 | top: "conv3" 202 | } 203 | layer { 204 | name: "conv4" 205 | type: "Convolution" 206 | bottom: "conv3" 207 | top: "conv4" 208 | param { 209 | lr_mult: 1 210 | decay_mult: 1 211 | } 212 | param { 213 | lr_mult: 2 214 | decay_mult: 0 215 | } 216 | convolution_param { 217 | num_output: 384 218 | pad: 1 219 | kernel_size: 3 220 | group: 2 221 | weight_filler { 222 | type: "gaussian" 223 | std: 0.01 224 | } 225 | bias_filler { 226 | type: "constant" 227 | value: 1 228 | } 229 | } 230 | } 231 | layer { 232 | name: "relu4" 233 | type: "ReLU" 234 | bottom: "conv4" 235 | top: "conv4" 236 | } 237 | layer { 238 | name: "conv5" 239 | type: "Convolution" 240 | bottom: "conv4" 241 | top: "conv5" 242 | param { 243 | lr_mult: 1 244 | decay_mult: 1 245 | } 246 | param { 247 | lr_mult: 2 248 | decay_mult: 0 249 | } 250 | convolution_param { 251 | num_output: 256 252 | pad: 1 253 | kernel_size: 3 254 | group: 2 255 | weight_filler { 256 | type: "gaussian" 257 | std: 0.01 258 | } 259 | bias_filler { 260 | type: "constant" 261 | value: 1 262 | } 263 | } 264 | } 265 | layer { 266 | name: "relu5" 267 | type: "ReLU" 268 | bottom: "conv5" 269 | top: "conv5" 270 | } 271 | layer { 272 | name: "pool5" 273 | type: "Pooling" 274 | bottom: "conv5" 275 | top: "pool5" 276 | pooling_param { 277 | pool: MAX 278 | kernel_size: 3 279 | stride: 2 280 | } 281 | } 282 | 283 | # ================================================================================================ 284 | # ======== Renamed Single Layers 285 | # ================================================================================================ 286 | 287 | layer { 288 | name: "fc6-ss" 289 | type: "InnerProduct" 290 | bottom: "pool5" 291 | top: "fc6-ss" 292 | param { 293 | lr_mult: 1 294 | decay_mult: 1 295 | } 296 | param { 297 | lr_mult: 2 298 | decay_mult: 0 299 | } 300 | inner_product_param { 301 | num_output: 4096 302 | weight_filler { 303 | type: "gaussian" 304 | std: 0.005 305 | } 306 | bias_filler { 307 | type: "constant" 308 | value: 1 309 | } 310 | } 311 | } 312 | layer { 313 | name: "relu6-ss" 314 | type: "ReLU" 315 | bottom: "fc6-ss" 316 | top: "fc6-ss" 317 | } 318 | layer { 319 | name: "drop6-ss" 320 | type: "Dropout" 321 | bottom: "fc6-ss" 322 | top: "fc6-ss" 323 | dropout_param { 324 | dropout_ratio: 0.1 325 | } 326 | } 327 | layer { 328 | name: "fc7-ss" 329 | type: "InnerProduct" 330 | bottom: "fc6-ss" 331 | top: "fc7-ss" 332 | param { 333 | lr_mult: 1 334 | decay_mult: 1 335 | } 336 | param { 337 | lr_mult: 2 338 | decay_mult: 0 339 | } 340 | inner_product_param { 341 | num_output: 4096 342 | weight_filler { 343 | type: "gaussian" 344 | std: 0.005 345 | } 346 | bias_filler { 347 | type: "constant" 348 | value: 1 349 | } 350 | } 351 | } 352 | layer { 353 | name: "relu7-ss" 354 | type: "ReLU" 355 | bottom: "fc7-ss" 356 | top: "fc7-ss" 357 | } 358 | layer { 359 | name: "drop7-ss" 360 | type: "Dropout" 361 | bottom: "fc7-ss" 362 | top: "fc7-ss" 363 | dropout_param { 364 | dropout_ratio: 0.1 365 | } 366 | } 367 | layer { 368 | name: "fc8-ss" 369 | type: "InnerProduct" 370 | bottom: "fc7-ss" 371 | top: "fc8-ss" 372 | param { 373 | lr_mult: 1 374 | decay_mult: 1 375 | } 376 | param { 377 | lr_mult: 2 378 | decay_mult: 0 379 | } 380 | inner_product_param { 381 | ###NUM_OUTPUT### 382 | weight_filler { 383 | type: "gaussian" 384 | std: 0.01 385 | } 386 | bias_filler { 387 | type: "constant" 388 | value: 0 389 | } 390 | } 391 | } 392 | layer { 393 | name: "accuracy" 394 | type: "Accuracy" 395 | bottom: "fc8-ss" 396 | bottom: "label" 397 | top: "accuracy" 398 | include { 399 | phase: TEST 400 | } 401 | } 402 | layer { 403 | name: "loss" 404 | type: "SoftmaxWithLoss" 405 | bottom: "fc8-ss" 406 | bottom: "label" 407 | top: "loss" 408 | } 409 | -------------------------------------------------------------------------------- /caffe_data/train_val_CaffeNet_triple.prototxt: -------------------------------------------------------------------------------- 1 | name: "CaffeNet" 2 | input: "data_1" 3 | input_dim: 10 4 | input_dim: 3 5 | input_dim: 96 6 | input_dim: 96 7 | input: "data_2" 8 | input_dim: 10 9 | input_dim: 3 10 | input_dim: 96 11 | input_dim: 96 12 | input: "data_3" 13 | input_dim: 10 14 | input_dim: 3 15 | input_dim: 96 16 | input_dim: 96 17 | input: "label" 18 | input_dim: 10 19 | input_dim: 1 20 | input_dim: 1 21 | input_dim: 1 22 | 23 | # ================================================================================================ 24 | # ================================================================================================ 25 | # ======== NETWORK 1 26 | # ================================================================================================ 27 | # ================================================================================================ 28 | 29 | layer { 30 | name: "conv1" 31 | type: "Convolution" 32 | bottom: "data_1" 33 | top: "conv1" 34 | param { 35 | lr_mult: 1 36 | decay_mult: 1 37 | name: "conv1_w" 38 | } 39 | param { 40 | lr_mult: 2 41 | decay_mult: 0 42 | name: "conv1_b" 43 | } 44 | convolution_param { 45 | num_output: 96 46 | kernel_size: 11 47 | stride: 4 48 | pad: 5 49 | weight_filler { 50 | type: "gaussian" 51 | std: 0.01 52 | } 53 | bias_filler { 54 | type: "constant" 55 | value: 0 56 | } 57 | } 58 | } 59 | 60 | layer { 61 | name: "relu1" 62 | type: "ReLU" 63 | bottom: "conv1" 64 | top: "conv1" 65 | } 66 | 67 | layer { 68 | name: "pool1" 69 | type: "Pooling" 70 | bottom: "conv1" 71 | top: "pool1" 72 | pooling_param { 73 | pool: MAX 74 | kernel_size: 3 75 | stride: 2 76 | } 77 | } 78 | 79 | layer { 80 | name: "norm1" 81 | type: "LRN" 82 | bottom: "pool1" 83 | top: "norm1" 84 | lrn_param { 85 | local_size: 5 86 | alpha: 0.0001 87 | beta: 0.75 88 | } 89 | } 90 | 91 | layer { 92 | name: "conv2" 93 | type: "Convolution" 94 | bottom: "norm1" 95 | top: "conv2" 96 | param { 97 | lr_mult: 1 98 | decay_mult: 1 99 | name: "conv2_w" 100 | } 101 | param { 102 | lr_mult: 2 103 | decay_mult: 0 104 | name: "conv2_b" 105 | } 106 | convolution_param { 107 | num_output: 256 108 | pad: 2 109 | kernel_size: 5 110 | group: 2 111 | weight_filler { 112 | type: "gaussian" 113 | std: 0.01 114 | } 115 | bias_filler { 116 | type: "constant" 117 | value: 1 118 | } 119 | } 120 | } 121 | 122 | layer { 123 | name: "relu2" 124 | type: "ReLU" 125 | bottom: "conv2" 126 | top: "conv2" 127 | } 128 | 129 | layer { 130 | name: "pool2" 131 | type: "Pooling" 132 | bottom: "conv2" 133 | top: "pool2" 134 | pooling_param { 135 | pool: MAX 136 | kernel_size: 3 137 | stride: 2 138 | } 139 | } 140 | layer { 141 | name: "norm2" 142 | type: "LRN" 143 | bottom: "pool2" 144 | top: "norm2" 145 | lrn_param { 146 | local_size: 5 147 | alpha: 0.0001 148 | beta: 0.75 149 | } 150 | } 151 | 152 | layer { 153 | name: "conv3" 154 | type: "Convolution" 155 | bottom: "norm2" 156 | top: "conv3" 157 | param { 158 | lr_mult: 1 159 | decay_mult: 1 160 | name: "conv3_w" 161 | } 162 | param { 163 | lr_mult: 2 164 | decay_mult: 0 165 | name: "conv3_b" 166 | } 167 | convolution_param { 168 | num_output: 384 169 | pad: 1 170 | kernel_size: 3 171 | weight_filler { 172 | type: "gaussian" 173 | std: 0.01 174 | } 175 | bias_filler { 176 | type: "constant" 177 | value: 0 178 | } 179 | } 180 | } 181 | 182 | layer { 183 | name: "relu3" 184 | type: "ReLU" 185 | bottom: "conv3" 186 | top: "conv3" 187 | } 188 | 189 | layer { 190 | name: "conv4" 191 | type: "Convolution" 192 | bottom: "conv3" 193 | top: "conv4" 194 | param { 195 | lr_mult: 1 196 | decay_mult: 1 197 | name: "conv4_w" 198 | } 199 | param { 200 | lr_mult: 2 201 | decay_mult: 0 202 | name: "conv4_b" 203 | } 204 | convolution_param { 205 | num_output: 384 206 | pad: 1 207 | kernel_size: 3 208 | group: 2 209 | weight_filler { 210 | type: "gaussian" 211 | std: 0.01 212 | } 213 | bias_filler { 214 | type: "constant" 215 | value: 1 216 | } 217 | } 218 | } 219 | 220 | layer { 221 | name: "relu4" 222 | type: "ReLU" 223 | bottom: "conv4" 224 | top: "conv4" 225 | } 226 | 227 | layer { 228 | name: "conv5" 229 | type: "Convolution" 230 | bottom: "conv4" 231 | top: "conv5" 232 | param { 233 | lr_mult: 1 234 | decay_mult: 1 235 | name: "conv5_w" 236 | } 237 | param { 238 | lr_mult: 2 239 | decay_mult: 0 240 | name: "conv5_b" 241 | } 242 | convolution_param { 243 | num_output: 256 244 | pad: 1 245 | kernel_size: 3 246 | group: 2 247 | weight_filler { 248 | type: "gaussian" 249 | std: 0.01 250 | } 251 | bias_filler { 252 | type: "constant" 253 | value: 1 254 | } 255 | } 256 | } 257 | 258 | layer { 259 | name: "relu5" 260 | type: "ReLU" 261 | bottom: "conv5" 262 | top: "conv5" 263 | } 264 | 265 | layer { 266 | name: "pool5" 267 | type: "Pooling" 268 | bottom: "conv5" 269 | top: "pool5" 270 | pooling_param { 271 | pool: MAX 272 | kernel_size: 3 273 | stride: 2 274 | } 275 | } 276 | 277 | # ================================================================================================ 278 | # ======== NEW PARTS 279 | # ================================================================================================ 280 | 281 | layer { 282 | name: "fc6" 283 | type: "InnerProduct" 284 | bottom: "pool5" 285 | top: "fc6" 286 | param { 287 | name: "fc6_w" 288 | lr_mult: 1 289 | decay_mult: 1 290 | } 291 | param { 292 | name: "fc6_b" 293 | lr_mult: 2 294 | decay_mult: 0 295 | } 296 | inner_product_param { 297 | num_output: 4096 298 | weight_filler { 299 | type: "gaussian" 300 | std: 0.005 301 | } 302 | bias_filler { 303 | type: "constant" 304 | value: 0.1 305 | } 306 | } 307 | } 308 | 309 | layer { 310 | name: "relu6" 311 | type: "ReLU" 312 | bottom: "fc6" 313 | top: "relu6" 314 | } 315 | 316 | # ================================================================================================ 317 | # ================================================================================================ 318 | # ======== NETWORK 2 319 | # ================================================================================================ 320 | # ================================================================================================ 321 | 322 | layer { 323 | name: "conv1_s" 324 | type: "Convolution" 325 | bottom: "data_2" 326 | top: "conv1_s" 327 | param { 328 | lr_mult: 1 329 | decay_mult: 1 330 | name: "conv1_w" 331 | } 332 | param { 333 | lr_mult: 2 334 | decay_mult: 0 335 | name: "conv1_b" 336 | } 337 | convolution_param { 338 | num_output: 96 339 | kernel_size: 11 340 | stride: 4 341 | pad: 5 342 | weight_filler { 343 | type: "gaussian" 344 | std: 0.01 345 | } 346 | bias_filler { 347 | type: "constant" 348 | value: 0 349 | } 350 | } 351 | } 352 | 353 | layer { 354 | name: "relu1_s" 355 | type: "ReLU" 356 | bottom: "conv1_s" 357 | top: "conv1_s" 358 | } 359 | 360 | layer { 361 | name: "pool1_s" 362 | type: "Pooling" 363 | bottom: "conv1_s" 364 | top: "pool1_s" 365 | pooling_param { 366 | pool: MAX 367 | kernel_size: 3 368 | stride: 2 369 | } 370 | } 371 | 372 | layer { 373 | name: "norm1_s" 374 | type: "LRN" 375 | bottom: "pool1_s" 376 | top: "norm1_s" 377 | lrn_param { 378 | local_size: 5 379 | alpha: 0.0001 380 | beta: 0.75 381 | } 382 | } 383 | 384 | layer { 385 | name: "conv2_s" 386 | type: "Convolution" 387 | bottom: "norm1_s" 388 | top: "conv2_s" 389 | param { 390 | lr_mult: 1 391 | decay_mult: 1 392 | name: "conv2_w" 393 | } 394 | param { 395 | lr_mult: 2 396 | decay_mult: 0 397 | name: "conv2_b" 398 | } 399 | convolution_param { 400 | num_output: 256 401 | pad: 2 402 | kernel_size: 5 403 | group: 2 404 | weight_filler { 405 | type: "gaussian" 406 | std: 0.01 407 | } 408 | bias_filler { 409 | type: "constant" 410 | value: 1 411 | } 412 | } 413 | } 414 | 415 | layer { 416 | name: "relu2_s" 417 | type: "ReLU" 418 | bottom: "conv2_s" 419 | top: "conv2_s" 420 | } 421 | 422 | layer { 423 | name: "pool2_s" 424 | type: "Pooling" 425 | bottom: "conv2_s" 426 | top: "pool2_s" 427 | pooling_param { 428 | pool: MAX 429 | kernel_size: 3 430 | stride: 2 431 | } 432 | } 433 | layer { 434 | name: "norm2_s" 435 | type: "LRN" 436 | bottom: "pool2_s" 437 | top: "norm2_s" 438 | lrn_param { 439 | local_size: 5 440 | alpha: 0.0001 441 | beta: 0.75 442 | } 443 | } 444 | 445 | layer { 446 | name: "conv3_s" 447 | type: "Convolution" 448 | bottom: "norm2_s" 449 | top: "conv3_s" 450 | param { 451 | lr_mult: 1 452 | decay_mult: 1 453 | name: "conv3_w" 454 | } 455 | param { 456 | lr_mult: 2 457 | decay_mult: 0 458 | name: "conv3_b" 459 | } 460 | convolution_param { 461 | num_output: 384 462 | pad: 1 463 | kernel_size: 3 464 | weight_filler { 465 | type: "gaussian" 466 | std: 0.01 467 | } 468 | bias_filler { 469 | type: "constant" 470 | value: 0 471 | } 472 | } 473 | } 474 | 475 | layer { 476 | name: "relu3_s" 477 | type: "ReLU" 478 | bottom: "conv3_s" 479 | top: "conv3_s" 480 | } 481 | 482 | layer { 483 | name: "conv4_s" 484 | type: "Convolution" 485 | bottom: "conv3_s" 486 | top: "conv4_s" 487 | param { 488 | lr_mult: 1 489 | decay_mult: 1 490 | name: "conv4_w" 491 | } 492 | param { 493 | lr_mult: 2 494 | decay_mult: 0 495 | name: "conv4_b" 496 | } 497 | convolution_param { 498 | num_output: 384 499 | pad: 1 500 | kernel_size: 3 501 | group: 2 502 | weight_filler { 503 | type: "gaussian" 504 | std: 0.01 505 | } 506 | bias_filler { 507 | type: "constant" 508 | value: 1 509 | } 510 | } 511 | } 512 | 513 | layer { 514 | name: "relu4_s" 515 | type: "ReLU" 516 | bottom: "conv4_s" 517 | top: "conv4_s" 518 | } 519 | 520 | layer { 521 | name: "conv5_s" 522 | type: "Convolution" 523 | bottom: "conv4_s" 524 | top: "conv5_s" 525 | param { 526 | lr_mult: 1 527 | decay_mult: 1 528 | name: "conv5_w" 529 | } 530 | param { 531 | lr_mult: 2 532 | decay_mult: 0 533 | name: "conv5_b" 534 | } 535 | convolution_param { 536 | num_output: 256 537 | pad: 1 538 | kernel_size: 3 539 | group: 2 540 | weight_filler { 541 | type: "gaussian" 542 | std: 0.01 543 | } 544 | bias_filler { 545 | type: "constant" 546 | value: 1 547 | } 548 | } 549 | } 550 | 551 | layer { 552 | name: "relu5_s" 553 | type: "ReLU" 554 | bottom: "conv5_s" 555 | top: "conv5_s" 556 | } 557 | 558 | layer { 559 | name: "pool5_s" 560 | type: "Pooling" 561 | bottom: "conv5_s" 562 | top: "pool5_s" 563 | pooling_param { 564 | pool: MAX 565 | kernel_size: 3 566 | stride: 2 567 | } 568 | } 569 | 570 | # ================================================================================================ 571 | # ======== NEW PARTS 572 | # ================================================================================================ 573 | 574 | layer { 575 | name: "fc6_s" 576 | type: "InnerProduct" 577 | bottom: "pool5_s" 578 | top: "fc6_s" 579 | param { 580 | name: "fc6_w" 581 | lr_mult: 1 582 | decay_mult: 1 583 | } 584 | param { 585 | name: "fc6_b" 586 | lr_mult: 2 587 | decay_mult: 0 588 | } 589 | inner_product_param { 590 | num_output: 4096 591 | weight_filler { 592 | type: "gaussian" 593 | std: 0.005 594 | } 595 | bias_filler { 596 | type: "constant" 597 | value: 0.1 598 | } 599 | } 600 | } 601 | 602 | 603 | layer { 604 | name: "relu6_s" 605 | type: "ReLU" 606 | bottom: "fc6_s" 607 | top: "relu6_s" 608 | } 609 | 610 | # ================================================================================================ 611 | # ================================================================================================ 612 | # ======== NETWORK 3 613 | # ================================================================================================ 614 | # ================================================================================================ 615 | 616 | layer { 617 | name: "conv1_t" 618 | type: "Convolution" 619 | bottom: "data_3" 620 | top: "conv1_t" 621 | param { 622 | lr_mult: 1 623 | decay_mult: 1 624 | name: "conv1_w" 625 | } 626 | param { 627 | lr_mult: 2 628 | decay_mult: 0 629 | name: "conv1_b" 630 | } 631 | convolution_param { 632 | num_output: 96 633 | kernel_size: 11 634 | stride: 4 635 | pad: 5 636 | weight_filler { 637 | type: "gaussian" 638 | std: 0.01 639 | } 640 | bias_filler { 641 | type: "constant" 642 | value: 0 643 | } 644 | } 645 | } 646 | 647 | layer { 648 | name: "relu1_t" 649 | type: "ReLU" 650 | bottom: "conv1_t" 651 | top: "conv1_t" 652 | } 653 | 654 | layer { 655 | name: "pool1_t" 656 | type: "Pooling" 657 | bottom: "conv1_t" 658 | top: "pool1_t" 659 | pooling_param { 660 | pool: MAX 661 | kernel_size: 3 662 | stride: 2 663 | } 664 | } 665 | 666 | layer { 667 | name: "norm1_t" 668 | type: "LRN" 669 | bottom: "pool1_t" 670 | top: "norm1_t" 671 | lrn_param { 672 | local_size: 5 673 | alpha: 0.0001 674 | beta: 0.75 675 | } 676 | } 677 | 678 | layer { 679 | name: "conv2_t" 680 | type: "Convolution" 681 | bottom: "norm1_t" 682 | top: "conv2_t" 683 | param { 684 | lr_mult: 1 685 | decay_mult: 1 686 | name: "conv2_w" 687 | } 688 | param { 689 | lr_mult: 2 690 | decay_mult: 0 691 | name: "conv2_b" 692 | } 693 | convolution_param { 694 | num_output: 256 695 | pad: 2 696 | kernel_size: 5 697 | group: 2 698 | weight_filler { 699 | type: "gaussian" 700 | std: 0.01 701 | } 702 | bias_filler { 703 | type: "constant" 704 | value: 1 705 | } 706 | } 707 | } 708 | 709 | layer { 710 | name: "relu2_t" 711 | type: "ReLU" 712 | bottom: "conv2_t" 713 | top: "conv2_t" 714 | } 715 | 716 | layer { 717 | name: "pool2_t" 718 | type: "Pooling" 719 | bottom: "conv2_t" 720 | top: "pool2_t" 721 | pooling_param { 722 | pool: MAX 723 | kernel_size: 3 724 | stride: 2 725 | } 726 | } 727 | layer { 728 | name: "norm2_t" 729 | type: "LRN" 730 | bottom: "pool2_t" 731 | top: "norm2_t" 732 | lrn_param { 733 | local_size: 5 734 | alpha: 0.0001 735 | beta: 0.75 736 | } 737 | } 738 | 739 | layer { 740 | name: "conv3_t" 741 | type: "Convolution" 742 | bottom: "norm2_t" 743 | top: "conv3_t" 744 | param { 745 | lr_mult: 1 746 | decay_mult: 1 747 | name: "conv3_w" 748 | } 749 | param { 750 | lr_mult: 2 751 | decay_mult: 0 752 | name: "conv3_b" 753 | } 754 | convolution_param { 755 | num_output: 384 756 | pad: 1 757 | kernel_size: 3 758 | weight_filler { 759 | type: "gaussian" 760 | std: 0.01 761 | } 762 | bias_filler { 763 | type: "constant" 764 | value: 0 765 | } 766 | } 767 | } 768 | 769 | layer { 770 | name: "relu3_t" 771 | type: "ReLU" 772 | bottom: "conv3_t" 773 | top: "conv3_t" 774 | } 775 | 776 | layer { 777 | name: "conv4_t" 778 | type: "Convolution" 779 | bottom: "conv3_t" 780 | top: "conv4_t" 781 | param { 782 | lr_mult: 1 783 | decay_mult: 1 784 | name: "conv4_w" 785 | } 786 | param { 787 | lr_mult: 2 788 | decay_mult: 0 789 | name: "conv4_b" 790 | } 791 | convolution_param { 792 | num_output: 384 793 | pad: 1 794 | kernel_size: 3 795 | group: 2 796 | weight_filler { 797 | type: "gaussian" 798 | std: 0.01 799 | } 800 | bias_filler { 801 | type: "constant" 802 | value: 1 803 | } 804 | } 805 | } 806 | 807 | layer { 808 | name: "relu4_t" 809 | type: "ReLU" 810 | bottom: "conv4_t" 811 | top: "conv4_t" 812 | } 813 | 814 | layer { 815 | name: "conv5_t" 816 | type: "Convolution" 817 | bottom: "conv4_t" 818 | top: "conv5_t" 819 | param { 820 | lr_mult: 1 821 | decay_mult: 1 822 | name: "conv5_w" 823 | } 824 | param { 825 | lr_mult: 2 826 | decay_mult: 0 827 | name: "conv5_b" 828 | } 829 | convolution_param { 830 | num_output: 256 831 | pad: 1 832 | kernel_size: 3 833 | group: 2 834 | weight_filler { 835 | type: "gaussian" 836 | std: 0.01 837 | } 838 | bias_filler { 839 | type: "constant" 840 | value: 1 841 | } 842 | } 843 | } 844 | 845 | layer { 846 | name: "relu5_t" 847 | type: "ReLU" 848 | bottom: "conv5_t" 849 | top: "conv5_t" 850 | } 851 | 852 | layer { 853 | name: "pool5_t" 854 | type: "Pooling" 855 | bottom: "conv5_t" 856 | top: "pool5_t" 857 | pooling_param { 858 | pool: MAX 859 | kernel_size: 3 860 | stride: 2 861 | } 862 | } 863 | 864 | # ================================================================================================ 865 | # ======== NEW PARTS 866 | # ================================================================================================ 867 | 868 | layer { 869 | name: "fc6_t" 870 | type: "InnerProduct" 871 | bottom: "pool5_t" 872 | top: "fc6_t" 873 | param { 874 | name: "fc6_w" 875 | lr_mult: 1 876 | decay_mult: 1 877 | } 878 | param { 879 | name: "fc6_b" 880 | lr_mult: 2 881 | decay_mult: 0 882 | } 883 | inner_product_param { 884 | num_output: 4096 885 | weight_filler { 886 | type: "gaussian" 887 | std: 0.005 888 | } 889 | bias_filler { 890 | type: "constant" 891 | value: 0.1 892 | } 893 | } 894 | } 895 | 896 | layer { 897 | name: "relu6_t" 898 | type: "ReLU" 899 | bottom: "fc6_t" 900 | top: "relu6_t" 901 | } 902 | 903 | # ================================================================================================ 904 | # ================================================================================================ 905 | # ======== JOIN NETWORK 1, 2 AND 3 906 | # ================================================================================================ 907 | # ================================================================================================ 908 | 909 | layer { 910 | name: "joined_concat" 911 | type: "Concat" 912 | bottom: "relu6" 913 | bottom: "relu6_s" 914 | bottom: "relu6_t" 915 | top: "joined_concat" 916 | } 917 | 918 | layer { 919 | name: "joined_fc1" 920 | type: "InnerProduct" 921 | bottom: "joined_concat" 922 | top: "joined_fc1" 923 | param { 924 | lr_mult: 1 925 | decay_mult: 1 926 | } 927 | param { 928 | lr_mult: 2 929 | decay_mult: 0 930 | } 931 | inner_product_param { 932 | num_output: 4096 933 | weight_filler { 934 | type: "gaussian" 935 | std: 0.01 936 | } 937 | bias_filler { 938 | type: "constant" 939 | value: 0 940 | } 941 | } 942 | } 943 | 944 | layer { 945 | name: "joined_relu1" 946 | type: "ReLU" 947 | bottom: "joined_fc1" 948 | top: "joined_relu1" 949 | } 950 | 951 | layer { 952 | name: "joined_drop1" 953 | type: "Dropout" 954 | bottom: "joined_relu1" 955 | top: "joined_drop1" 956 | dropout_param { 957 | dropout_ratio: 0.5 958 | } 959 | } 960 | 961 | layer { 962 | name: "joined_fc2" 963 | type: "InnerProduct" 964 | bottom: "joined_drop1" 965 | top: "joined_fc2" 966 | param { 967 | lr_mult: 1 968 | decay_mult: 1 969 | } 970 | param { 971 | lr_mult: 2 972 | decay_mult: 0 973 | } 974 | inner_product_param { 975 | num_output: 4096 976 | weight_filler { 977 | type: "gaussian" 978 | std: 0.01 979 | } 980 | bias_filler { 981 | type: "constant" 982 | value: 0 983 | } 984 | } 985 | } 986 | 987 | layer { 988 | name: "joined_relu2" 989 | type: "ReLU" 990 | bottom: "joined_fc2" 991 | top: "joined_relu2" 992 | } 993 | 994 | layer { 995 | name: "joined_drop2" 996 | type: "Dropout" 997 | bottom: "joined_relu2" 998 | top: "joined_drop2" 999 | dropout_param { 1000 | dropout_ratio: 0.5 1001 | } 1002 | } 1003 | 1004 | # ================================================================================================ 1005 | # ======== Loss 1006 | # ================================================================================================ 1007 | 1008 | layer { 1009 | name: "joined_fc3" 1010 | type: "InnerProduct" 1011 | bottom: "joined_drop2" 1012 | #bottom: "joined_relu2" 1013 | top: "joined_fc3" 1014 | param { 1015 | lr_mult: 1 1016 | decay_mult: 1 1017 | } 1018 | param { 1019 | lr_mult: 2 1020 | decay_mult: 0 1021 | } 1022 | inner_product_param { 1023 | num_output: 80 1024 | weight_filler { 1025 | type: "gaussian" 1026 | std: 0.01 1027 | } 1028 | bias_filler { 1029 | type: "constant" 1030 | value: 0 1031 | } 1032 | } 1033 | } 1034 | 1035 | layer { 1036 | name: "accuracy" 1037 | type: "Accuracy" 1038 | bottom: "joined_fc3" 1039 | bottom: "label" 1040 | top: "accuracy" 1041 | include { 1042 | phase: TEST 1043 | } 1044 | } 1045 | 1046 | layer { 1047 | name: "loss" 1048 | type: "SoftmaxWithLoss" 1049 | bottom: "joined_fc3" 1050 | bottom: "label" 1051 | top: "loss" 1052 | } 1053 | 1054 | layer { 1055 | name: "softmax_plain" 1056 | type: "Softmax" 1057 | bottom: "joined_fc3" 1058 | top: "softmax_plain" 1059 | loss_weight: 0 1060 | } 1061 | -------------------------------------------------------------------------------- /figures/README.md: -------------------------------------------------------------------------------- 1 | Training figures will be saved here. -------------------------------------------------------------------------------- /moab_output/README.md: -------------------------------------------------------------------------------- 1 | Generic logging dir. -------------------------------------------------------------------------------- /py_proj/README.md: -------------------------------------------------------------------------------- 1 | Your training scripts will be instantiated here in their own folders. -------------------------------------------------------------------------------- /python/LoadFunctions.py: -------------------------------------------------------------------------------- 1 | import cv2 2 | import numpy as np 3 | import copy 4 | import sys 5 | import signal 6 | 7 | def ctrl_c_handler(signum, frame): 8 | # do nothing, return and absorb the ctrl-c 9 | # otherwise, we go bananas when we try to exit. 10 | pass 11 | 12 | 13 | # ******************************************************************************************************************* 14 | def get_resize_flag(): 15 | """This function when called will randomly return one of four resize flags for OpenCV to use. 16 | """ 17 | resize_flag = np.random.randint(0,4) 18 | 19 | if resize_flag == 0: 20 | flag = cv2.INTER_LINEAR 21 | elif resize_flag == 1: 22 | flag = cv2.INTER_AREA 23 | elif resize_flag == 2: 24 | flag = cv2.INTER_CUBIC 25 | elif resize_flag == 3: 26 | flag = cv2.INTER_LANCZOS4 27 | 28 | return flag 29 | 30 | # ******************************************************************************************************************* 31 | def flip_and_rotate_w_classification(in_class, img1, img2, img3): 32 | """We will will mirror flip patches and then do rotation with classification (RWC). 33 | 34 | The odds of a left-right mirror flip are 50%. If we do a left-right flip, we have to relabel the class to match the 35 | new configuration. So, a good deal of this function body is dedicated to relabeling a class on left-right mirror flipping. 36 | 37 | When we do rotation with classification, we increment the class in steps of 20. 38 | 39 | This function returns the updated class and the new flipped and/or rotated patches. 40 | """ 41 | 42 | # roll a 0 or 1 43 | flip = np.random.randint(0,2) 44 | 45 | assert(flip < 2) 46 | assert(flip >= 0) 47 | 48 | in_class_val = int(in_class) 49 | 50 | # To flip or not to flip 51 | if flip == 0: 52 | nimg1 = img1 53 | nimg2 = img2 54 | nimg3 = img3 55 | out_class = in_class_val 56 | else: 57 | # Notes: 58 | # (1) some classes return the same since they are symmetric 59 | # (2) The 4 patches have to be swapped 60 | 61 | if in_class_val > 11: 62 | 63 | inv_flip = False 64 | 65 | nimg1 = np.fliplr(img1) 66 | 67 | if in_class_val == 12: 68 | out_class = 14 69 | elif in_class_val == 14: 70 | out_class = 12 71 | elif in_class_val == 17: 72 | out_class = 19 73 | elif in_class_val == 19: 74 | out_class = 17 75 | elif in_class_val == 13: 76 | out_class = 15 77 | inv_flip = True 78 | elif in_class_val == 15: 79 | out_class = 13 80 | inv_flip = True 81 | elif in_class_val == 16: 82 | out_class = 18 83 | inv_flip = True 84 | elif in_class_val == 18: 85 | out_class = 16 86 | inv_flip = True 87 | 88 | if inv_flip: 89 | nimg2 = np.fliplr(img3) 90 | nimg3 = np.fliplr(img2) 91 | else: 92 | nimg2 = np.fliplr(img2) 93 | nimg3 = np.fliplr(img3) 94 | 95 | elif in_class_val > 7: 96 | nimg1 = np.fliplr(img3) 97 | nimg2 = np.fliplr(img2) 98 | nimg3 = np.fliplr(img1) 99 | if in_class_val == 8: 100 | out_class = 11 101 | elif in_class_val == 11: 102 | out_class = 8 103 | elif in_class_val == 9: 104 | out_class = 10 105 | elif in_class_val == 10: 106 | out_class = 9 107 | else: 108 | nimg1 = np.fliplr(img1) 109 | nimg2 = np.fliplr(img2) 110 | nimg3 = np.fliplr(img3) 111 | 112 | if in_class_val == 0: 113 | out_class = 2 114 | elif in_class_val == 2: 115 | out_class = 0 116 | elif in_class_val == 3: 117 | out_class = 7 118 | elif in_class_val == 7: 119 | out_class = 3 120 | elif in_class_val == 4: 121 | out_class = 6 122 | elif in_class_val == 6: 123 | out_class = 4 124 | elif in_class_val == 1: 125 | out_class = 1 126 | elif in_class_val == 5: 127 | out_class = 5 128 | 129 | # 80 Class rotation with classification 130 | 131 | # Roll a 0,1,2 or 3 132 | flip = np.random.randint(0,4) 133 | 134 | if flip == 1: # Rotate 90 Degrees 135 | nimg1 = np.rot90(nimg1) 136 | nimg2 = np.rot90(nimg2) 137 | nimg3 = np.rot90(nimg3) 138 | out_class += 20 139 | elif flip == 2: # Rotate 180 Degrees 140 | nimg1 = np.rot90(nimg1,k=2) 141 | nimg2 = np.rot90(nimg2,k=2) 142 | nimg3 = np.rot90(nimg3,k=2) 143 | out_class += 40 144 | elif flip == 3: # Rotate 270 Degrees 145 | nimg1 = np.rot90(nimg1,k=3) 146 | nimg2 = np.rot90(nimg2,k=3) 147 | nimg3 = np.rot90(nimg3,k=3) 148 | out_class += 60 149 | 150 | return out_class, nimg1, nimg2, nimg3 151 | 152 | # ******************************************************************************************************************* 153 | def load_cv_image_triple(proc, img_name_1, img_name_2, img_name_3, in_class, min_size, max_size, patch_size, rand_seed, mean_color): 154 | """This function will launch in its own process from a pool to load and augment image patches. 155 | 156 | We use OpenCV to load in three patches. We then decide how much to resize the patches, and then we do that. 157 | We then decide on the size of the internal aperture and apply it randomly to two of the patches. 158 | 159 | We return the process ID so we can do some safety book keeping. We also return the three patches and the 160 | updated class from executing flipping and rotation with classification. 161 | """ 162 | try: 163 | # Put in a dummy ctrl-c handler so this process will ignore it 164 | signal.signal(signal.SIGINT, ctrl_c_handler) 165 | 166 | assert(patch_size >= min_size) 167 | assert(max_size >= min_size) 168 | assert(patch_size <= max_size) 169 | 170 | do_exit = False 171 | 172 | # set our random number seed 173 | np.random.seed(rand_seed) 174 | 175 | # load in images 176 | img_1 = cv2.imread(img_name_1) 177 | img_2 = cv2.imread(img_name_2) 178 | img_3 = cv2.imread(img_name_3) 179 | 180 | # Check images to make sure they loaded 181 | if type(img_1) is not np.ndarray: 182 | print (">>>>> Bad Image {}".format(img_name_1)) 183 | do_exit = True 184 | 185 | if type(img_2) is not np.ndarray: 186 | print (">>>>> Bad Image {}".format(img_name_2)) 187 | do_exit = True 188 | 189 | if type(img_3) is not np.ndarray: 190 | print (">>>>> Bad Image {}".format(img_name_3)) 191 | do_exit = True 192 | 193 | # determine patch size and how to rescale it 194 | y_size = float(img_1.shape[0]) 195 | x_size = float(img_1.shape[1]) 196 | 197 | new_size = np.random.randint(min_size,max_size+1) 198 | 199 | if y_size > x_size: 200 | new_y_size = new_size 201 | new_x_size = int(round(new_size * (x_size/y_size))) 202 | elif y_size < x_size: 203 | new_x_size = new_size 204 | new_y_size = int(round(new_size * (y_size/x_size))) 205 | else: 206 | new_x_size = new_size 207 | new_y_size = new_size 208 | 209 | # Apply yoked random jitter on patches while we resize the patches 210 | new_pos_x = np.random.randint(0,new_x_size-patch_size+1) 211 | new_pos_y = np.random.randint(0,new_y_size-patch_size+1) 212 | 213 | new_img_1 = cv2.resize(img_1, (new_x_size,new_y_size))[new_pos_y:new_pos_y+patch_size,new_pos_x:new_pos_x+patch_size,:] 214 | 215 | new_img_2 = cv2.resize(img_2, (new_x_size,new_y_size))[new_pos_y:new_pos_y+patch_size,new_pos_x:new_pos_x+patch_size,:] 216 | 217 | new_img_3 = cv2.resize(img_3, (new_x_size,new_y_size))[new_pos_y:new_pos_y+patch_size,new_pos_x:new_pos_x+patch_size,:] 218 | 219 | # Determine the size and position of the random aperture 220 | # Also, add 1 since randint is exlusive with the last number in range 221 | internal_size = np.random.randint(patch_size-32,patch_size+1) 222 | 223 | internal_pos_x = np.random.randint(0,patch_size-internal_size+1) 224 | internal_pos_y = np.random.randint(0,patch_size-internal_size+1) 225 | 226 | # Which patch does not get apertured? 227 | leave_ok_roll = np.random.randint(0,3) 228 | 229 | # create a new empty image template and set it to mean gray 230 | return_img_tmpl = np.empty((patch_size, patch_size, 3),dtype=np.uint8) 231 | 232 | return_img_tmpl[:,:,0] = mean_color[0] 233 | return_img_tmpl[:,:,1] = mean_color[1] 234 | return_img_tmpl[:,:,2] = mean_color[2] 235 | 236 | # set pointers to the images we will apply aperture to 237 | if leave_ok_roll == 0: 238 | new_img_A = new_img_1 239 | new_img_B = new_img_2 240 | elif leave_ok_roll == 1: 241 | new_img_A = new_img_1 242 | new_img_B = new_img_3 243 | else: 244 | new_img_A = new_img_2 245 | new_img_B = new_img_3 246 | 247 | # make one more copy of mean gray image 248 | return_img_A = return_img_tmpl 249 | return_img_B = copy.deepcopy(return_img_tmpl) 250 | 251 | # copy the visible area over the mean gray image to create the aperture images 252 | return_img_A[internal_pos_y:internal_pos_y+internal_size, internal_pos_x:internal_pos_x+internal_size, :] = \ 253 | new_img_A[internal_pos_y:internal_pos_y+internal_size, internal_pos_x:internal_pos_x+internal_size, :] 254 | return_img_B[internal_pos_y:internal_pos_y+internal_size, internal_pos_x:internal_pos_x+internal_size, :] = \ 255 | new_img_B[internal_pos_y:internal_pos_y+internal_size, internal_pos_x:internal_pos_x+internal_size, :] 256 | 257 | # return the aperture image patches in the correct order. 258 | if leave_ok_roll == 0: 259 | out_class, nimg1, nimg2, nimg3 = flip_and_rotate_w_classification(in_class, return_img_A, return_img_B, new_img_3) 260 | elif leave_ok_roll == 1: 261 | out_class, nimg1, nimg2, nimg3 = flip_and_rotate_w_classification(in_class, return_img_A, new_img_2, return_img_B) 262 | else: 263 | out_class, nimg1, nimg2, nimg3 = flip_and_rotate_w_classification(in_class, new_img_1, return_img_A, return_img_B) 264 | 265 | except Exception as e: 266 | print('Error LoadFunctions on line {}'.format(sys.exc_info()[-1].tb_lineno), type(e).__name__, e) 267 | do_exit = True 268 | 269 | return proc, nimg1, nimg2, nimg3, out_class, do_exit 270 | 271 | # ******************************************************************************************************************* 272 | # ******************************************************************************************************************* 273 | # EXPERIMENTAL 274 | # ******************************************************************************************************************* 275 | # ******************************************************************************************************************* 276 | 277 | 278 | -------------------------------------------------------------------------------- /python/LoadTrainImages.py: -------------------------------------------------------------------------------- 1 | import caffe 2 | import numpy as np 3 | import random 4 | import copy 5 | from multiprocessing import Pool, TimeoutError 6 | import sys 7 | import LoadFunctions as loadf 8 | 9 | # ******************************************************************************************************************* 10 | class Load: 11 | """ This is a loader class. We will create a few instance in different threads so we have a rotating buffer. 12 | 13 | The class has a variety of parameters that need to be set once. Otherwise, we just call load_triple_batch each 14 | time we want to load a bunch of patches. The class will take care of shuffling the order of samples as well as 15 | any processing we need to apply to each patch. Loading patches is handled via process pool since Python threading 16 | is weird. Take a look at PrefetchTrain.py for how this is initialized and called. 17 | """ 18 | def __init__(self, batch_size, patch_size, solver, mean_val, image_file): 19 | """This initialized the loader class. See: train_batch_pre_load_triple for example of how this works 20 | """ 21 | # Initial default values 22 | self._use_triple = False 23 | self._use_load_pool = False 24 | self._use_ra = False 25 | 26 | self._mean_color = np.float32(mean_val) 27 | 28 | # Simple variables that we need to set local as a copy 29 | self._batch_size = copy.deepcopy(batch_size) 30 | self._patch_size = copy.deepcopy(patch_size) 31 | 32 | # complex objects and lists 33 | self._image_file = image_file 34 | self._path_prefix = "" 35 | 36 | # Allocate some memories we will use several times that hold the loaded patches 37 | self._image_set_1 = np.zeros((batch_size,3,patch_size,patch_size)) 38 | self._image_set_2 = np.zeros((batch_size,3,patch_size,patch_size)) 39 | self._image_set_3 = np.zeros((batch_size,3,patch_size,patch_size)) 40 | self._label_set = batch_size*[None] 41 | 42 | self._load_pool = None 43 | self._img_idx = [] 44 | 45 | # Set up default random aperture parameters, we will override these later 46 | self._ra_min_size = 0 47 | self._ra_max_size = 0 48 | self._ra_patch_size = 0 49 | 50 | # go ahead and do an initial shuffle of the data 51 | self.shuffleIdx() 52 | 53 | def useRandomAperture(self, min_size, max_size, patch_size): 54 | """Set up random aperture parameters 55 | """ 56 | self._use_ra = True 57 | self._ra_min_size = min_size 58 | self._ra_max_size = max_size 59 | self._ra_patch_size = patch_size 60 | 61 | def useTriple(self, solver): 62 | """Set up triple patch parameters and transformers 63 | """ 64 | self._use_triple = True 65 | 66 | self._train_transformer_1 = copy.deepcopy(caffe.io.Transformer({'data_1': solver.net.blobs['data_1'].data.shape})) 67 | self._train_transformer_1.set_transpose('data_1', (2,0,1)) 68 | self._train_transformer_1.set_mean('data_1', self._mean_color) # mean pixel 69 | 70 | self._train_transformer_2 = copy.deepcopy(caffe.io.Transformer({'data_2': solver.net.blobs['data_2'].data.shape})) 71 | self._train_transformer_2.set_transpose('data_2', (2,0,1)) 72 | self._train_transformer_2.set_mean('data_2', self._mean_color) # mean pixel 73 | 74 | self._train_transformer_3 = copy.deepcopy(caffe.io.Transformer({'data_3': solver.net.blobs['data_3'].data.shape})) 75 | self._train_transformer_3.set_transpose('data_3', (2,0,1)) 76 | self._train_transformer_3.set_mean('data_3', self._mean_color) # mean pixel 77 | 78 | def useLoadPool(self, procs = 0): 79 | """Set up the loading pool processes. 80 | """ 81 | self._use_load_pool = True 82 | if procs == 0: 83 | self._load_pool = Pool(processes=self._batch_size) # probably a bad idea 84 | else: 85 | self._load_pool = Pool(processes=procs) 86 | 87 | def shuffleIdx(self): 88 | """Shuffle the data by shuffling an index which is much faster and easier than shuffling the actual data. 89 | """ 90 | 91 | print ("Shuffle data") 92 | 93 | self._img_idx = [] 94 | 95 | list_length = self._image_file.length() 96 | 97 | print (" {} items".format(list_length)) 98 | 99 | for x in range(list_length ): 100 | self._img_idx.append(x) 101 | 102 | random.shuffle(self._img_idx) 103 | 104 | print ("... Done") 105 | 106 | def loadTripleBatch(self): 107 | """Loader for the triple patches which run in their own thread and call a pool of loader processes. 108 | 109 | This function will get the shuffled list of patches along with their labels. These are then fed to the loading 110 | pool of processes. This will then get batch_size number of patch sets. Note that if we are using multiple GPU's 111 | the local batch size is divided by the total number of GPU's requested. 112 | """ 113 | try: 114 | 115 | assert(self._use_ra == True) 116 | assert(self._use_triple == True) 117 | assert(self._use_load_pool == True) 118 | assert(self._image_file.__class__.__name__ == "CompactList") 119 | 120 | do_exit = False 121 | 122 | #if we have fewer indices than the batch size, do a new shuffle 123 | if len(self._img_idx) < self._batch_size: 124 | self.shuffleIdx() 125 | 126 | my_cv_img1 = [] 127 | my_cv_img2 = [] 128 | my_cv_img3 = [] 129 | my_label = [] 130 | 131 | my_im_name1 = [] 132 | my_im_name2 = [] 133 | my_im_name3 = [] 134 | 135 | # for each patch set in the local batch size, get the file name and label 136 | for i in range(self._batch_size): 137 | 138 | # if we run out of indices, do a new shuffle 139 | if len(self._img_idx) == 0: 140 | self.shuffleIdx() 141 | 142 | # get the new index 143 | idx = self._img_idx.pop() 144 | 145 | # get the file name 146 | files, ilabel = self._image_file.getFileNames(idx) 147 | im_name1 = self._path_prefix + files[0] 148 | im_name2 = self._path_prefix + files[1] 149 | im_name3 = self._path_prefix + files[2] 150 | 151 | # append file names and class labels to our list 152 | my_im_name1.append(im_name1) 153 | my_im_name2.append(im_name2) 154 | my_im_name3.append(im_name3) 155 | my_label.append(ilabel) 156 | 157 | assert(self._use_load_pool == True) 158 | 159 | # roll a different random number seed for each loading process 160 | rand_seed = [] 161 | for i in range(self._batch_size): 162 | r = np.random.randint(0,pow(2,32)) 163 | rand_seed.append(r) 164 | 165 | # Launch the load pool processes and load/process the patch images. 166 | multiple_results = [self._load_pool.apply_async(loadf.load_cv_image_triple, 167 | (i, my_im_name1[i], my_im_name2[i], my_im_name3[i], my_label[i], 168 | self._ra_min_size, self._ra_max_size, self._ra_patch_size, rand_seed[i], self._mean_color)) 169 | for i in range(self._batch_size)] 170 | 171 | # for each process, return the patch images and class 172 | # check a counter to make sure things were kept in order 173 | try: 174 | vals = [res.get() for res in multiple_results] 175 | counter = 0 176 | for i in vals: 177 | my_cv_img1.append(i[1]) 178 | my_cv_img2.append(i[2]) 179 | my_cv_img3.append(i[3]) 180 | self._label_set[counter] = i[4] 181 | if i[5] == True: do_exit = True 182 | assert(i[0] == counter) # Make sure order was preserved by process pool 183 | counter += 1 184 | 185 | except TimeoutError: 186 | print "multiprocessing.TimeoutError" 187 | 188 | # For each patch set, apply Caffe's preprocessor and check sizing to be paranoid. 189 | for i in range(self._batch_size): 190 | 191 | nimg1 = my_cv_img1[i] 192 | nimg2 = my_cv_img2[i] 193 | nimg3 = my_cv_img3[i] 194 | 195 | in_img = self._train_transformer_1.preprocess('data_1', nimg1) 196 | if in_img.shape[2] is not self._patch_size: 197 | print("1 Patch wrong sized!!!") 198 | print("Should be {} got {}".format(self._patch_size,in_img.shape[2])) 199 | do_exit = True 200 | if in_img.shape[1] is not self._patch_size: 201 | print("2 Patch wrong sized!!!") 202 | print("Should be {} got {}".format(self._patch_size,in_img.shape[1])) 203 | do_exit = True 204 | self._image_set_1[i,:,:,:] = in_img 205 | 206 | in_img = self._train_transformer_2.preprocess('data_2', nimg2) 207 | if in_img.shape[2] is not self._patch_size: 208 | print("3 Patch wrong sized!!!") 209 | print ("Should be {} got {}".format(self._patch_size,in_img.shape[2])) 210 | do_exit = True 211 | if in_img.shape[1] is not self._patch_size: 212 | print("4 Patch wrong sized!!!") 213 | print("Should be {} got {}".format(self._patch_size,in_img.shape[1])) 214 | do_exit = True 215 | self._image_set_2[i,:,:,:] = in_img 216 | 217 | in_img = self._train_transformer_3.preprocess('data_3', nimg3) 218 | if in_img.shape[2] is not self._patch_size: 219 | print("5 Patch wrong sized!!!") 220 | print("Should be {} got {}".format(self._patch_size,in_img.shape[2])) 221 | do_exit = True 222 | if in_img.shape[1] is not self._patch_size: 223 | print("6 Patch wrong sized!!!") 224 | print("Should be {} got {}".format(self._patch_size,in_img.shape[1])) 225 | do_exit = True 226 | self._image_set_3[i,:,:,:] = in_img 227 | 228 | except Exception as e: 229 | print('Error in LoadTrainImages on line {}'.format(sys.exc_info()[-1].tb_lineno), type(e).__name__, e) 230 | do_exit = True 231 | # return the patch sets in the batch along with the class labels 232 | return self._image_set_1, self._image_set_2, self._image_set_3, self._label_set, do_exit 233 | -------------------------------------------------------------------------------- /python/NumpyFileList.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import string 3 | 4 | class NatesDict(dict): 5 | """We use a dictionary to find very basic redundancies in file names. 6 | """ 7 | def __missing__(self, key): 8 | return -1 9 | 10 | class CompactList: 11 | """This is a simple class to store sample file names in a simple compressed form. 12 | 13 | Using a text file with as many as 25 million file names can take up too much memory. So, 14 | we split the file names by redundant parts. It's fairly quick and good enough. 15 | """ 16 | def __init__(self): 17 | """Init various containers 18 | """ 19 | self._file_root_list = list() 20 | self._file_root_dict = NatesDict() 21 | self._file_variant_list = list() 22 | self._file_variant_dict = NatesDict() 23 | self._item_root_list = list() 24 | self._item_variant_list = list() 25 | self._item_class_list = list() 26 | self._max_class = 0 27 | self._variant_divider = "." 28 | self._load = False 29 | 30 | def load(self, file_name): 31 | """Load in a list and connect to the correct objects 32 | """ 33 | 34 | npz_file = np.load(file_name) 35 | 36 | self._item_root_list = npz_file['item_root_list'] 37 | self._item_variant_list = npz_file['item_variant_list'] 38 | self._item_class_list = npz_file['item_class_list'] 39 | self._file_root_list = npz_file['file_root_list'] 40 | self._file_variant_list = npz_file['file_variant_list'] 41 | self._max_class = npz_file['max_class'] 42 | self._variant_divider = npz_file['variant_divider'] 43 | self._load = True 44 | 45 | def save(self, file_name): 46 | """Save a list. Note that we do not support editing of already existing lists. 47 | """ 48 | assert(self._load == False) # NOT SUPPORTED TO WRITE BACK 49 | 50 | np.savez_compressed(file_name, 51 | item_root_list = self._item_root_list, 52 | item_variant_list = self._item_variant_list, 53 | item_class_list = self._item_class_list, 54 | file_root_list = self._file_root_list, 55 | file_variant_list = self._file_variant_list, 56 | variant_divider = self._variant_divider, 57 | max_class = self._max_class) 58 | 59 | def length(self): 60 | """How many samples are in this list? 61 | """ 62 | return len(self._item_root_list) 63 | 64 | def addVariantWithCheck(self,variant): 65 | 66 | v_idx = self._file_variant_dict[variant] 67 | 68 | if v_idx == -1: 69 | v_idx = len(self._file_variant_list) 70 | self._file_variant_list.append(variant) 71 | self._file_variant_dict[variant] = v_idx 72 | 73 | return v_idx 74 | 75 | def addRootWithCheck(self,root): 76 | 77 | r_idx = self._file_root_dict[root] 78 | 79 | if r_idx == -1: 80 | r_idx = len(self._file_root_list) 81 | self._file_root_list.append(root) 82 | self._file_root_dict[root] = r_idx 83 | 84 | return r_idx 85 | 86 | def insertFileNames(self, file_names, item_class): 87 | """Insert a new sample file name in to the list 88 | """ 89 | root_list = [] 90 | variant_list = [] 91 | 92 | for file_name in file_names: 93 | 94 | lparts = string.split(file_name,'/') 95 | 96 | rp = "" 97 | variants = [] 98 | 99 | for n in range(len(lparts)-1): 100 | rp = rp + lparts[n] + "/" 101 | 102 | idx = self.addRootWithCheck(rp) 103 | roots = idx 104 | 105 | lparts = string.split(lparts[len(lparts)-1],self._variant_divider) 106 | 107 | for part in lparts: 108 | idx = self.addVariantWithCheck(part) 109 | variants.append(idx) 110 | 111 | root_list.append(roots) 112 | variant_list.append(variants) 113 | 114 | if int(item_class) > self._max_class: 115 | self._max_class = int(item_class) 116 | 117 | self._item_root_list.append(root_list) 118 | self._item_variant_list.append(variant_list) 119 | self._item_class_list.append(item_class) 120 | 121 | def getFileNames(self, idx): 122 | """Reassemble the file names for samples at the given index 123 | """ 124 | roots = self._item_root_list[idx] 125 | variants = self._item_variant_list[idx] 126 | 127 | vc = 0 128 | file_names = [] 129 | 130 | for root in roots: 131 | 132 | variant = variants[vc] 133 | 134 | file_name = self._file_root_list[root] 135 | 136 | for i in range(len(variant)-1): 137 | file_name = file_name + self._file_variant_list[variant[i]] + "{}".format(self._variant_divider) 138 | 139 | file_name = file_name + self._file_variant_list[variant[len(variant)-1]] 140 | 141 | vc += 1 142 | 143 | file_names.append(file_name) 144 | 145 | return file_names, self._item_class_list[idx] 146 | 147 | def parseList(self, list_in, variant_divider = "."): 148 | """From a raw list, create our numpy file list 149 | """ 150 | self._variant_divider = variant_divider 151 | 152 | for line in list_in.readlines(): 153 | #split by white spaces 154 | lparse = string.strip(line, '\n') 155 | lparts = string.split(lparse) 156 | 157 | self.insertFileNames(lparts[0:len(lparts)-1], lparts[len(lparts)-1]) 158 | 159 | 160 | 161 | 162 | 163 | -------------------------------------------------------------------------------- /python/PrefetchTrain.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | import cv2 3 | import sys 4 | import pylab 5 | import types 6 | import os 7 | import LoadTrainImages 8 | import matplotlib.pyplot as plt 9 | import multiprocessing 10 | import string 11 | import subprocess 12 | import shutil 13 | from multiprocessing.pool import ThreadPool 14 | from caffe.proto import caffe_pb2 15 | from google.protobuf import text_format 16 | 17 | # ******************************************************************************************************************* 18 | class TrainParams: 19 | """Stand alone container class for parameters used in training. 20 | 21 | Typically, you will see this class object abbreviated as 'tp'. 22 | """ 23 | def __init__(self, solver, patch_size, patch_marg_1, patch_marg_2, train_batch_size, test_batch_size, 24 | bin_num, image_mean, loss_layer, softmax_layer): 25 | 26 | self.solver = solver 27 | self.patch_size = patch_size 28 | self.patch_marg_1 = patch_marg_1 29 | self.patch_marg_2 = patch_marg_2 30 | self.train_batch_size = train_batch_size 31 | self.test_batch_size = test_batch_size 32 | self.bin_num = bin_num # Number of class bins 33 | self.image_mean = image_mean 34 | self.loss_layer = loss_layer 35 | self.softmax_layer = softmax_layer 36 | self.test_skip = 0 37 | self.pool_procs = multiprocessing.cpu_count() 38 | self.test_iters = -1 39 | self.ra_min_size = patch_size 40 | self.ra_max_size = patch_size 41 | self.ra_patch_size = patch_size 42 | self.path_prefix = "" 43 | 44 | # ******************************************************************************************************************* 45 | def check_file(file_name,rank = 0): 46 | # Check if your files are here 47 | if os.path.isfile(file_name): 48 | return 1 49 | else: 50 | if rank == 0: 51 | print("Error: file not found \'{}\'".format(file_name)) 52 | print("Be sure and edit the python path to match your location") 53 | return 0 54 | 55 | # ******************************************************************************************************************* 56 | def check_dir(dir_name,rank = 0): 57 | # Check if your files are here 58 | if os.path.isdir(dir_name): 59 | return 1 60 | else: 61 | if rank == 0: 62 | print("Error: directory not found \'{}\'".format(dir_name)) 63 | print("Be sure and edit the python path to match your location") 64 | return 0 65 | 66 | # ******************************************************************************************************************* 67 | def check_create_dir(dir_name,rank = 0): 68 | # Check if your files are here 69 | if os.path.isdir(dir_name): 70 | pass 71 | else: 72 | if rank == 0: 73 | print("Creating Directory: \'{}\'".format(dir_name)) 74 | os.mkdir(dir_name) 75 | 76 | # ******************************************************************************************************************* 77 | def check_X_is_running(): 78 | from subprocess import Popen, PIPE 79 | p = Popen(["xset", "-q"], stdout=PIPE, stderr=PIPE) 80 | p.communicate() 81 | return p.returncode == 0 82 | 83 | # ******************************************************************************************************************* 84 | def read_proto_solver_file(filepath): 85 | """Helper to read Caffe solver files 86 | """ 87 | _ = check_file(filepath) 88 | solver_config = caffe_pb2.SolverParameter() 89 | return read_proto_file(filepath, solver_config) 90 | 91 | # ******************************************************************************************************************* 92 | def read_proto_file(filepath, parser_object): 93 | """Helper to read Caffe solver files 94 | """ 95 | _ = check_file(filepath) 96 | my_file = open(filepath, "r") 97 | 98 | text_format.Merge(str(my_file.read()), parser_object) 99 | my_file.close() 100 | return parser_object 101 | 102 | # ******************************************************************************************************************* 103 | class AsyncFunction: 104 | """This is a basic wrapper to create a rotating buffer of loading threads. 105 | 106 | Note that each thread will create a pool of processes to load the actual patch images. 107 | """ 108 | def __init__(self, num_process, f): 109 | self.func = f 110 | self.pool = ThreadPool(processes=num_process) 111 | 112 | def run(self, *args): 113 | result = self.pool.apply_async(self.func, args) 114 | return result 115 | 116 | # ******************************************************************************************************************* 117 | def train_batch_triple_init(image_file, tp): 118 | """Given training parameters (see above) we initialize two loading threads and return their handles. 119 | 120 | This function runs only once at the beginning of training. 121 | """ 122 | 123 | print ("Init loader") 124 | # create two loader objects 125 | lti_1 = LoadTrainImages.Load(tp.train_batch_size, tp.patch_size, tp.solver, tp.image_mean, image_file) 126 | lti_2 = LoadTrainImages.Load(tp.train_batch_size, tp.patch_size, tp.solver, tp.image_mean, image_file) 127 | 128 | # We have extra function calls as a legacy from older code. 129 | lti_1.useTriple(tp.solver) 130 | lti_2.useTriple(tp.solver) 131 | 132 | lti_1.useRandomAperture(tp.ra_min_size, tp.ra_max_size, tp.ra_patch_size) # Set up random aperture 133 | lti_2.useRandomAperture(tp.ra_min_size, tp.ra_max_size, tp.ra_patch_size) # Set up random aperture 134 | 135 | lti_1.useLoadPool(tp.pool_procs) 136 | lti_2.useLoadPool(tp.pool_procs) 137 | 138 | lti_1._path_prefix = tp.path_prefix 139 | lti_2._path_prefix = tp.path_prefix 140 | 141 | # attach loaders objects to its own thread, and run one. 142 | f = [] 143 | 144 | f.append(AsyncFunction(1,lti_1.loadTripleBatch)) 145 | f.append(AsyncFunction(1,lti_2.loadTripleBatch)) 146 | 147 | r1 = f[0].run() 148 | 149 | # return the running loader's handle so we can later fetch results. 150 | # Also return the function handles to the threads. 151 | return f, r1 152 | 153 | # ******************************************************************************************************************* 154 | def train_batch_triple(batch_toggle, f, tp, r1): 155 | """This will load the images and set them into Caffe's blobs. It calls Caffe for one solver step and returns results. 156 | 157 | We are using a rotating buffer where one thread is told to load images while we grab prior loaded images 158 | from the other. 159 | """ 160 | 161 | # Run one of the loader threads to fetch a batch of image patches and labels 162 | r2 = f[batch_toggle].run() 163 | 164 | # From the other thread, get the loaded patches and labels 165 | image_set_1, image_set_2, image_set_3, label_set, do_exit = r1.get() 166 | 167 | # Toggle which thread will run next iteration. 168 | if batch_toggle == 0: 169 | batch_toggle = 1 170 | else: 171 | batch_toggle = 0 172 | 173 | # Get images and labels into Caffe's blobs 174 | tp.solver.net.blobs['data_1'].data[:,:,:,:] = image_set_1 175 | tp.solver.net.blobs['data_2'].data[:,:,:,:] = image_set_2 176 | tp.solver.net.blobs['data_3'].data[:,:,:,:] = image_set_3 177 | tp.solver.net.blobs['label'].data[:,0,0,0] = label_set 178 | 179 | # Run the Caffe solver for one iteration. 180 | tp.solver.step(1) 181 | 182 | # Get loss and softmax output 183 | layer_loss = tp.solver.net.blobs[tp.loss_layer].data 184 | softmax_output = tp.solver.net.blobs[tp.softmax_layer].data 185 | 186 | # return outputs, labels, batch loading thread toggle and a handle to the running loader thread 187 | return layer_loss, softmax_output, label_set, batch_toggle, r2, do_exit 188 | 189 | # ******************************************************************************************************************* 190 | def test_batch_context_triple(image_file, test_transformer_1, test_transformer_2, test_transformer_3, tp): 191 | """Run test on the trained network given the testing data set. 192 | 193 | We only test on patches without rotation, flipping and aperture. This way we can measure how much adding these 194 | widgets detracts from performance on a standard right side up image. 195 | 196 | We do not run all test patches because there are way too many. So, we skip a large number of them. We use an odd 197 | skip size so we get an even sampling of all classes. 198 | """ 199 | 200 | # Set up all kinds of guys 201 | net_label = [] 202 | 203 | do_exit = False 204 | 205 | assert(image_file.__class__.__name__ == "CompactList") 206 | 207 | samples = image_file.length() 208 | for _ in range(tp.test_batch_size): 209 | net_label.append(0) 210 | 211 | print ('********************************************') 212 | print ("TESTING {} Samples From {}".format(int(samples/tp.test_skip),samples)) 213 | print ('********************************************') 214 | 215 | count_list = [] 216 | correct_list = [] 217 | 218 | for _ in range(tp.bin_num): 219 | count_list.append(0) 220 | correct_list.append(0) 221 | 222 | cv_img = [] 223 | 224 | counter = 0 225 | correct_counter = 0 226 | incorrect_counter = 0 227 | bx = 0 228 | 229 | # for all testing samples 230 | for ii in range(samples): 231 | 232 | # We will probably skip most of them with a skip offset like 199 233 | if (tp.test_skip and ii % tp.test_skip == 0) or not tp.test_skip: 234 | 235 | files, label = image_file.getFileNames(ii) 236 | i1 = tp.path_prefix + files[0] 237 | i2 = tp.path_prefix + files[1] 238 | i3 = tp.path_prefix + files[2] 239 | 240 | # When we run out of samples, finish 241 | if counter == samples: 242 | break 243 | 244 | # for each sample, load it in, crop it and apply Caffe's preprocess transformer. 245 | cv_img = cv2.imread(i1) 246 | if type(cv_img) is not np.ndarray: 247 | print (">>>>> Bad Test Image {}".format(i1)) 248 | do_exit = True 249 | break 250 | crop_img = cv_img[tp.patch_marg_1:tp.patch_marg_2,tp.patch_marg_1:tp.patch_marg_2,:] 251 | in_img_1 = test_transformer_1.preprocess('data_1', crop_img) 252 | 253 | cv_img = cv2.imread(i2) 254 | if type(cv_img) is not np.ndarray: 255 | print (">>>>> Bad Test Image {}".format(i2)) 256 | do_exit = True 257 | break 258 | crop_img = cv_img[tp.patch_marg_1:tp.patch_marg_2,tp.patch_marg_1:tp.patch_marg_2,:] 259 | in_img_2 = test_transformer_2.preprocess('data_2', crop_img) 260 | 261 | cv_img = cv2.imread(i3) 262 | if type(cv_img) is not np.ndarray: 263 | print (">>>>> Bad Test Image {}".format(i3)) 264 | do_exit = True 265 | break 266 | crop_img = cv_img[tp.patch_marg_1:tp.patch_marg_2,tp.patch_marg_1:tp.patch_marg_2,:] 267 | in_img_3 = test_transformer_3.preprocess('data_3', crop_img) 268 | 269 | # Get the sample into Caffe's blobs 270 | tp.solver.test_nets[0].blobs['data_1'].data[bx,:,:,:] = in_img_1 271 | tp.solver.test_nets[0].blobs['data_2'].data[bx,:,:,:] = in_img_2 272 | tp.solver.test_nets[0].blobs['data_3'].data[bx,:,:,:] = in_img_3 273 | tp.solver.test_nets[0].blobs['label'].data[bx] = label 274 | net_label[bx] = float(label) 275 | 276 | # once we have loaded the the buffer with a full batch, run testing 277 | if bx == tp.test_batch_size - 1 or ii == samples - 1: 278 | 279 | # pull the lever and run Caffe forward once 280 | tp.solver.test_nets[0].forward() 281 | 282 | # get the results from Caffe 283 | gt = net_label 284 | smax4 = tp.solver.test_nets[0].blobs[tp.softmax_layer].data 285 | 286 | # Process the results. This is a little brutal, but I'm too lazy to make 287 | # this look clean. Deal with it. 288 | for x in range(bx+1): 289 | 290 | # little user feedback on progress. 291 | if counter > 0: 292 | if counter%100 == 0: 293 | sys.stdout.write('.') 294 | sys.stdout.flush() 295 | 296 | if counter%1000 == 0: 297 | c = "{}".format(counter) 298 | sys.stdout.write(c) 299 | sys.stdout.flush() 300 | 301 | if counter%10000 == 0: 302 | print (' ') 303 | 304 | # check if the returned label matches the ground truth. 305 | max_bin = -1 306 | max_val = -1 307 | bin_count = 0 308 | for sbin in smax4[x]: 309 | if sbin > max_val: 310 | max_val = sbin 311 | max_bin = bin_count # my class as a bin construct, not same as bin_size 312 | bin_count += 1 313 | 314 | gt_x = int(gt[x]) # int ground truth 315 | 316 | count_list[gt_x] += 1 317 | 318 | # is the ground truth the same as the network's returned value? 319 | if gt[x] == max_bin: 320 | correct_counter += 1 321 | correct_list[gt_x] += 1 322 | else: 323 | incorrect_counter += 1 324 | 325 | counter = counter + 1 326 | 327 | # When we run out of samples, finish 328 | if counter == samples: 329 | break 330 | 331 | bx = 0 332 | cv_img = [] 333 | else: 334 | bx += 1 335 | 336 | # Compute percent accuracy. 337 | total_samples = correct_counter + incorrect_counter 338 | correct_p = float(correct_counter)/float(total_samples) 339 | 340 | print ('********************************************') 341 | print (' ') 342 | print ("TEST : count {} percent correct {}".format(total_samples,correct_p*100.0)) 343 | print (' ') 344 | print ('********************************************') 345 | 346 | return correct_p, do_exit 347 | 348 | # ******************************************************************************************************************* 349 | def save_plot_data(file_name, x, y): 350 | """Plot data is preserved between runs so we don't mess up our graphs if we exit or crash 351 | """ 352 | print("save plot data as: {}.npz".format(file_name)) 353 | np.savez_compressed(file_name, x = x, y = y) 354 | 355 | # ******************************************************************************************************************* 356 | def load_plot_data(file_name): 357 | """Plot data is preserved between runs so we don't mess up our graphs if we exit or crash 358 | """ 359 | print("load plot data as: {}".format(file_name)) 360 | _ = check_file(file_name) 361 | npz_file = np.load(file_name) 362 | 363 | x = npz_file['x'] 364 | y = npz_file['y'] 365 | 366 | return x, y 367 | 368 | # ******************************************************************************************************************* 369 | def mr_plot(layer_loss, i, fig_prop, fig=False, ax=False, fig_file='', title='', tp=None): 370 | """Plot the accuracy on our testing data. 371 | 372 | This function does a few notable things. We have to hang onto the figure handle because the grapher package 373 | does not do a good job of collecting stale handles. They just seem to hang around slowing things down. We 374 | also save and restore the graph so we can keep a nice clean graph even if we exit training or the program crashes. 375 | 376 | """ 377 | assert(type(tp) != types.NoneType) # needs to be set 378 | assert(tp.test_iters > 0) # This needs to be set in the code, copied over 379 | assert(i >= 0) 380 | 381 | # Should we init or load graph data? 382 | if fig_file is not '' and os.path.isfile(fig_file + '.npz') and i > tp.test_iters: 383 | old_x, old_y = load_plot_data(fig_file + '.npz') 384 | else: 385 | print('Init new plot data') 386 | old_x = np.zeros(0) 387 | old_y = np.zeros(0) 388 | 389 | # Delete any i "newer" than the current one for continuity 390 | # This way if we start over from an earlier state, we have a clean graph 391 | # by deleting graph data that would essentially be from the future. 392 | new_x = np.empty((0)) 393 | new_y = np.empty((0)) 394 | 395 | for t in range(old_x.shape[0]): 396 | if old_x[t] < i: 397 | new_x = np.append(new_x, old_x[t]) 398 | new_y = np.append(new_y, old_y[t]) 399 | 400 | new_x = np.append(new_x, i) 401 | new_y = np.append(new_y, layer_loss) 402 | 403 | if fig_file is not '': 404 | save_plot_data(fig_file, new_x, new_y) 405 | 406 | # pyplot crashes without recovery such as exception handling 407 | # if it cannot make an X connection. 408 | if check_X_is_running(): 409 | 410 | # only create a figure once. 411 | if not fig: 412 | fig = pylab.figure() 413 | ax = fig.add_subplot(111) 414 | 415 | pylab.ion() 416 | 417 | ax.set_title(title) 418 | ax.plot(new_x, new_y, fig_prop) 419 | pylab.draw() 420 | pylab.show() 421 | 422 | # Save our graph and its data 423 | if fig_file is not '': 424 | fig.savefig(fig_file) 425 | 426 | # return just the figure handles. Note that the data is stored on drive 427 | return fig, ax 428 | 429 | # ******************************************************************************************************************* 430 | def vis_square(in_data, title='', fig=False, ax=False, transpose_data=True, fig_file=''): 431 | """Take an array of shape (n, height, width) or (n, height, width, 3) 432 | and visualize each (height, width) thing in a grid of size approx. sqrt(n) by sqrt(n) 433 | 434 | This is primarily used to visualize the first layer features. 435 | """ 436 | 437 | if transpose_data: 438 | data = in_data.transpose(0, 2, 3, 1) 439 | else: 440 | data = in_data 441 | 442 | # normalize data for display 443 | data = (data - data.min()) / (data.max() - data.min()) 444 | 445 | # force the number of filters to be square 446 | n = int(np.ceil(np.sqrt(data.shape[0]))) 447 | padding = (((0, n ** 2 - data.shape[0]), 448 | (0, 1), (0, 1)) # add some space between filters 449 | + ((0, 0),) * (data.ndim - 3)) # don't pad the last dimension (if there is one) 450 | data = np.pad(data, padding, mode='constant', constant_values=1) # pad with ones (white) 451 | 452 | # tile the filters into an image 453 | data = data.reshape((n, n) + data.shape[1:]).transpose((0, 2, 1, 3) + tuple(range(4, data.ndim + 1))) 454 | data = data.reshape((n * data.shape[1], n * data.shape[3]) + data.shape[4:]) 455 | 456 | #Only create the figure object once 457 | if not fig: 458 | fig = plt.figure() 459 | ax = fig.add_subplot(111) 460 | 461 | plt.ion() 462 | 463 | ax.cla() 464 | ax.set_title(title) 465 | ax.imshow(data); ax.axis('off') 466 | plt.draw() 467 | plt.show() 468 | 469 | # Save the figure to drive 470 | if fig_file is not '': 471 | fig.savefig(fig_file) 472 | 473 | # return just the figure handles 474 | return fig, ax 475 | 476 | # ******************************************************************************************************************* 477 | def create_solver_file(solver_file_name, network_file_name, snapshot_path, condition, use_batch_norm_sched, 478 | use_nccl, snaphot_interval): 479 | 480 | print("**********************************************************************************************************************") 481 | print("NOTE: Creating Solver File: " + solver_file_name) 482 | print("**********************************************************************************************************************") 483 | 484 | sf = open(solver_file_name,'w') 485 | 486 | net_line = "net: \"" + network_file_name + "\"\n" 487 | sf.write(net_line) 488 | 489 | snapshot_line = "snapshot_prefix: \"" + snapshot_path + "/train_val_" + condition + "\"\n" 490 | sf.write(snapshot_line) 491 | 492 | if use_batch_norm_sched: 493 | default = "\ 494 | test_iter: 20000\n\ 495 | test_interval: 1000000000\n\ 496 | test_initialization: false\n\ 497 | display: 20\n\ 498 | average_loss: 40\n\ 499 | lr_policy: \"step\"\n\ 500 | gamma: 0.1\n\ 501 | stepsize: 300000\n\ 502 | max_iter: 750000\n\ 503 | base_lr: 0.01\n\ 504 | momentum: 0.9\n\ 505 | weight_decay: 0.0002\n\ 506 | solver_mode: GPU\n\ 507 | random_seed: 34234562302122\n" 508 | else: 509 | default = "\ 510 | test_iter: 20000\n\ 511 | test_interval: 1000000000\n\ 512 | test_initialization: false\n\ 513 | display: 20\n\ 514 | average_loss: 40\n\ 515 | lr_policy: \"step\"\n\ 516 | gamma: 0.96806001063\n\ 517 | stepsize: 10000\n\ 518 | max_iter: 1500000\n\ 519 | base_lr: 0.00666\n\ 520 | momentum: 0.9\n\ 521 | weight_decay: 0.0002\n\ 522 | solver_mode: GPU\n\ 523 | random_seed: 34234562302122\n" 524 | 525 | sf.write(default) 526 | 527 | sf.write("snapshot: {}\n".format(snaphot_interval)) 528 | 529 | if use_nccl: 530 | sf.write("layer_wise_reduce: false\n") 531 | 532 | sf.close() 533 | 534 | # ******************************************************************************************************************* 535 | def create_msub(proj_snapshot_base, log_dir, profile, condition): 536 | 537 | log_file = log_dir + "/RunTripleContext." + condition + ".log" 538 | msub_file = proj_snapshot_base + "/slurm.msub" 539 | 540 | File = open(msub_file,'w') 541 | 542 | preamble = "\ 543 | #!/bin/bash\n\ 544 | #MSUB -l nodes=1 # use 1 node\n\ 545 | #MSUB -l walltime=12:00:00 # ask for 12 hours\n\ 546 | #MSUB -q gpgpu # use the gpgpu partition\n\ 547 | #MSUB -A hpcdl # use my account\n" 548 | File.write(preamble) 549 | 550 | job_line = "#MSUB -N {} # user defined job name\n".format(condition) 551 | File.write(job_line) 552 | log_line = "#MSUB -o {} # user defined job log file\n".format(log_file) 553 | File.write(log_line) 554 | 555 | more_lines = "\ 556 | # print message that a new run is starting\n\ 557 | echo \"Starting new run: $SLURM_JOBID\"\n\ 558 | date\n\ 559 | \n\ 560 | # to create a chain of dependent jobs (optional)\n\ 561 | echo \"Submitting dependent job\"\n" 562 | File.write(more_lines) 563 | 564 | depend_line = "msub -l depend=$SLURM_JOBID {}\n".format(msub_file) 565 | File.write(depend_line) 566 | profile_line = "source {}\n".format(profile) 567 | File.write(profile_line) 568 | run_line = "python {}/python/RunTripleContextTrain.py\n\n".format(proj_snapshot_base) 569 | File.write(run_line) 570 | 571 | print("**********************************************************************************************************************") 572 | print("NOTE: Run me stand alone as: python {}/python/RunTripleContextTrain.py".format(proj_snapshot_base)) 573 | print("OR: Run me as a batch as: msub --slurm {}".format(msub_file)) 574 | print("**********************************************************************************************************************") 575 | 576 | 577 | File.close() 578 | 579 | # ******************************************************************************************************************* 580 | def save_project_state(curr_py_file, state_py_dir): 581 | 582 | assert(curr_py_file != state_py_dir) 583 | 584 | lparts = string.split(curr_py_file,'/') 585 | 586 | root_dir = "/" 587 | 588 | for i in range(len(lparts) - 1): 589 | root_dir += (lparts[i] + '/') 590 | 591 | if not os.path.isdir(state_py_dir): 592 | os.mkdir(state_py_dir) 593 | 594 | cp_command = "cp -R " + root_dir + " " + state_py_dir 595 | #print("Backing up project state to {} ".format(state_py_dir)) 596 | #print("Command: {}".format(cp_command)) 597 | os.system(cp_command) 598 | 599 | # ******************************************************************************************************************* 600 | def instantiate_slurm(proj_snapshot_dir, network_file_name, 601 | condition, log_dir, profile, snaphot_interval, 602 | use_batch_norm_sched, 603 | rank, mp_cond, proc_comm, init_new = False): 604 | 605 | snapshot_file = "" 606 | 607 | proj_snapshot_base = proj_snapshot_dir + '/slurm.' + condition + '/' 608 | caffe_snapshot_path = proj_snapshot_base + '/caffe_model/' 609 | solver_file = proj_snapshot_base + "solver.prototxt" 610 | caffe_snapshot_prefix = caffe_snapshot_path + "/train_val_" + condition 611 | new_network_file = proj_snapshot_base + "train_val.prototxt" 612 | 613 | # IF YOU START IT WRONG, DELETE THIS FILE 614 | if os.path.isfile(solver_file) and init_new == False: 615 | 616 | my_file = "{}_iter*.solverstate".format(caffe_snapshot_prefix) 617 | 618 | call = "ls -t1 " + my_file 619 | load_me = True 620 | 621 | try: 622 | subprocess.check_call(call, shell=True) 623 | except Exception as e: 624 | print('Cannot find past solverstate in line {}'.format(sys.exc_info()[-1].tb_lineno), type(e).__name__, e) 625 | load_me = False 626 | 627 | if rank == 0: 628 | print("**********************************************************************************************************************") 629 | print("NOTE: Using project folder: {}".format(proj_snapshot_base)) 630 | print("**********************************************************************************************************************") 631 | 632 | if load_me: 633 | files = subprocess.check_output(call,shell=True) 634 | 635 | new_str = string.split(files,'\n') 636 | 637 | snapshot_file = new_str[0] 638 | 639 | if rank == 0: 640 | print("**********************************************************************************************************************") 641 | print("NOTE: AUTO LOAD: {}".format(snapshot_file)) 642 | print("**********************************************************************************************************************") 643 | else: 644 | if rank == 0: 645 | print("**********************************************************************************************************************") 646 | print("NOTE: AUTO LOAD: NEW ... CANNOT FIND {}".format(my_file)) 647 | print("**********************************************************************************************************************") 648 | 649 | do_exit = False 650 | # Create a new project and exit 651 | else: 652 | 653 | with mp_cond: 654 | 655 | if check_file(network_file_name,rank) == 0: 656 | do_exit = True 657 | else: 658 | if rank == 0: 659 | print("**********************************************************************************************************************") 660 | print("NOTE: Creating project folder: {}".format(proj_snapshot_base)) 661 | print("**********************************************************************************************************************") 662 | print("**********************************************************************************************************************") 663 | print("NOTE: Copying network to project as: " + new_network_file) 664 | print("**********************************************************************************************************************") 665 | 666 | if os.path.exists(proj_snapshot_dir): 667 | pass 668 | else: 669 | print("**********************************************************************************************************************") 670 | print("NOTE: Creating project directory: " + proj_snapshot_dir) 671 | print("**********************************************************************************************************************") 672 | 673 | os.mkdir(proj_snapshot_dir) 674 | 675 | # copy this project to its own folder so we preserve it in this state 676 | save_project_state(__file__, proj_snapshot_base) 677 | # make the solver file 678 | create_solver_file(solver_file, new_network_file, caffe_snapshot_path, condition, use_batch_norm_sched, 679 | use_nccl=True, snaphot_interval=snaphot_interval) 680 | # copy the network file into the package 681 | shutil.copy(network_file_name, new_network_file) 682 | # make the slurm batch msub file 683 | create_msub(proj_snapshot_base, log_dir, profile, condition) 684 | # Create the place to put our snapshots 685 | os.mkdir(caffe_snapshot_path) 686 | 687 | proc_comm[1] = True 688 | mp_cond.notify_all() 689 | else: 690 | if proc_comm[1] == False: 691 | mp_cond.wait() 692 | 693 | do_exit = True 694 | 695 | return solver_file, snapshot_file, do_exit 696 | 697 | 698 | 699 | 700 | 701 | -------------------------------------------------------------------------------- /python/README.md: -------------------------------------------------------------------------------- 1 | Pythons scripts 2 | -------------------------------------------------------------------------------- /python/RunTripleContextTrain.py: -------------------------------------------------------------------------------- 1 | import PrefetchTrain 2 | import signal 3 | import caffe 4 | import numpy as np 5 | import time 6 | import multiprocessing 7 | import NumpyFileList 8 | import os 9 | from multiprocessing import Process 10 | 11 | MP_COND = multiprocessing.Condition() 12 | TRAIN_LOOP = False 13 | FINISH_EXIT = False 14 | 15 | # ******************************************************************************************************************* 16 | def ctrl_c_handler(signum, frame): 17 | """Handle ctrl-c save the current state and exit. 18 | 19 | If Caffe has not yet started, we will simply exit. Otherwise, we ask the main loop to 20 | exit after the completion of the current iteration. Each caffe loop will pass through this function 21 | on sigint. The loader processes have a dummy function handle (in another file) so they don't hose since sigint is sent 22 | to all processes. 23 | """ 24 | global MP_COND 25 | global TRAIN_LOOP 26 | global FINISH_EXIT 27 | 28 | # probably don't need a lock, but lets keep clean 29 | with MP_COND: 30 | 31 | if rank == 0: 32 | print("Quade, start the reactor. Free Mars ...") 33 | 34 | if TRAIN_LOOP == True: 35 | FINISH_EXIT = True 36 | else: 37 | exit() 38 | 39 | # ******************************************************************************************************************* 40 | # ******************************************************************************************************************* 41 | # ******************************************************************************************************************* 42 | 43 | # We set up N processes for each GPU. 44 | # Change this for how many GPUs you have and want to use. 45 | gpus = [0,1,2,3] 46 | 47 | def caffe_loop(gpus, uid, rank, avg_guys, proc_comm): 48 | """Main loop for each GPU process. 49 | 50 | At the bottom is the main process which creates each GPU process (this guy). We set up all the parameters here and 51 | then run the Caffe loop. NCCL links each GPU process implicitly. So, you will not see semaphores or other similars, 52 | but NCCL is doing this in the background when Caffe is called. So for example, all processes will sync up when 53 | Caffe step is called (in PrefetchTrain). 54 | """ 55 | global MP_COND 56 | global TRAIN_LOOP 57 | global FINISH_EXIT 58 | 59 | # Where is this project located? 60 | project_home = '/home/mundhenk/selfsupervised/' 61 | # Path to training image set 62 | path_prefix = '/home/mundhenk/images/patches_84h_110x110_13x13-blur-ab_compact/' 63 | 64 | # Condition is a label used for graphing, display purposes and saving snap shots 65 | # This can be any valid string, but must by file name friendly. 66 | condition = 'my_awesome_selfsupervised_run' 67 | # Base for where a lot of files are kept or go such as network files 68 | caffe_data_dir = project_home + '/caffe_data/' 69 | # Where to save figures 70 | fig_root = project_home + '/figures/' 71 | # where to save this project 72 | proj_snapshot_dir = project_home + '/py_proj/' 73 | # where to save moab files 74 | log_dir = project_home + '/moab_output/' 75 | # extra profile to run to set enviroment on node 76 | profile = project_home + '/scripts/profile.sh' 77 | # Your caffe network prototxt file 78 | network_file_name = caffe_data_dir + '/train_val_AlexNet-Custom_triple.prototxt' 79 | 80 | # Name of a caffemodel to use to initialize our weights from 81 | weight_file = '' 82 | 83 | # Alexnet layer names from the network prototxt file 84 | start_layer_vis = 'conv1' # Visualize This layer 85 | softmax_layer = 'softmax_plain' # For testing, we need this guy 86 | loss_layer = 'loss' # Your loss layer 87 | # Are we using a batch normalized network schedule. For plain CaffeNet, set to False 88 | use_batch_norm_sched = True 89 | # Re-init project files? 90 | init_new = False 91 | 92 | # ImageNet mean gray 93 | image_mean = [104.0, 117.0, 123.0] # ImageNET 94 | # Given a 110x110 size patch, what are the range of scales we can resize it to before cropping out 96x96? 95 | ra_max_size = 128 # Goes to a max size corresponding to an image of 448x448 96 | ra_min_size = 96 # Goes to a min size corresponding to an image of 171x171 97 | # Training batch size. The script will auto resize this when using more than one GPU 98 | train_batch_size = 128 99 | # Testing batch size. 100 | test_batch_size = 20 101 | # How many classes you will test over. 102 | bin_num = 20 103 | # The actual size of the patchs (96x96) 104 | patch_size = 96 105 | # Tells us where to center crop during testing 106 | patch_marg_1 = 7 107 | patch_marg_2 = 110 108 | # How many iters should we wait to display info? 109 | display_iters = 20 110 | # How many iters should we wait to test the network 111 | test_iters = 5000 112 | # Smoothing parameter over displayed loss 113 | loss_lambda = 20 114 | # Stride over the testing data set so we only use a subset. 115 | test_skip = 199 116 | # How often to snapshot the solver state 117 | snaphot_interval = 5000 118 | 119 | # training and testing list files 120 | test_list_file = path_prefix + 'val/val_list.nfl.npz' 121 | train_list_file = path_prefix + 'train/train_list.nfl.npz' 122 | 123 | # ******************************************************************************************************************* 124 | # ******************************************************************************************************************* 125 | # Dont edit after here 126 | # ******************************************************************************************************************* 127 | # ******************************************************************************************************************* 128 | 129 | # check to make sure files and dirs exist 130 | if PrefetchTrain.check_file(train_list_file,rank) == 0: return 131 | if PrefetchTrain.check_file(test_list_file,rank) == 0: return 132 | if PrefetchTrain.check_file(profile,rank) == 0: return 133 | 134 | if PrefetchTrain.check_dir(path_prefix,rank) == 0: return 135 | if PrefetchTrain.check_dir(project_home,rank) == 0: return 136 | if PrefetchTrain.check_dir(caffe_data_dir,rank) == 0: return 137 | 138 | # Create some directories if needed 139 | PrefetchTrain.check_create_dir(log_dir,rank) 140 | PrefetchTrain.check_create_dir(fig_root,rank) 141 | 142 | solver_file_name, snapshot_file, do_exit = PrefetchTrain.instantiate_slurm(proj_snapshot_dir, network_file_name, 143 | condition, log_dir, profile, snaphot_interval, 144 | use_batch_norm_sched, 145 | rank, MP_COND, proc_comm, init_new = init_new) 146 | 147 | # We just init-ed the whole thing. Now we exit 148 | if do_exit: 149 | return 150 | 151 | fig_model = condition 152 | fig_name_err = fig_root + fig_model + '.err.png' 153 | fig_name_sqr = fig_root + fig_model + '.sqr.jpg' 154 | fig_prop = 'b--' 155 | 156 | ''' 157 | We will now configure a bunch of things before we run the main loop. NCCL needs some things to be in a 158 | particular order. Some tasks are reserved for a single process alone. These always run on the first GPU 159 | in the list. 160 | ''' 161 | 162 | batch_toggle = 0 163 | 164 | print('GPU:{} Set Caffe Device'.format(gpus[rank])) 165 | 166 | print('GPU:{} Set Device'.format(gpus[rank])) 167 | caffe.set_device(gpus[rank]) ### THIS ALWAYS HAS TO COME BEFORE OTHER CAFFE SETTERS!!! 168 | 169 | # Set up multi processing 170 | if uid: 171 | print('GPU:{} Set Solver Count to {}'.format(gpus[rank],len(gpus))) 172 | caffe.set_solver_count(len(gpus)) 173 | print('GPU:{} Set Solver Rank to {}'.format(gpus[rank],rank)) 174 | caffe.set_solver_rank(rank) 175 | print('GPU:{} Set Multiprocess'.format(gpus[rank])) 176 | caffe.set_multiprocess(True) 177 | 178 | # Use GPU like a civilized human being 179 | print('GPU:{} Set to Use GPU'.format(gpus[rank])) 180 | caffe.set_mode_gpu() 181 | 182 | # resize the training batch size by number of GPU's we are using 183 | train_batch_size /= len(gpus) 184 | 185 | print('GPU:{} New Train Batch Size {}'.format(gpus[rank],train_batch_size)) 186 | 187 | print('GPU:{} Load Network and Files'.format(gpus[rank])) 188 | print("GPU:{} Solver: {}".format(gpus[rank],solver_file_name)) 189 | 190 | # Create the Caffe solver and read the solver file so we can use some of its parameters 191 | solver = caffe.SGDSolver(solver_file_name) 192 | solver_params = PrefetchTrain.read_proto_solver_file(solver_file_name) 193 | max_iters = solver_params.max_iter 194 | 195 | print("GPU:{} Adjusted Batch Size For Each GPU : {}".format(gpus[rank],train_batch_size)) 196 | 197 | # This script does not support iters. 198 | assert(solver_params.iter_size < 2) 199 | 200 | # Open our training and testing lists, but don't do anything with them yet. 201 | print("GPU:{} Loading: {}".format(gpus[rank],test_list_file)) 202 | if rank == 0: 203 | test_list_in = open(test_list_file) 204 | print("GPU:{} Loading: {}".format(gpus[rank],train_list_file)) 205 | train_list_in = open(train_list_file) 206 | 207 | # Do we have a weight file? If so, use it. 208 | if weight_file != '': 209 | print('GPU:{} Loading weight file: {} '.format(gpus[rank],weight_file)) 210 | solver.net.copy_from(weight_file) 211 | 212 | # Do we have a snapshot file? If so, use it. 213 | if snapshot_file != '': 214 | print('GPU:{} Loading Snapshot file: {}'.format(gpus[rank],snapshot_file)) 215 | solver.restore(snapshot_file) 216 | 217 | if uid: 218 | # Create NCCL callback 219 | nccl = caffe.NCCL(solver, uid) 220 | nccl.bcast() 221 | solver.add_callback(nccl) 222 | 223 | if solver.param.layer_wise_reduce: 224 | solver.net.after_backward(nccl) 225 | 226 | print("GPU:{} Network and Files Loaded".format(gpus[rank])) 227 | 228 | # reshape our training blobs 229 | solver.net.blobs['data_1'].reshape(train_batch_size,3,patch_size,patch_size) 230 | solver.net.blobs['data_2'].reshape(train_batch_size,3,patch_size,patch_size) 231 | solver.net.blobs['data_3'].reshape(train_batch_size,3,patch_size,patch_size) 232 | solver.net.blobs['label'].reshape(train_batch_size,1,1,1) 233 | 234 | print ("GPU:{} Network Train Blobs Set".format(gpus[rank])) 235 | 236 | # reshape testing blobs, but only process will do this. 237 | if rank == 0: 238 | solver.test_nets[0].blobs['data_1'].reshape(test_batch_size,3,patch_size,patch_size) 239 | solver.test_nets[0].blobs['data_2'].reshape(test_batch_size,3,patch_size,patch_size) 240 | solver.test_nets[0].blobs['data_3'].reshape(test_batch_size,3,patch_size,patch_size) 241 | solver.test_nets[0].blobs['label'].reshape(test_batch_size,1,1,1) 242 | 243 | print ("GPU:{} Network Test Blobs Set".format(gpus[rank])) 244 | 245 | test_transformer_1 = caffe.io.Transformer({'data_1': solver.test_nets[0].blobs['data_1'].data.shape}) 246 | test_transformer_1.set_transpose('data_1', (2,0,1)) 247 | test_transformer_1.set_mean('data_1', np.float32(image_mean)) # mean pixel 248 | test_transformer_2 = caffe.io.Transformer({'data_2': solver.test_nets[0].blobs['data_2'].data.shape}) 249 | test_transformer_2.set_transpose('data_2', (2,0,1)) 250 | test_transformer_2.set_mean('data_2', np.float32(image_mean)) # mean pixel 251 | test_transformer_3 = caffe.io.Transformer({'data_3': solver.test_nets[0].blobs['data_3'].data.shape}) 252 | test_transformer_3.set_transpose('data_3', (2,0,1)) 253 | test_transformer_3.set_mean('data_3', np.float32(image_mean)) # mean pixel 254 | 255 | print ("GPU:{} Network Test Transformer Set".format(gpus[rank])) 256 | 257 | # Set up our training parameters object 258 | tp = PrefetchTrain.TrainParams(solver, patch_size, patch_marg_1, patch_marg_2, train_batch_size, test_batch_size, 259 | bin_num, image_mean, loss_layer, softmax_layer) 260 | 261 | # copy a few more items over into our training parameters object 262 | tp.path_prefix = path_prefix 263 | tp.test_skip = test_skip 264 | tp.test_iters = test_iters 265 | tp.ra_patch_size = patch_size 266 | tp.ra_max_size = ra_max_size 267 | tp.ra_min_size = ra_min_size 268 | 269 | # Process and load our training data set 270 | print ("GPU:{} Parse nfl context train list".format(gpus[rank])) 271 | NFL = NumpyFileList.CompactList() 272 | NFL.load(train_list_in) 273 | train_image_file = NFL 274 | 275 | train_list_in.close() 276 | 277 | # process and load our testing data set. Only one GPU will do this. 278 | if rank == 0: 279 | print ("GPU:{} Parse nfl context test list".format(gpus[rank])) 280 | NFL = NumpyFileList.CompactList() 281 | NFL.load(test_list_in) 282 | test_image_file = NFL 283 | 284 | test_list_in.close() 285 | 286 | print ("GPU:{} Lists Parsed".format(gpus[rank])) 287 | 288 | # Once we launch the threads, we need to exit gently 289 | TRAIN_LOOP = True 290 | 291 | # Init the two main loader threads and return handles 292 | f, r = PrefetchTrain.train_batch_triple_init(train_image_file, tp) 293 | 294 | # set some things we need to set. 295 | loss_avg = 0.0 296 | cstart = 0.0 297 | 298 | print("GPU:{} PREFETCH TRAIN".format(gpus[rank])) 299 | 300 | start_iter = True 301 | layer_loss = 0 302 | 303 | vis_fig = False 304 | vis_ax = False 305 | 306 | plot_fig = False 307 | plot_ax = False 308 | 309 | print("GPU:{} START LOOP".format(gpus[rank])) 310 | 311 | ''' 312 | This is our main training loop. From here on out we will stay in this loop until exit. Most of the code here is for 313 | display and control. train_batch_triple is the only thing that needs to be called to train the network. 314 | ''' 315 | while True: 316 | 317 | i = int(solver.iter) 318 | display = False 319 | 320 | # Do we compute display timing data this iteration? 321 | if (i % display_iters == 0 or start_iter): 322 | cend = time.time() 323 | timer = cend - cstart 324 | cstart = cend 325 | 326 | # It's annoying and useless to print stats like this on the first iter 327 | if not start_iter: 328 | t = timer/float(display_iters) 329 | # Only once process prints this stuff out. 330 | if rank == 0: 331 | print("GPU:{} ({}) {} ".format(gpus[rank], i, condition)) 332 | print("GPU:{} Average TIME {}".format(gpus[rank], t)) 333 | 334 | display = True 335 | 336 | # run the actual training step on Caffe. Get back a run handle r and performance data 337 | layer_loss, _, _, batch_toggle, r, do_exit = PrefetchTrain.train_batch_triple(batch_toggle, f, tp, r) 338 | 339 | if do_exit == True: proc_comm[2] = True 340 | 341 | # compute a running average over loss 342 | if start_iter: 343 | loss_avg = layer_loss 344 | else: 345 | loss_avg = (layer_loss + loss_avg*loss_lambda) / (1.0 + loss_lambda) 346 | 347 | avg_guys[rank] = loss_avg 348 | 349 | # Update the figure showing the first layer filters. Only one process does this. 350 | if display and rank == 0: 351 | # check if we have an x server connection to output to 352 | if PrefetchTrain.check_X_is_running(): 353 | vis_fig, vis_ax = PrefetchTrain.vis_square(solver.net.params[start_layer_vis][0].data, condition, vis_fig, vis_ax, True, fig_name_sqr) 354 | 355 | # when we reach the right iteration, we will test the network and plot the performance 356 | if (rank == 0) or i == int(max_iters): 357 | 358 | if (i != 0 and i % test_iters == 0) or i == int(max_iters): 359 | print ("TESTING") 360 | # Get weights over 361 | solver.test_nets[0].share_with(solver.net) 362 | 363 | # Run the test network 364 | correct_p, do_exit = PrefetchTrain.test_batch_context_triple(test_image_file, 365 | test_transformer_1, test_transformer_2, test_transformer_3, tp) 366 | 367 | # Plot the results of the test. 368 | plot_fig,plot_ax = PrefetchTrain.mr_plot(correct_p, i, fig_prop, plot_fig, plot_ax, fig_name_err, condition, tp = tp) 369 | 370 | if do_exit == True: proc_comm[2] = True 371 | 372 | # one process will collect and display loss over all GPU processes. 373 | if display: 374 | #print("GPU:{} Average LOSS {}".format(gpus[rank],loss_avg)) 375 | if rank == 0: 376 | avg = 0.0 377 | for ar in avg_guys: 378 | avg += ar 379 | 380 | avg /= len(avg_guys) 381 | 382 | print("GPU:{} ALL Average LOSS {}".format(gpus[rank],ar)) 383 | 384 | # Exit when maximum iteration is reached. 385 | if i == int(max_iters): 386 | print ("GPU:{} Reaches Maxed Iters".format(gpus[rank])) 387 | break 388 | 389 | # Exit on ctrl-c 390 | if FINISH_EXIT: 391 | print ("GPU:{} Got CTRL-C. Exiting ...".format(gpus[rank])) 392 | break 393 | 394 | if proc_comm[2] == True: 395 | print ("GPU:{} Got ERROR. Exiting ...".format(gpus[rank])) 396 | return 397 | 398 | start_iter = False 399 | 400 | # When we exit, we always save the current state. Only one process does this. 401 | if rank == 0: 402 | # just in case 403 | solver.snapshot() 404 | 405 | print ('done : Saving and exiting ...') 406 | 407 | 408 | # ******************************************************************************************************************* 409 | # ******************************************************************************************************************* 410 | # ******************************************************************************************************************* 411 | # *********** MAIN THREAD 412 | # ******************************************************************************************************************* 413 | # ******************************************************************************************************************* 414 | # ******************************************************************************************************************* 415 | 416 | # We start by attaching a signal handler for ctrl-c (SIGINT) so we can exit and save state whenever. 417 | signal.signal(signal.SIGINT, ctrl_c_handler) 418 | 419 | procs = [] 420 | 421 | # Get an ID to share between NCCL/GPU processes 422 | # If we set this to False, then we will not use NCCL at all 423 | if len(gpus) > 1: 424 | uid = caffe.NCCL.new_uid() 425 | else: 426 | uid = False 427 | 428 | # This passes some performance data between GPU processes 429 | avg_guys = multiprocessing.Array('f',len(gpus),lock=False) 430 | # So we can pass messages between processes etc. 431 | proc_comm = multiprocessing.Array('i',3,lock=True) 432 | 433 | proc_comm[0] = False 434 | proc_comm[1] = False 435 | proc_comm[2] = False 436 | 437 | # Set up each process to run caffe_loop 438 | for rank in range(len(gpus)): 439 | 440 | proc = Process(target=caffe_loop, args=(gpus, uid, rank, avg_guys, proc_comm)) 441 | 442 | proc.start() 443 | procs.append(proc) 444 | 445 | # Start the processes. 446 | for proc in procs: 447 | proc.join() 448 | 449 | -------------------------------------------------------------------------------- /scripts/README.txt: -------------------------------------------------------------------------------- 1 | You probably won't use this directory. 2 | -------------------------------------------------------------------------------- /tests/CUB/dataset.txt: -------------------------------------------------------------------------------- 1 | https://gdo-datasci.llnl.gov/selfsupervised/download/image_sets/cub/basic_set_256x256.tgz 2 | -------------------------------------------------------------------------------- /tests/CUB/solver_AlexNet.prototxt: -------------------------------------------------------------------------------- 1 | net: 2 | test_iter: 3 | test_interval: 5000 4 | test_initialization: false 5 | 6 | base_lr: 0.01 7 | lr_policy: "poly" 8 | power: 0.5 9 | max_iter: 100000 10 | momentum: 0.9 11 | weight_decay: 0.0002 12 | 13 | 14 | display: 20 15 | 16 | snapshot: 5000 17 | snapshot_prefix: 18 | 19 | solver_mode: GPU 20 | 21 | random_seed: 34234562302122 22 | 23 | iter_size: 1 -------------------------------------------------------------------------------- /tests/CompCars/dataset.txt: -------------------------------------------------------------------------------- 1 | https://gdo-datasci.llnl.gov/selfsupervised/download/image_sets/compcars/resize_crop_24_permutes.tgz 2 | https://gdo-datasci.llnl.gov/selfsupervised/download/image_sets/compcars/resize_int_crop.tgz 3 | -------------------------------------------------------------------------------- /tests/CompCars/solver_AlexNet.prototxt: -------------------------------------------------------------------------------- 1 | net: 2 | test_iter: 3 | test_interval: 5000 4 | test_initialization: false 5 | 6 | base_lr: 0.01 7 | lr_policy: "poly" 8 | power: 0.5 9 | max_iter: 100000 10 | momentum: 0.9 11 | weight_decay: 0.0002 12 | 13 | 14 | display: 20 15 | 16 | snapshot: 5000 17 | snapshot_prefix: 18 | 19 | solver_mode: GPU 20 | 21 | random_seed: 34234562302122 22 | 23 | iter_size: 1 -------------------------------------------------------------------------------- /tests/README.md: -------------------------------------------------------------------------------- 1 | Files for running validation. --------------------------------------------------------------------------------