├── LICENSE ├── Makefile ├── README.md ├── corpus.cpp ├── corpus.h ├── main.cpp ├── opt.cpp ├── opt.h ├── sample-data ├── images.tgz ├── uiuc-sports-annotations.txt └── uiuc-sports-info.txt ├── settings.h ├── settings.txt ├── slda.cpp ├── slda.h ├── utils.cpp └── utils.h /LICENSE: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 2, June 1991 3 | 4 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., 5 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 6 | Everyone is permitted to copy and distribute verbatim copies 7 | of this license document, but changing it is not allowed. 8 | 9 | Preamble 10 | 11 | The licenses for most software are designed to take away your 12 | freedom to share and change it. By contrast, the GNU General Public 13 | License is intended to guarantee your freedom to share and change free 14 | software--to make sure the software is free for all its users. This 15 | General Public License applies to most of the Free Software 16 | Foundation's software and to any other program whose authors commit to 17 | using it. (Some other Free Software Foundation software is covered by 18 | the GNU Lesser General Public License instead.) You can apply it to 19 | your programs, too. 20 | 21 | When we speak of free software, we are referring to freedom, not 22 | price. Our General Public Licenses are designed to make sure that you 23 | have the freedom to distribute copies of free software (and charge for 24 | this service if you wish), that you receive source code or can get it 25 | if you want it, that you can change the software or use pieces of it 26 | in new free programs; and that you know you can do these things. 27 | 28 | To protect your rights, we need to make restrictions that forbid 29 | anyone to deny you these rights or to ask you to surrender the rights. 30 | These restrictions translate to certain responsibilities for you if you 31 | distribute copies of the software, or if you modify it. 32 | 33 | For example, if you distribute copies of such a program, whether 34 | gratis or for a fee, you must give the recipients all the rights that 35 | you have. You must make sure that they, too, receive or can get the 36 | source code. And you must show them these terms so they know their 37 | rights. 38 | 39 | We protect your rights with two steps: (1) copyright the software, and 40 | (2) offer you this license which gives you legal permission to copy, 41 | distribute and/or modify the software. 42 | 43 | Also, for each author's protection and ours, we want to make certain 44 | that everyone understands that there is no warranty for this free 45 | software. If the software is modified by someone else and passed on, we 46 | want its recipients to know that what they have is not the original, so 47 | that any problems introduced by others will not reflect on the original 48 | authors' reputations. 49 | 50 | Finally, any free program is threatened constantly by software 51 | patents. We wish to avoid the danger that redistributors of a free 52 | program will individually obtain patent licenses, in effect making the 53 | program proprietary. To prevent this, we have made it clear that any 54 | patent must be licensed for everyone's free use or not licensed at all. 55 | 56 | The precise terms and conditions for copying, distribution and 57 | modification follow. 58 | 59 | GNU GENERAL PUBLIC LICENSE 60 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 61 | 62 | 0. This License applies to any program or other work which contains 63 | a notice placed by the copyright holder saying it may be distributed 64 | under the terms of this General Public License. The "Program", below, 65 | refers to any such program or work, and a "work based on the Program" 66 | means either the Program or any derivative work under copyright law: 67 | that is to say, a work containing the Program or a portion of it, 68 | either verbatim or with modifications and/or translated into another 69 | language. (Hereinafter, translation is included without limitation in 70 | the term "modification".) Each licensee is addressed as "you". 71 | 72 | Activities other than copying, distribution and modification are not 73 | covered by this License; they are outside its scope. The act of 74 | running the Program is not restricted, and the output from the Program 75 | is covered only if its contents constitute a work based on the 76 | Program (independent of having been made by running the Program). 77 | Whether that is true depends on what the Program does. 78 | 79 | 1. You may copy and distribute verbatim copies of the Program's 80 | source code as you receive it, in any medium, provided that you 81 | conspicuously and appropriately publish on each copy an appropriate 82 | copyright notice and disclaimer of warranty; keep intact all the 83 | notices that refer to this License and to the absence of any warranty; 84 | and give any other recipients of the Program a copy of this License 85 | along with the Program. 86 | 87 | You may charge a fee for the physical act of transferring a copy, and 88 | you may at your option offer warranty protection in exchange for a fee. 89 | 90 | 2. You may modify your copy or copies of the Program or any portion 91 | of it, thus forming a work based on the Program, and copy and 92 | distribute such modifications or work under the terms of Section 1 93 | above, provided that you also meet all of these conditions: 94 | 95 | a) You must cause the modified files to carry prominent notices 96 | stating that you changed the files and the date of any change. 97 | 98 | b) You must cause any work that you distribute or publish, that in 99 | whole or in part contains or is derived from the Program or any 100 | part thereof, to be licensed as a whole at no charge to all third 101 | parties under the terms of this License. 102 | 103 | c) If the modified program normally reads commands interactively 104 | when run, you must cause it, when started running for such 105 | interactive use in the most ordinary way, to print or display an 106 | announcement including an appropriate copyright notice and a 107 | notice that there is no warranty (or else, saying that you provide 108 | a warranty) and that users may redistribute the program under 109 | these conditions, and telling the user how to view a copy of this 110 | License. (Exception: if the Program itself is interactive but 111 | does not normally print such an announcement, your work based on 112 | the Program is not required to print an announcement.) 113 | 114 | These requirements apply to the modified work as a whole. If 115 | identifiable sections of that work are not derived from the Program, 116 | and can be reasonably considered independent and separate works in 117 | themselves, then this License, and its terms, do not apply to those 118 | sections when you distribute them as separate works. But when you 119 | distribute the same sections as part of a whole which is a work based 120 | on the Program, the distribution of the whole must be on the terms of 121 | this License, whose permissions for other licensees extend to the 122 | entire whole, and thus to each and every part regardless of who wrote it. 123 | 124 | Thus, it is not the intent of this section to claim rights or contest 125 | your rights to work written entirely by you; rather, the intent is to 126 | exercise the right to control the distribution of derivative or 127 | collective works based on the Program. 128 | 129 | In addition, mere aggregation of another work not based on the Program 130 | with the Program (or with a work based on the Program) on a volume of 131 | a storage or distribution medium does not bring the other work under 132 | the scope of this License. 133 | 134 | 3. You may copy and distribute the Program (or a work based on it, 135 | under Section 2) in object code or executable form under the terms of 136 | Sections 1 and 2 above provided that you also do one of the following: 137 | 138 | a) Accompany it with the complete corresponding machine-readable 139 | source code, which must be distributed under the terms of Sections 140 | 1 and 2 above on a medium customarily used for software interchange; or, 141 | 142 | b) Accompany it with a written offer, valid for at least three 143 | years, to give any third party, for a charge no more than your 144 | cost of physically performing source distribution, a complete 145 | machine-readable copy of the corresponding source code, to be 146 | distributed under the terms of Sections 1 and 2 above on a medium 147 | customarily used for software interchange; or, 148 | 149 | c) Accompany it with the information you received as to the offer 150 | to distribute corresponding source code. (This alternative is 151 | allowed only for noncommercial distribution and only if you 152 | received the program in object code or executable form with such 153 | an offer, in accord with Subsection b above.) 154 | 155 | The source code for a work means the preferred form of the work for 156 | making modifications to it. For an executable work, complete source 157 | code means all the source code for all modules it contains, plus any 158 | associated interface definition files, plus the scripts used to 159 | control compilation and installation of the executable. However, as a 160 | special exception, the source code distributed need not include 161 | anything that is normally distributed (in either source or binary 162 | form) with the major components (compiler, kernel, and so on) of the 163 | operating system on which the executable runs, unless that component 164 | itself accompanies the executable. 165 | 166 | If distribution of executable or object code is made by offering 167 | access to copy from a designated place, then offering equivalent 168 | access to copy the source code from the same place counts as 169 | distribution of the source code, even though third parties are not 170 | compelled to copy the source along with the object code. 171 | 172 | 4. You may not copy, modify, sublicense, or distribute the Program 173 | except as expressly provided under this License. Any attempt 174 | otherwise to copy, modify, sublicense or distribute the Program is 175 | void, and will automatically terminate your rights under this License. 176 | However, parties who have received copies, or rights, from you under 177 | this License will not have their licenses terminated so long as such 178 | parties remain in full compliance. 179 | 180 | 5. You are not required to accept this License, since you have not 181 | signed it. However, nothing else grants you permission to modify or 182 | distribute the Program or its derivative works. These actions are 183 | prohibited by law if you do not accept this License. Therefore, by 184 | modifying or distributing the Program (or any work based on the 185 | Program), you indicate your acceptance of this License to do so, and 186 | all its terms and conditions for copying, distributing or modifying 187 | the Program or works based on it. 188 | 189 | 6. Each time you redistribute the Program (or any work based on the 190 | Program), the recipient automatically receives a license from the 191 | original licensor to copy, distribute or modify the Program subject to 192 | these terms and conditions. You may not impose any further 193 | restrictions on the recipients' exercise of the rights granted herein. 194 | You are not responsible for enforcing compliance by third parties to 195 | this License. 196 | 197 | 7. If, as a consequence of a court judgment or allegation of patent 198 | infringement or for any other reason (not limited to patent issues), 199 | conditions are imposed on you (whether by court order, agreement or 200 | otherwise) that contradict the conditions of this License, they do not 201 | excuse you from the conditions of this License. If you cannot 202 | distribute so as to satisfy simultaneously your obligations under this 203 | License and any other pertinent obligations, then as a consequence you 204 | may not distribute the Program at all. For example, if a patent 205 | license would not permit royalty-free redistribution of the Program by 206 | all those who receive copies directly or indirectly through you, then 207 | the only way you could satisfy both it and this License would be to 208 | refrain entirely from distribution of the Program. 209 | 210 | If any portion of this section is held invalid or unenforceable under 211 | any particular circumstance, the balance of the section is intended to 212 | apply and the section as a whole is intended to apply in other 213 | circumstances. 214 | 215 | It is not the purpose of this section to induce you to infringe any 216 | patents or other property right claims or to contest validity of any 217 | such claims; this section has the sole purpose of protecting the 218 | integrity of the free software distribution system, which is 219 | implemented by public license practices. Many people have made 220 | generous contributions to the wide range of software distributed 221 | through that system in reliance on consistent application of that 222 | system; it is up to the author/donor to decide if he or she is willing 223 | to distribute software through any other system and a licensee cannot 224 | impose that choice. 225 | 226 | This section is intended to make thoroughly clear what is believed to 227 | be a consequence of the rest of this License. 228 | 229 | 8. If the distribution and/or use of the Program is restricted in 230 | certain countries either by patents or by copyrighted interfaces, the 231 | original copyright holder who places the Program under this License 232 | may add an explicit geographical distribution limitation excluding 233 | those countries, so that distribution is permitted only in or among 234 | countries not thus excluded. In such case, this License incorporates 235 | the limitation as if written in the body of this License. 236 | 237 | 9. The Free Software Foundation may publish revised and/or new versions 238 | of the General Public License from time to time. Such new versions will 239 | be similar in spirit to the present version, but may differ in detail to 240 | address new problems or concerns. 241 | 242 | Each version is given a distinguishing version number. If the Program 243 | specifies a version number of this License which applies to it and "any 244 | later version", you have the option of following the terms and conditions 245 | either of that version or of any later version published by the Free 246 | Software Foundation. If the Program does not specify a version number of 247 | this License, you may choose any version ever published by the Free Software 248 | Foundation. 249 | 250 | 10. If you wish to incorporate parts of the Program into other free 251 | programs whose distribution conditions are different, write to the author 252 | to ask for permission. For software which is copyrighted by the Free 253 | Software Foundation, write to the Free Software Foundation; we sometimes 254 | make exceptions for this. Our decision will be guided by the two goals 255 | of preserving the free status of all derivatives of our free software and 256 | of promoting the sharing and reuse of software generally. 257 | 258 | NO WARRANTY 259 | 260 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 261 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 262 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 263 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 264 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 265 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 266 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 267 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 268 | REPAIR OR CORRECTION. 269 | 270 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 271 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 272 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 273 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 274 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 275 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 276 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 277 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 278 | POSSIBILITY OF SUCH DAMAGES. 279 | 280 | END OF TERMS AND CONDITIONS 281 | 282 | How to Apply These Terms to Your New Programs 283 | 284 | If you develop a new program, and you want it to be of the greatest 285 | possible use to the public, the best way to achieve this is to make it 286 | free software which everyone can redistribute and change under these terms. 287 | 288 | To do so, attach the following notices to the program. It is safest 289 | to attach them to the start of each source file to most effectively 290 | convey the exclusion of warranty; and each file should have at least 291 | the "copyright" line and a pointer to where the full notice is found. 292 | 293 | {description} 294 | Copyright (C) {year} {fullname} 295 | 296 | This program is free software; you can redistribute it and/or modify 297 | it under the terms of the GNU General Public License as published by 298 | the Free Software Foundation; either version 2 of the License, or 299 | (at your option) any later version. 300 | 301 | This program is distributed in the hope that it will be useful, 302 | but WITHOUT ANY WARRANTY; without even the implied warranty of 303 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 304 | GNU General Public License for more details. 305 | 306 | You should have received a copy of the GNU General Public License along 307 | with this program; if not, write to the Free Software Foundation, Inc., 308 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 309 | 310 | Also add information on how to contact you by electronic and paper mail. 311 | 312 | If the program is interactive, make it output a short notice like this 313 | when it starts in an interactive mode: 314 | 315 | Gnomovision version 69, Copyright (C) year name of author 316 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 317 | This is free software, and you are welcome to redistribute it 318 | under certain conditions; type `show c' for details. 319 | 320 | The hypothetical commands `show w' and `show c' should show the appropriate 321 | parts of the General Public License. Of course, the commands you use may 322 | be called something other than `show w' and `show c'; they could even be 323 | mouse-clicks or menu items--whatever suits your program. 324 | 325 | You should also get your employer (if you work as a programmer) or your 326 | school, if any, to sign a "copyright disclaimer" for the program, if 327 | necessary. Here is a sample; alter the names: 328 | 329 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 330 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 331 | 332 | {signature of Ty Coon}, 1 April 1989 333 | Ty Coon, President of Vice 334 | 335 | This General Public License does not permit incorporating your program into 336 | proprietary programs. If your program is a subroutine library, you may 337 | consider it more useful to permit linking proprietary applications with the 338 | library. If this is what you want to do, use the GNU Lesser General 339 | Public License instead of this License. 340 | 341 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | CC = g++ 2 | LDFLAGS = -lgsl -lm -lgslcblas 3 | 4 | 5 | LSOURCE = main.cpp corpus.cpp slda.cpp utils.cpp opt.cpp 6 | LHEADER = corpus.h slda.h utils.h opt.h settings.h 7 | 8 | slda: $(LSOURCE) $(HEADER) 9 | $(CC) $(LSOURCE) -o $@ $(LDFLAGS) 10 | 11 | clean: 12 | -rm -f *.o slda 13 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Supervised latent Dirichlet allocation for classification 2 | 3 | (C) Copyright 2009, Chong Wang, David Blei and Li Fei-Fei. Written by Chong Wang, chongw@cs.princeton.edu, part of code 4 | is from [lda-c](https://github.com/Blei-Lab/lda-c). 5 | 6 | This is a C++ implementation of supervised latent Dirichlet allocation (sLDA) for classification. Note that the code here is slightly different from what was described in [2] in order to speed up. Note that this is only the sLDA. The annotation part is not yet posted. 7 | 8 | ## Sample data 9 | 10 | A preprocessed 8-class image dataset [2] from [Labelme](http://labelme.csail.mit.edu/). See [images.tgz](./sample-data/images.tgz). 11 | 12 | UIUC Sports annotation files: [annotations](./sample-data/uiuc-sports-annotations.txt) and [meta information](./sample-data/uiuc-sports-info.txt). The source image files can be found [here](http://vision.stanford.edu/lijiali/event_dataset/). (Note: there might be some discrepancies and it is unclear why...) 13 | 14 | ## References 15 | 16 | [1] Chong Wang, David M. Blei and Li Fei-Fei. Simultaneous image classification and annotation. In CVPR, 2009. [[PDF]](http://www.cs.columbia.edu/~blei/papers/WangBleiFeiFei2009.pdf) 17 | 18 | [2] David M. Blei and Jon McAuliffe. Supervised topic models. In NIPS, 2007. [[PDF]](http://www.cs.columbia.edu/~blei/papers/BleiMcAuliffe2007.pdf) 19 | 20 | 21 | --- 22 | 23 | ## README 24 | 25 | Note that this code requires the Gnu Scientific Library, http://www.gnu.org/software/gsl/ 26 | 27 | ------------------------------------------------------------------------ 28 | 29 | 30 | TABLE OF CONTENTS 31 | 32 | 33 | A. COMPILING 34 | 35 | B. ESTIMATION 36 | 37 | C. INFERENCE 38 | 39 | 40 | ------------------------------------------------------------------------ 41 | 42 | A. COMPILING 43 | 44 | Type "make" in a shell. Make sure the GSL is installed. 45 | 46 | 47 | ------------------------------------------------------------------------ 48 | 49 | B. ESTIMATION 50 | 51 | Estimate the model by executing: 52 | 53 | slda [est] [data] [label] [settings] [alpha] [k] [seeded/random/model_path] [directory] 54 | 55 | The saved models are in two files: 56 | 57 | .model is the model saved in the binary format, which is easy and 58 | fast to use for inference. 59 | 60 | .model.txt is the model saved in the text format, which is 61 | convenient for printing topics or analysis using python. 62 | 63 | 64 | The variational posterior Dirichlets are in: 65 | 66 | .gamma 67 | 68 | 69 | Data format 70 | 71 | (1) [data] is a file where each line is of the form: 72 | 73 | [M] [term_1]:[count] [term_2]:[count] ... [term_N]:[count] 74 | 75 | where [M] is the number of unique terms in the document, and the 76 | [count] associated with each term is how many times that term appeared 77 | in the document. 78 | 79 | (2) [label] is a file where each line is the corresponding label for [data]. 80 | The labels must be 0, 1, ..., C-1, if we have C classes. 81 | 82 | 83 | ------------------------------------------------------------------------ 84 | 85 | C. INFERENCE 86 | 87 | To perform inference on a different set of data (in the same format as 88 | for estimation), execute: 89 | 90 | slda [inf] [data] [label] [settings] [model] [directory] 91 | 92 | where [model] is the binary file from the estimation. 93 | 94 | The predictive labels are in: 95 | 96 | inf-labels.dat 97 | 98 | The variational posterior Dirichlets are in: 99 | 100 | inf-gamma.dat 101 | -------------------------------------------------------------------------------- /corpus.cpp: -------------------------------------------------------------------------------- 1 | // (C) Copyright 2009, Chong Wang, David Blei and Li Fei-Fei 2 | 3 | // written by Chong Wang, chongw@cs.princeton.edu 4 | 5 | // This file is part of slda. 6 | 7 | // slda is free software; you can redistribute it and/or modify it under 8 | // the terms of the GNU General Public License as published by the Free 9 | // Software Foundation; either version 2 of the License, or (at your 10 | // option) any later version. 11 | 12 | // slda is distributed in the hope that it will be useful, but WITHOUT 13 | // ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 14 | // FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License 15 | // for more details. 16 | 17 | // You should have received a copy of the GNU General Public License 18 | // along with this program; if not, write to the Free Software 19 | // Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 20 | // USA 21 | 22 | #include "corpus.h" 23 | #include 24 | #include 25 | 26 | corpus::corpus() 27 | { 28 | num_docs = 0; 29 | size_vocab = 0; 30 | num_classes = 0; 31 | num_total_words = 0; 32 | } 33 | 34 | corpus::~corpus() 35 | { 36 | for (int i = 0; i < num_docs; i ++) 37 | { 38 | document * doc = docs[i]; 39 | delete doc; 40 | } 41 | docs.clear(); 42 | 43 | num_docs = 0; 44 | size_vocab = 0; 45 | num_classes = 0; 46 | num_total_words = 0; 47 | } 48 | 49 | void corpus::read_data(const char * data_filename, 50 | const char * label_filename) 51 | { 52 | int OFFSET = 0; 53 | int length = 0, count = 0, word = 0, 54 | n = 0, nd = 0, nw = 0, label = -1; 55 | 56 | FILE * fileptr; 57 | fileptr = fopen(data_filename, "r"); 58 | printf("\nreading data from %s\n", data_filename); 59 | nd = 0; 60 | nw = 0; 61 | 62 | while ((fscanf(fileptr, "%10d", &length) != EOF)) 63 | { 64 | document * doc = new document(length); 65 | for (n = 0; n < length; n++) 66 | { 67 | fscanf(fileptr, "%10d:%10d", &word, &count); 68 | word = word - OFFSET; 69 | doc->words[n] = word; 70 | doc->counts[n] = count; 71 | doc->total += count; 72 | if (word >= nw) 73 | { 74 | nw = word + 1; 75 | } 76 | } 77 | num_total_words += doc->total; 78 | docs.push_back(doc); 79 | nd++; 80 | } 81 | fclose(fileptr); 82 | num_docs = nd; 83 | size_vocab = nw; 84 | printf("number of docs : %d\n", nd); 85 | printf("number of terms : %d\n", nw); 86 | printf("number of total words : %d\n", num_total_words); 87 | 88 | fileptr = fopen(label_filename, "r"); 89 | printf("\nreading labels from %s\n", label_filename); 90 | nd = 0; 91 | while ((fscanf(fileptr, "%10d", &label) != EOF)) 92 | { 93 | document * doc = docs[nd]; 94 | doc->label = label; 95 | if (label >= num_classes) 96 | { 97 | num_classes = label + 1; 98 | } 99 | nd ++; 100 | } 101 | assert(nd == int(docs.size())); 102 | printf("number of classes : %d\n\n", num_classes); 103 | } 104 | 105 | int corpus::max_corpus_length() { 106 | int max_length = 0; 107 | 108 | for (int d = 0; d < num_docs; d++) { 109 | if (docs[d]->length > max_length) 110 | max_length = docs[d]->length; 111 | } 112 | return max_length; 113 | } 114 | -------------------------------------------------------------------------------- /corpus.h: -------------------------------------------------------------------------------- 1 | // (C) Copyright 2009, Chong Wang, David Blei and Li Fei-Fei 2 | 3 | // written by Chong Wang, chongw@cs.princeton.edu 4 | 5 | // This file is part of slda. 6 | 7 | // slda is free software; you can redistribute it and/or modify it under 8 | // the terms of the GNU General Public License as published by the Free 9 | // Software Foundation; either version 2 of the License, or (at your 10 | // option) any later version. 11 | 12 | // slda is distributed in the hope that it will be useful, but WITHOUT 13 | // ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 14 | // FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License 15 | // for more details. 16 | 17 | // You should have received a copy of the GNU General Public License 18 | // along with this program; if not, write to the Free Software 19 | // Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 20 | // USA 21 | 22 | #ifndef CORPUS_H 23 | #define CORPUS_H 24 | 25 | #include 26 | using namespace std; 27 | 28 | class document 29 | { 30 | public: 31 | int * words; 32 | int * counts; 33 | int length; 34 | int total; 35 | int label; 36 | public: 37 | document() 38 | { 39 | words = NULL; 40 | counts = NULL; 41 | length = 0; 42 | total = 0; 43 | label = -1; 44 | } 45 | document(int len) 46 | { 47 | length = len; 48 | words = new int [length]; 49 | counts = new int [length]; 50 | total = 0; 51 | label = -1; 52 | } 53 | ~document() 54 | { 55 | if (words != NULL) 56 | { 57 | delete [] words; 58 | delete [] counts; 59 | length = 0; 60 | total = 0; 61 | label = -1; 62 | } 63 | } 64 | }; 65 | 66 | class corpus 67 | { 68 | public: 69 | corpus(); 70 | ~corpus(); 71 | void read_data(const char * data_filename, const char * label_filename); 72 | int max_corpus_length(); 73 | public: 74 | int num_docs; 75 | int size_vocab; 76 | int num_classes; 77 | int num_total_words; 78 | vector docs; 79 | }; 80 | 81 | #endif // CORPUS_H 82 | -------------------------------------------------------------------------------- /main.cpp: -------------------------------------------------------------------------------- 1 | // (C) Copyright 2009, Chong Wang, David Blei and Li Fei-Fei 2 | 3 | // written by Chong Wang, chongw@cs.princeton.edu 4 | 5 | // This file is part of slda. 6 | 7 | // slda is free software; you can redistribute it and/or modify it under 8 | // the terms of the GNU General Public License as published by the Free 9 | // Software Foundation; either version 2 of the License, or (at your 10 | // option) any later version. 11 | 12 | // slda is distributed in the hope that it will be useful, but WITHOUT 13 | // ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 14 | // FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License 15 | // for more details. 16 | 17 | // You should have received a copy of the GNU General Public License 18 | // along with this program; if not, write to the Free Software 19 | // Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 20 | // USA 21 | 22 | #include 23 | #include 24 | #include "corpus.h" 25 | #include "utils.h" 26 | #include "slda.h" 27 | 28 | void help( void ) { 29 | printf("usage: slda [est] [data] [label] [settings] [alpha] [k] [random/seeded/model_path] [directory]\n"); 30 | printf(" slda [inf] [data] [label] [settings] [model] [directory]\n"); 31 | } 32 | 33 | int main(int argc, char* argv[]) 34 | { 35 | if (argc < 2) 36 | { 37 | help(); 38 | return 0; 39 | } 40 | if (strcmp(argv[1], "est") == 0) 41 | { 42 | corpus c; 43 | char * data_filename = argv[2]; 44 | char * label_filename = argv[3]; 45 | c.read_data(data_filename, label_filename); 46 | settings setting; 47 | char * setting_filename = argv[4]; 48 | setting.read_settings(setting_filename); 49 | 50 | double alpha = atof(argv[5]); 51 | int num_topics = atoi(argv[6]); 52 | printf("number of topics is %d\n", num_topics); 53 | char * init_method = argv[7]; 54 | char * directory = argv[8]; 55 | printf("models will be saved in %s\n", directory); 56 | make_directory(directory); 57 | 58 | slda model; 59 | model.init(alpha, num_topics, &c); 60 | model.v_em(&c, &setting, init_method, directory); 61 | } 62 | 63 | if (strcmp(argv[1], "inf") == 0) 64 | { 65 | corpus c; 66 | char * data_filename = argv[2]; 67 | char * label_filename = argv[3]; 68 | c.read_data(data_filename, label_filename); 69 | settings setting; 70 | char * setting_filename = argv[4]; 71 | setting.read_settings(setting_filename); 72 | 73 | char * model_filename = argv[5]; 74 | char * directory = argv[6]; 75 | printf("\nresults will be saved in %s\n", directory); 76 | make_directory(directory); 77 | 78 | slda model; 79 | model.load_model(model_filename); 80 | model.infer_only(&c, &setting, directory); 81 | } 82 | 83 | return 0; 84 | } 85 | -------------------------------------------------------------------------------- /opt.cpp: -------------------------------------------------------------------------------- 1 | // (C) Copyright 2009, Chong Wang, David Blei and Li Fei-Fei 2 | 3 | // written by Chong Wang, chongw@cs.princeton.edu 4 | 5 | // This file is part of slda. 6 | 7 | // slda is free software; you can redistribute it and/or modify it under 8 | // the terms of the GNU General Public License as published by the Free 9 | // Software Foundation; either version 2 of the License, or (at your 10 | // option) any later version. 11 | 12 | // slda is distributed in the hope that it will be useful, but WITHOUT 13 | // ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 14 | // FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License 15 | // for more details. 16 | 17 | // You should have received a copy of the GNU General Public License 18 | // along with this program; if not, write to the Free Software 19 | // Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 20 | // USA 21 | #include "opt.h" 22 | #include "slda.h" 23 | #include "utils.h" 24 | /* 25 | * Here the implementation is slightly different from the equations 26 | * in the paper, we instead use a second-order taylor expansion to approximate 27 | * the second line in eqaution (6). 28 | */ 29 | 30 | double softmax_f(const gsl_vector * x, void * opt_param) 31 | { 32 | opt_parameter * gsl_param = (opt_parameter *)opt_param; 33 | double PENALTY = gsl_param->PENALTY; 34 | slda * model = gsl_param->model; 35 | suffstats * ss = gsl_param->ss; 36 | 37 | double f, t, a1 = 0.0, a2 = 0.0; 38 | 39 | int k, d, j, l, idx; 40 | 41 | double f_regularization = 0.0; 42 | 43 | 44 | for (l = 0; l < model->num_classes-1; l ++) 45 | { 46 | for (k = 0; k < model->num_topics; k ++) 47 | { 48 | model->eta[l][k] = gsl_vector_get(x, l*model->num_topics + k); 49 | f_regularization -= pow(model->eta[l][k], 2) * PENALTY/2.0; 50 | } 51 | } 52 | f = 0.0; //log likelihood 53 | for (d = 0; d < ss->num_docs; d ++) 54 | { 55 | for (k = 0; k < model->num_topics; k ++) 56 | { 57 | if (ss->labels[d] < model->num_classes-1) 58 | { 59 | f += model->eta[ss->labels[d]][k] * ss->z_bar[d].z_bar_m[k]; 60 | } 61 | } 62 | 63 | t = 0.0; // in log space, 1+exp()+exp()... 64 | for (l = 0; l < model->num_classes-1; l ++) 65 | { 66 | a1 = 0.0; // \eta_k^T * \bar{\phi}_d 67 | a2 = 0.0; // 1 + 0.5 * \eta_k^T * Var(z_bar)\eta_k 68 | for (k = 0; k < model->num_topics; k ++) 69 | { 70 | a1 += model->eta[l][k] * ss->z_bar[d].z_bar_m[k]; 71 | for (j = 0; j < model->num_topics; j ++) 72 | { 73 | idx = map_idx(k, j, model->num_topics); 74 | a2 += model->eta[l][k] * ss->z_bar[d].z_bar_var[idx] * model->eta[l][j]; 75 | } 76 | } 77 | a2 = 1.0 + 0.5 * a2; 78 | t = log_sum(t, a1 + log(a2)); 79 | } 80 | f -= t; 81 | } 82 | 83 | return -(f + f_regularization); 84 | } 85 | void softmax_df(const gsl_vector * x, void * opt_param, gsl_vector * df) 86 | { 87 | 88 | opt_parameter * gsl_param = (opt_parameter *)opt_param; 89 | double PENALTY = gsl_param->PENALTY; 90 | slda * model = gsl_param->model; 91 | suffstats * ss = gsl_param->ss; 92 | gsl_vector_set_zero(df); 93 | gsl_vector * df_tmp = gsl_vector_alloc(df->size); 94 | 95 | double t, a1 = 0.0, a2 = 0.0, g; 96 | int k, d, j, l, idx; 97 | 98 | double * eta_aux = new double [model->num_topics]; 99 | 100 | for (l = 0; l < model->num_classes-1; l ++) 101 | { 102 | for (k = 0; k < model->num_topics; k ++) 103 | { 104 | idx = l*model->num_topics + k; 105 | model->eta[l][k] = gsl_vector_get(x, idx); 106 | g = -PENALTY * model->eta[l][k]; 107 | gsl_vector_set(df, idx, g); 108 | } 109 | } 110 | for (d = 0; d < ss->num_docs; d ++) 111 | { 112 | for (k = 0; k < model->num_topics; k ++) 113 | { 114 | l = ss->labels[d]; 115 | if (l < model->num_classes-1) 116 | { 117 | idx = l*model->num_topics + k; 118 | g = gsl_vector_get(df, idx) + ss->z_bar[d].z_bar_m[k]; 119 | gsl_vector_set(df, idx, g); 120 | } 121 | } 122 | 123 | t = 0.0; // in log space, 1+exp()+exp()+.... 124 | gsl_vector_memcpy(df_tmp, df); 125 | gsl_vector_set_zero(df); 126 | for (l = 0; l < model->num_classes-1; l ++) 127 | { 128 | memset(eta_aux, 0, sizeof(double)*model->num_topics); 129 | a1 = 0.0; // \eta_k^T * \bar{\phi}_d 130 | a2 = 0.0; // 1 + 0.5*\eta_k^T * Var(z_bar)\eta_k 131 | for (k = 0; k < model->num_topics; k ++) 132 | { 133 | a1 += model->eta[l][k] * ss->z_bar[d].z_bar_m[k]; 134 | for (j = 0; j < model->num_topics; j ++) 135 | { 136 | idx = map_idx(k, j, model->num_topics); 137 | a2 += model->eta[l][k] * ss->z_bar[d].z_bar_var[idx] * model->eta[l][j]; 138 | eta_aux[k] += ss->z_bar[d].z_bar_var[idx] * model->eta[l][j]; 139 | } 140 | } 141 | a2 = 1.0 + 0.5 * a2; 142 | t = log_sum(t, a1 + log(a2)); 143 | 144 | for (k = 0; k < model->num_topics; k ++) 145 | { 146 | idx = l*model->num_topics + k; 147 | g = gsl_vector_get(df, idx) - 148 | exp(a1) * (ss->z_bar[d].z_bar_m[k] * a2 + eta_aux[k]); 149 | gsl_vector_set(df, idx, g); 150 | } 151 | } 152 | gsl_vector_scale(df, exp(-t)); 153 | gsl_vector_add(df, df_tmp); 154 | } 155 | gsl_vector_scale(df, -1.0); 156 | delete [] eta_aux; 157 | gsl_vector_free(df_tmp); 158 | } 159 | void softmax_fdf(const gsl_vector * x, void * opt_param, double * f, gsl_vector * df) 160 | { 161 | opt_parameter * gsl_param = (opt_parameter *)opt_param; 162 | double PENALTY = gsl_param->PENALTY; 163 | slda * model = gsl_param->model; 164 | suffstats * ss = gsl_param->ss; 165 | gsl_vector_set_zero(df); 166 | gsl_vector * df_tmp = gsl_vector_alloc(df->size); 167 | 168 | double t, a1 = 0.0, a2 = 0.0, g; 169 | int k, d, j, l, idx; 170 | 171 | double f_regularization = 0.0; 172 | 173 | double* eta_aux = new double [model->num_topics]; 174 | 175 | for (l = 0; l < model->num_classes-1; l ++) 176 | { 177 | for (k = 0; k < model->num_topics; k ++) 178 | { 179 | model->eta[l][k] = gsl_vector_get(x, l*model->num_topics + k); 180 | f_regularization -= pow(model->eta[l][k], 2) * PENALTY/2.0; 181 | idx = l*model->num_topics + k; 182 | g = -PENALTY * model->eta[l][k]; 183 | gsl_vector_set(df, idx, g); 184 | } 185 | } 186 | *f = 0.0; //log likelihood 187 | for (d = 0; d < ss->num_docs; d ++) 188 | { 189 | for (k = 0; k < model->num_topics; k ++) 190 | { 191 | l = ss->labels[d]; 192 | if (l < model->num_classes-1) 193 | { 194 | *f += model->eta[l][k] * ss->z_bar[d].z_bar_m[k]; 195 | idx = l*model->num_topics + k; 196 | g = gsl_vector_get(df, idx) + ss->z_bar[d].z_bar_m[k]; 197 | gsl_vector_set(df, idx, g); 198 | } 199 | } 200 | t = 0.0; // in log space, base class 1+exp()+exp() 201 | gsl_vector_memcpy(df_tmp, df); 202 | gsl_vector_set_zero(df); 203 | for (l = 0; l < model->num_classes-1; l ++) 204 | { 205 | memset(eta_aux, 0, sizeof(double)*model->num_topics); 206 | a1 = 0.0; // \eta_k^T * \bar{\phi}_d 207 | a2 = 0.0; // 1 + 0.5 * \eta_k^T * Var(z_bar)\eta_k 208 | for (k = 0; k < model->num_topics; k ++) 209 | { 210 | a1 += model->eta[l][k] * ss->z_bar[d].z_bar_m[k]; 211 | for (j = 0; j < model->num_topics; j ++) 212 | { 213 | idx = map_idx(k, j, model->num_topics); 214 | a2 += model->eta[l][k] * ss->z_bar[d].z_bar_var[idx] * model->eta[l][j]; 215 | eta_aux[k] += ss->z_bar[d].z_bar_var[idx] * model->eta[l][j]; 216 | } 217 | } 218 | a2 = 1.0 + 0.5 * a2; 219 | t = log_sum(t, a1 + log(a2)); 220 | 221 | for (k = 0; k < model->num_topics; k ++) 222 | { 223 | idx = l*model->num_topics + k; 224 | g = gsl_vector_get(df, idx) - 225 | exp(a1) * (ss->z_bar[d].z_bar_m[k] * a2 + eta_aux[k]); 226 | gsl_vector_set(df, idx, g); 227 | } 228 | } 229 | gsl_vector_scale(df, exp(-t)); 230 | gsl_vector_add(df, df_tmp); 231 | *f -= t; 232 | } 233 | gsl_vector_scale(df, -1.0); 234 | *f = -(*f + f_regularization); 235 | delete [] eta_aux; 236 | gsl_vector_free(df_tmp); 237 | } 238 | 239 | -------------------------------------------------------------------------------- /opt.h: -------------------------------------------------------------------------------- 1 | // (C) Copyright 2009, Chong Wang, David Blei and Li Fei-Fei 2 | 3 | // written by Chong Wang, chongw@cs.princeton.edu 4 | 5 | // This file is part of slda. 6 | 7 | // slda is free software; you can redistribute it and/or modify it under 8 | // the terms of the GNU General Public License as published by the Free 9 | // Software Foundation; either version 2 of the License, or (at your 10 | // option) any later version. 11 | 12 | // slda is distributed in the hope that it will be useful, but WITHOUT 13 | // ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 14 | // FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License 15 | // for more details. 16 | 17 | // You should have received a copy of the GNU General Public License 18 | // along with this program; if not, write to the Free Software 19 | // Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 20 | // USA 21 | #ifndef OPT_H_INCLUDED 22 | #define OPT_H_INCLUDED 23 | #include 24 | #include "slda.h" 25 | 26 | /* 27 | * structure for the gsl optimization routine 28 | * 29 | */ 30 | 31 | struct opt_parameter 32 | { 33 | suffstats * ss; 34 | slda * model; 35 | double PENALTY; 36 | }; 37 | 38 | /* 39 | * function to compute the value of the obj function, then 40 | * return it 41 | */ 42 | 43 | double softmax_f(const gsl_vector * x, void * opt_param); 44 | 45 | /* 46 | * function to compute the derivatives of function 47 | * 48 | */ 49 | 50 | void softmax_df(const gsl_vector * x, void * opt_param, gsl_vector * df); 51 | 52 | /* 53 | * function to compute the value and derivatives of the function 54 | * 55 | */ 56 | 57 | void softmax_fdf(const gsl_vector * x, void * opt_param, double * f, gsl_vector * df); 58 | 59 | #endif // OPT_H_INCLUDED 60 | 61 | -------------------------------------------------------------------------------- /sample-data/images.tgz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/blei-lab/class-slda/166c7aaa950649e7b4607de9b894e0824bec1932/sample-data/images.tgz -------------------------------------------------------------------------------- /sample-data/uiuc-sports-info.txt: -------------------------------------------------------------------------------- 1 | badminton image_0001.jpg 2 | badminton image_0002.jpg 3 | badminton image_0003.jpg 4 | badminton image_0004.jpg 5 | badminton image_0005.jpg 6 | badminton image_0006.jpg 7 | badminton image_0007.jpg 8 | badminton image_0008.jpg 9 | badminton image_0009.jpg 10 | badminton image_0010.jpg 11 | badminton image_0011.jpg 12 | badminton image_0012.jpg 13 | badminton image_0013.jpg 14 | badminton image_0014.jpg 15 | badminton image_0015.jpg 16 | badminton image_0016.jpg 17 | badminton image_0017.jpg 18 | badminton image_0018.jpg 19 | badminton image_0019.jpg 20 | badminton image_0020.jpg 21 | badminton image_0021.jpg 22 | badminton image_0022.jpg 23 | badminton image_0023.jpg 24 | badminton image_0024.jpg 25 | badminton image_0025.jpg 26 | badminton image_0026.jpg 27 | badminton image_0027.jpg 28 | badminton image_0028.jpg 29 | badminton image_0029.jpg 30 | badminton image_0030.jpg 31 | badminton image_0031.jpg 32 | badminton image_0032.jpg 33 | badminton image_0033.jpg 34 | badminton image_0034.jpg 35 | badminton image_0035.jpg 36 | badminton image_0036.jpg 37 | badminton image_0037.jpg 38 | badminton image_0038.jpg 39 | badminton image_0039.jpg 40 | badminton image_0040.jpg 41 | badminton image_0041.jpg 42 | badminton image_0042.jpg 43 | badminton image_0043.jpg 44 | badminton image_0044.jpg 45 | badminton image_0045.jpg 46 | badminton image_0046.jpg 47 | badminton image_0047.jpg 48 | badminton image_0048.jpg 49 | badminton image_0049.jpg 50 | badminton image_0050.jpg 51 | badminton image_0051.jpg 52 | badminton image_0052.jpg 53 | badminton image_0053.jpg 54 | badminton image_0054.jpg 55 | badminton image_0055.jpg 56 | badminton image_0056.jpg 57 | badminton image_0057.jpg 58 | badminton image_0058.jpg 59 | badminton image_0059.jpg 60 | badminton image_0060.jpg 61 | badminton image_0061.jpg 62 | badminton image_0062.jpg 63 | badminton image_0063.jpg 64 | badminton image_0064.jpg 65 | badminton image_0065.jpg 66 | badminton image_0066.jpg 67 | badminton image_0067.jpg 68 | badminton image_0068.jpg 69 | badminton image_0069.jpg 70 | badminton image_0070.jpg 71 | badminton image_0071.jpg 72 | badminton image_0072.jpg 73 | badminton image_0073.jpg 74 | badminton image_0074.jpg 75 | badminton image_0075.jpg 76 | badminton image_0076.jpg 77 | badminton image_0077.jpg 78 | badminton image_0078.jpg 79 | badminton image_0079.jpg 80 | badminton image_0080.jpg 81 | badminton image_0081.jpg 82 | badminton image_0082.jpg 83 | badminton image_0083.jpg 84 | badminton image_0084.jpg 85 | badminton image_0085.jpg 86 | badminton image_0086.jpg 87 | badminton image_0087.jpg 88 | badminton image_0088.jpg 89 | badminton image_0089.jpg 90 | badminton image_0090.jpg 91 | badminton image_0091.jpg 92 | badminton image_0092.jpg 93 | badminton image_0093.jpg 94 | badminton image_0094.jpg 95 | badminton image_0095.jpg 96 | badminton image_0096.jpg 97 | badminton image_0097.jpg 98 | badminton image_0098.jpg 99 | badminton image_0099.jpg 100 | badminton image_0100.jpg 101 | badminton image_0101.jpg 102 | badminton image_0102.jpg 103 | badminton image_0103.jpg 104 | badminton image_0104.jpg 105 | badminton image_0105.jpg 106 | badminton image_0106.jpg 107 | badminton image_0107.jpg 108 | badminton image_0108.jpg 109 | badminton image_0109.jpg 110 | badminton image_0110.jpg 111 | badminton image_0111.jpg 112 | badminton image_0112.jpg 113 | badminton image_0113.jpg 114 | badminton image_0114.jpg 115 | badminton image_0115.jpg 116 | badminton image_0116.jpg 117 | badminton image_0117.jpg 118 | badminton image_0118.jpg 119 | badminton image_0119.jpg 120 | badminton image_0120.jpg 121 | badminton image_0121.jpg 122 | badminton image_0122.jpg 123 | badminton image_0123.jpg 124 | badminton image_0124.jpg 125 | badminton image_0125.jpg 126 | badminton image_0126.jpg 127 | badminton image_0127.jpg 128 | badminton image_0128.jpg 129 | badminton image_0129.jpg 130 | badminton image_0130.jpg 131 | badminton image_0131.jpg 132 | badminton image_0132.jpg 133 | badminton image_0133.jpg 134 | badminton image_0134.jpg 135 | badminton image_0135.jpg 136 | badminton image_0136.jpg 137 | badminton image_0137.jpg 138 | badminton image_0138.jpg 139 | badminton image_0139.jpg 140 | badminton image_0140.jpg 141 | badminton image_0141.jpg 142 | badminton image_0142.jpg 143 | badminton image_0143.jpg 144 | badminton image_0144.jpg 145 | badminton image_0145.jpg 146 | badminton image_0146.jpg 147 | badminton image_0147.jpg 148 | badminton image_0148.jpg 149 | badminton image_0149.jpg 150 | badminton image_0150.jpg 151 | badminton image_0151.jpg 152 | badminton image_0152.jpg 153 | badminton image_0153.jpg 154 | badminton image_0154.jpg 155 | badminton image_0155.jpg 156 | badminton image_0156.jpg 157 | badminton image_0157.jpg 158 | badminton image_0158.jpg 159 | badminton image_0159.jpg 160 | badminton image_0160.jpg 161 | badminton image_0161.jpg 162 | badminton image_0162.jpg 163 | badminton image_0163.jpg 164 | badminton image_0164.jpg 165 | badminton image_0165.jpg 166 | badminton image_0166.jpg 167 | badminton image_0167.jpg 168 | badminton image_0168.jpg 169 | badminton image_0169.jpg 170 | badminton image_0170.jpg 171 | badminton image_0171.jpg 172 | badminton image_0172.jpg 173 | badminton image_0173.jpg 174 | badminton image_0174.jpg 175 | badminton image_0175.jpg 176 | badminton image_0176.jpg 177 | badminton image_0177.jpg 178 | badminton image_0178.jpg 179 | badminton image_0179.jpg 180 | badminton image_0180.jpg 181 | badminton image_0181.jpg 182 | badminton image_0182.jpg 183 | badminton image_0183.jpg 184 | badminton image_0184.jpg 185 | badminton image_0185.jpg 186 | badminton image_0186.jpg 187 | badminton image_0187.jpg 188 | badminton image_0188.jpg 189 | badminton image_0189.jpg 190 | badminton image_0190.jpg 191 | badminton image_0191.jpg 192 | badminton image_0192.jpg 193 | badminton image_0193.jpg 194 | badminton image_0194.jpg 195 | badminton image_0195.jpg 196 | badminton image_0196.jpg 197 | badminton image_0197.jpg 198 | badminton image_0198.jpg 199 | badminton image_0199.jpg 200 | badminton image_0200.jpg 201 | badminton image_0201.jpg 202 | badminton image_0202.jpg 203 | badminton image_0203.jpg 204 | badminton image_0204.jpg 205 | badminton image_0205.jpg 206 | badminton image_0206.jpg 207 | badminton image_0207.jpg 208 | badminton image_0208.jpg 209 | badminton image_0209.jpg 210 | badminton image_0210.jpg 211 | badminton image_0211.jpg 212 | badminton image_0212.jpg 213 | badminton image_0213.jpg 214 | badminton image_0214.jpg 215 | badminton image_0215.jpg 216 | badminton image_0216.jpg 217 | badminton image_0217.jpg 218 | badminton image_0218.jpg 219 | badminton image_0219.jpg 220 | badminton image_0220.jpg 221 | badminton image_0221.jpg 222 | badminton image_0222.jpg 223 | badminton image_0223.jpg 224 | badminton image_0224.jpg 225 | badminton image_0225.jpg 226 | badminton image_0226.jpg 227 | badminton image_0227.jpg 228 | badminton image_0228.jpg 229 | badminton image_0229.jpg 230 | badminton image_0230.jpg 231 | badminton image_0231.jpg 232 | badminton image_0232.jpg 233 | badminton image_0233.jpg 234 | badminton image_0234.jpg 235 | badminton image_0235.jpg 236 | badminton image_0236.jpg 237 | badminton image_0237.jpg 238 | badminton image_0238.jpg 239 | badminton image_0239.jpg 240 | badminton image_0240.jpg 241 | badminton image_0241.jpg 242 | badminton image_0242.jpg 243 | badminton image_0243.jpg 244 | badminton image_0244.jpg 245 | badminton image_0245.jpg 246 | badminton image_0246.jpg 247 | badminton image_0247.jpg 248 | badminton image_0248.jpg 249 | badminton image_0249.jpg 250 | badminton image_0250.jpg 251 | badminton image_0251.jpg 252 | badminton image_0252.jpg 253 | badminton image_0253.jpg 254 | badminton image_0254.jpg 255 | badminton image_0255.jpg 256 | badminton image_0256.jpg 257 | badminton image_0257.jpg 258 | badminton image_0258.jpg 259 | badminton image_0259.jpg 260 | badminton image_0260.jpg 261 | badminton image_0261.jpg 262 | badminton image_0262.jpg 263 | badminton image_0263.jpg 264 | badminton image_0264.jpg 265 | badminton image_0265.jpg 266 | badminton image_0266.jpg 267 | badminton image_0267.jpg 268 | badminton image_0268.jpg 269 | badminton image_0269.jpg 270 | badminton image_0270.jpg 271 | badminton image_0271.jpg 272 | badminton image_0272.jpg 273 | badminton image_0273.jpg 274 | badminton image_0274.jpg 275 | badminton image_0275.jpg 276 | badminton image_0276.jpg 277 | badminton image_0277.jpg 278 | badminton image_0278.jpg 279 | badminton image_0279.jpg 280 | badminton image_0280.jpg 281 | badminton image_0281.jpg 282 | badminton image_0282.jpg 283 | badminton image_0283.jpg 284 | badminton image_0284.jpg 285 | badminton image_0285.jpg 286 | badminton image_0286.jpg 287 | badminton image_0287.jpg 288 | badminton image_0288.jpg 289 | badminton image_0289.jpg 290 | badminton image_0290.jpg 291 | badminton image_0291.jpg 292 | badminton image_0292.jpg 293 | badminton image_0293.jpg 294 | badminton image_0294.jpg 295 | badminton image_0295.jpg 296 | badminton image_0296.jpg 297 | badminton image_0297.jpg 298 | badminton image_0298.jpg 299 | badminton image_0299.jpg 300 | badminton image_0300.jpg 301 | badminton image_0301.jpg 302 | badminton image_0302.jpg 303 | badminton image_0303.jpg 304 | badminton image_0304.jpg 305 | badminton image_0305.jpg 306 | badminton image_0306.jpg 307 | badminton image_0307.jpg 308 | badminton image_0308.jpg 309 | badminton image_0309.jpg 310 | badminton image_0310.jpg 311 | badminton image_0311.jpg 312 | badminton image_0312.jpg 313 | badminton image_0313.jpg 314 | badminton image_0314.jpg 315 | bocce image_0001.jpg 316 | bocce image_0002.jpg 317 | bocce image_0003.jpg 318 | bocce image_0004.jpg 319 | bocce image_0005.jpg 320 | bocce image_0006.jpg 321 | bocce image_0007.jpg 322 | bocce image_0008.jpg 323 | bocce image_0009.jpg 324 | bocce image_0010.jpg 325 | bocce image_0011.jpg 326 | bocce image_0012.jpg 327 | bocce image_0013.jpg 328 | bocce image_0014.jpg 329 | bocce image_0015.jpg 330 | bocce image_0016.jpg 331 | bocce image_0017.jpg 332 | bocce image_0018.jpg 333 | bocce image_0019.jpg 334 | bocce image_0020.jpg 335 | bocce image_0021.jpg 336 | bocce image_0022.jpg 337 | bocce image_0023.jpg 338 | bocce image_0024.jpg 339 | bocce image_0025.jpg 340 | bocce image_0026.jpg 341 | bocce image_0027.jpg 342 | bocce image_0028.jpg 343 | bocce image_0029.jpg 344 | bocce image_0030.jpg 345 | bocce image_0031.jpg 346 | bocce image_0032.jpg 347 | bocce image_0033.jpg 348 | bocce image_0034.jpg 349 | bocce image_0035.jpg 350 | bocce image_0036.jpg 351 | bocce image_0037.jpg 352 | bocce image_0038.jpg 353 | bocce image_0039.jpg 354 | bocce image_0040.jpg 355 | bocce image_0041.jpg 356 | bocce image_0042.jpg 357 | bocce image_0043.jpg 358 | bocce image_0044.jpg 359 | bocce image_0045.jpg 360 | bocce image_0046.jpg 361 | bocce image_0047.jpg 362 | bocce image_0048.jpg 363 | bocce image_0049.jpg 364 | bocce image_0050.jpg 365 | bocce image_0051.jpg 366 | bocce image_0052.jpg 367 | bocce image_0053.jpg 368 | bocce image_0054.jpg 369 | bocce image_0055.jpg 370 | bocce image_0056.jpg 371 | bocce image_0057.jpg 372 | bocce image_0058.jpg 373 | bocce image_0059.jpg 374 | bocce image_0060.jpg 375 | bocce image_0061.jpg 376 | bocce image_0062.jpg 377 | bocce image_0063.jpg 378 | bocce image_0064.jpg 379 | bocce image_0065.jpg 380 | bocce image_0066.jpg 381 | bocce image_0067.jpg 382 | bocce image_0068.jpg 383 | bocce image_0069.jpg 384 | bocce image_0070.jpg 385 | bocce image_0071.jpg 386 | bocce image_0072.jpg 387 | bocce image_0073.jpg 388 | bocce image_0074.jpg 389 | bocce image_0075.jpg 390 | bocce image_0076.jpg 391 | bocce image_0077.jpg 392 | bocce image_0078.jpg 393 | bocce image_0079.jpg 394 | bocce image_0080.jpg 395 | bocce image_0081.jpg 396 | bocce image_0082.jpg 397 | bocce image_0083.jpg 398 | bocce image_0084.jpg 399 | bocce image_0085.jpg 400 | bocce image_0086.jpg 401 | bocce image_0087.jpg 402 | bocce image_0088.jpg 403 | bocce image_0089.jpg 404 | bocce image_0090.jpg 405 | bocce image_0091.jpg 406 | bocce image_0092.jpg 407 | bocce image_0093.jpg 408 | bocce image_0094.jpg 409 | bocce image_0095.jpg 410 | bocce image_0096.jpg 411 | bocce image_0097.jpg 412 | bocce image_0098.jpg 413 | bocce image_0099.jpg 414 | bocce image_0100.jpg 415 | bocce image_0101.jpg 416 | bocce image_0102.jpg 417 | bocce image_0103.jpg 418 | bocce image_0104.jpg 419 | bocce image_0105.jpg 420 | bocce image_0106.jpg 421 | bocce image_0107.jpg 422 | bocce image_0108.jpg 423 | bocce image_0109.jpg 424 | bocce image_0110.jpg 425 | bocce image_0111.jpg 426 | bocce image_0112.jpg 427 | bocce image_0113.jpg 428 | bocce image_0114.jpg 429 | bocce image_0115.jpg 430 | bocce image_0116.jpg 431 | bocce image_0117.jpg 432 | bocce image_0118.jpg 433 | bocce image_0119.jpg 434 | bocce image_0120.jpg 435 | bocce image_0121.jpg 436 | bocce image_0122.jpg 437 | bocce image_0123.jpg 438 | bocce image_0124.jpg 439 | bocce image_0125.jpg 440 | bocce image_0126.jpg 441 | bocce image_0127.jpg 442 | bocce image_0128.jpg 443 | bocce image_0129.jpg 444 | bocce image_0130.jpg 445 | bocce image_0131.jpg 446 | bocce image_0132.jpg 447 | bocce image_0133.jpg 448 | bocce image_0134.jpg 449 | bocce image_0135.jpg 450 | bocce image_0136.jpg 451 | bocce image_0137.jpg 452 | croquet image_0001.jpg 453 | croquet image_0002.jpg 454 | croquet image_0003.jpg 455 | croquet image_0004.jpg 456 | croquet image_0005.jpg 457 | croquet image_0006.jpg 458 | croquet image_0007.jpg 459 | croquet image_0008.jpg 460 | croquet image_0009.jpg 461 | croquet image_0010.jpg 462 | croquet image_0011.jpg 463 | croquet image_0012.jpg 464 | croquet image_0013.jpg 465 | croquet image_0014.jpg 466 | croquet image_0015.jpg 467 | croquet image_0016.jpg 468 | croquet image_0017.jpg 469 | croquet image_0018.jpg 470 | croquet image_0019.jpg 471 | croquet image_0020.jpg 472 | croquet image_0021.jpg 473 | croquet image_0022.jpg 474 | croquet image_0023.jpg 475 | croquet image_0024.jpg 476 | croquet image_0025.jpg 477 | croquet image_0026.jpg 478 | croquet image_0027.jpg 479 | croquet image_0028.jpg 480 | croquet image_0029.jpg 481 | croquet image_0030.jpg 482 | croquet image_0031.jpg 483 | croquet image_0032.jpg 484 | croquet image_0033.jpg 485 | croquet image_0034.jpg 486 | croquet image_0035.jpg 487 | croquet image_0036.jpg 488 | croquet image_0037.jpg 489 | croquet image_0038.jpg 490 | croquet image_0039.jpg 491 | croquet image_0040.jpg 492 | croquet image_0041.jpg 493 | croquet image_0042.jpg 494 | croquet image_0043.jpg 495 | croquet image_0044.jpg 496 | croquet image_0045.jpg 497 | croquet image_0046.jpg 498 | croquet image_0047.jpg 499 | croquet image_0048.jpg 500 | croquet image_0049.jpg 501 | croquet image_0050.jpg 502 | croquet image_0051.jpg 503 | croquet image_0052.jpg 504 | croquet image_0053.jpg 505 | croquet image_0054.jpg 506 | croquet image_0055.jpg 507 | croquet image_0056.jpg 508 | croquet image_0057.jpg 509 | croquet image_0058.jpg 510 | croquet image_0059.jpg 511 | croquet image_0060.jpg 512 | croquet image_0061.jpg 513 | croquet image_0062.jpg 514 | croquet image_0063.jpg 515 | croquet image_0064.jpg 516 | croquet image_0065.jpg 517 | croquet image_0066.jpg 518 | croquet image_0067.jpg 519 | croquet image_0068.jpg 520 | croquet image_0069.jpg 521 | croquet image_0070.jpg 522 | croquet image_0071.jpg 523 | croquet image_0072.jpg 524 | croquet image_0073.jpg 525 | croquet image_0074.jpg 526 | croquet image_0075.jpg 527 | croquet image_0076.jpg 528 | croquet image_0077.jpg 529 | croquet image_0078.jpg 530 | croquet image_0079.jpg 531 | croquet image_0080.jpg 532 | croquet image_0081.jpg 533 | croquet image_0082.jpg 534 | croquet image_0083.jpg 535 | croquet image_0084.jpg 536 | croquet image_0085.jpg 537 | croquet image_0086.jpg 538 | croquet image_0087.jpg 539 | croquet image_0088.jpg 540 | croquet image_0089.jpg 541 | croquet image_0090.jpg 542 | croquet image_0091.jpg 543 | croquet image_0092.jpg 544 | croquet image_0093.jpg 545 | croquet image_0094.jpg 546 | croquet image_0095.jpg 547 | croquet image_0096.jpg 548 | croquet image_0097.jpg 549 | croquet image_0098.jpg 550 | croquet image_0099.jpg 551 | croquet image_0100.jpg 552 | croquet image_0101.jpg 553 | croquet image_0102.jpg 554 | croquet image_0103.jpg 555 | croquet image_0104.jpg 556 | croquet image_0105.jpg 557 | croquet image_0106.jpg 558 | croquet image_0107.jpg 559 | croquet image_0108.jpg 560 | croquet image_0109.jpg 561 | croquet image_0110.jpg 562 | croquet image_0111.jpg 563 | croquet image_0112.jpg 564 | croquet image_0113.jpg 565 | croquet image_0114.jpg 566 | croquet image_0115.jpg 567 | croquet image_0116.jpg 568 | croquet image_0117.jpg 569 | croquet image_0118.jpg 570 | croquet image_0119.jpg 571 | croquet image_0120.jpg 572 | croquet image_0121.jpg 573 | croquet image_0122.jpg 574 | croquet image_0123.jpg 575 | croquet image_0124.jpg 576 | croquet image_0125.jpg 577 | croquet image_0126.jpg 578 | croquet image_0127.jpg 579 | croquet image_0128.jpg 580 | croquet image_0129.jpg 581 | croquet image_0130.jpg 582 | croquet image_0131.jpg 583 | croquet image_0132.jpg 584 | croquet image_0133.jpg 585 | croquet image_0134.jpg 586 | croquet image_0135.jpg 587 | croquet image_0136.jpg 588 | croquet image_0137.jpg 589 | croquet image_0138.jpg 590 | croquet image_0139.jpg 591 | croquet image_0140.jpg 592 | croquet image_0141.jpg 593 | croquet image_0142.jpg 594 | croquet image_0143.jpg 595 | croquet image_0144.jpg 596 | croquet image_0145.jpg 597 | croquet image_0146.jpg 598 | croquet image_0147.jpg 599 | croquet image_0148.jpg 600 | croquet image_0149.jpg 601 | croquet image_0150.jpg 602 | croquet image_0151.jpg 603 | croquet image_0152.jpg 604 | croquet image_0153.jpg 605 | croquet image_0154.jpg 606 | croquet image_0155.jpg 607 | croquet image_0156.jpg 608 | croquet image_0157.jpg 609 | croquet image_0158.jpg 610 | croquet image_0159.jpg 611 | croquet image_0160.jpg 612 | croquet image_0161.jpg 613 | croquet image_0162.jpg 614 | croquet image_0163.jpg 615 | croquet image_0164.jpg 616 | croquet image_0165.jpg 617 | croquet image_0166.jpg 618 | croquet image_0167.jpg 619 | croquet image_0168.jpg 620 | croquet image_0169.jpg 621 | croquet image_0170.jpg 622 | croquet image_0171.jpg 623 | croquet image_0172.jpg 624 | croquet image_0173.jpg 625 | croquet image_0174.jpg 626 | croquet image_0175.jpg 627 | croquet image_0176.jpg 628 | croquet image_0177.jpg 629 | croquet image_0178.jpg 630 | croquet image_0179.jpg 631 | croquet image_0180.jpg 632 | croquet image_0181.jpg 633 | croquet image_0182.jpg 634 | croquet image_0183.jpg 635 | croquet image_0184.jpg 636 | croquet image_0185.jpg 637 | croquet image_0186.jpg 638 | croquet image_0187.jpg 639 | croquet image_0188.jpg 640 | croquet image_0189.jpg 641 | croquet image_0190.jpg 642 | croquet image_0191.jpg 643 | croquet image_0192.jpg 644 | croquet image_0193.jpg 645 | croquet image_0194.jpg 646 | croquet image_0195.jpg 647 | croquet image_0196.jpg 648 | croquet image_0197.jpg 649 | croquet image_0198.jpg 650 | croquet image_0199.jpg 651 | croquet image_0200.jpg 652 | croquet image_0201.jpg 653 | croquet image_0202.jpg 654 | croquet image_0203.jpg 655 | croquet image_0204.jpg 656 | croquet image_0205.jpg 657 | croquet image_0206.jpg 658 | croquet image_0207.jpg 659 | croquet image_0208.jpg 660 | croquet image_0209.jpg 661 | croquet image_0210.jpg 662 | croquet image_0211.jpg 663 | croquet image_0212.jpg 664 | croquet image_0213.jpg 665 | croquet image_0214.jpg 666 | croquet image_0215.jpg 667 | croquet image_0216.jpg 668 | croquet image_0217.jpg 669 | croquet image_0218.jpg 670 | croquet image_0219.jpg 671 | croquet image_0220.jpg 672 | croquet image_0221.jpg 673 | croquet image_0222.jpg 674 | croquet image_0223.jpg 675 | croquet image_0224.jpg 676 | croquet image_0225.jpg 677 | croquet image_0226.jpg 678 | croquet image_0227.jpg 679 | croquet image_0228.jpg 680 | croquet image_0229.jpg 681 | croquet image_0230.jpg 682 | croquet image_0231.jpg 683 | croquet image_0232.jpg 684 | croquet image_0233.jpg 685 | croquet image_0234.jpg 686 | croquet image_0235.jpg 687 | croquet image_0236.jpg 688 | croquet image_0237.jpg 689 | croquet image_0238.jpg 690 | croquet image_0239.jpg 691 | croquet image_0240.jpg 692 | croquet image_0241.jpg 693 | croquet image_0242.jpg 694 | croquet image_0243.jpg 695 | croquet image_0244.jpg 696 | croquet image_0245.jpg 697 | croquet image_0246.jpg 698 | croquet image_0247.jpg 699 | croquet image_0248.jpg 700 | croquet image_0249.jpg 701 | croquet image_0250.jpg 702 | croquet image_0251.jpg 703 | croquet image_0252.jpg 704 | croquet image_0253.jpg 705 | croquet image_0254.jpg 706 | croquet image_0255.jpg 707 | croquet image_0256.jpg 708 | croquet image_0257.jpg 709 | croquet image_0258.jpg 710 | croquet image_0259.jpg 711 | croquet image_0260.jpg 712 | croquet image_0261.jpg 713 | croquet image_0262.jpg 714 | croquet image_0263.jpg 715 | croquet image_0264.jpg 716 | croquet image_0265.jpg 717 | croquet image_0266.jpg 718 | croquet image_0267.jpg 719 | croquet image_0268.jpg 720 | croquet image_0269.jpg 721 | croquet image_0270.jpg 722 | croquet image_0271.jpg 723 | croquet image_0272.jpg 724 | croquet image_0273.jpg 725 | croquet image_0274.jpg 726 | croquet image_0275.jpg 727 | croquet image_0276.jpg 728 | croquet image_0277.jpg 729 | croquet image_0278.jpg 730 | croquet image_0279.jpg 731 | croquet image_0280.jpg 732 | croquet image_0281.jpg 733 | croquet image_0282.jpg 734 | croquet image_0283.jpg 735 | croquet image_0284.jpg 736 | croquet image_0285.jpg 737 | croquet image_0286.jpg 738 | croquet image_0287.jpg 739 | croquet image_0288.jpg 740 | croquet image_0289.jpg 741 | croquet image_0290.jpg 742 | croquet image_0291.jpg 743 | croquet image_0292.jpg 744 | croquet image_0293.jpg 745 | croquet image_0294.jpg 746 | croquet image_0295.jpg 747 | croquet image_0296.jpg 748 | croquet image_0297.jpg 749 | croquet image_0298.jpg 750 | croquet image_0299.jpg 751 | croquet image_0300.jpg 752 | croquet image_0301.jpg 753 | croquet image_0302.jpg 754 | croquet image_0303.jpg 755 | croquet image_0304.jpg 756 | croquet image_0305.jpg 757 | croquet image_0306.jpg 758 | croquet image_0307.jpg 759 | croquet image_0308.jpg 760 | croquet image_0309.jpg 761 | croquet image_0310.jpg 762 | croquet image_0311.jpg 763 | croquet image_0312.jpg 764 | croquet image_0313.jpg 765 | croquet image_0314.jpg 766 | croquet image_0315.jpg 767 | croquet image_0316.jpg 768 | croquet image_0317.jpg 769 | croquet image_0318.jpg 770 | croquet image_0319.jpg 771 | croquet image_0320.jpg 772 | croquet image_0321.jpg 773 | croquet image_0322.jpg 774 | croquet image_0323.jpg 775 | croquet image_0324.jpg 776 | croquet image_0325.jpg 777 | croquet image_0326.jpg 778 | croquet image_0327.jpg 779 | croquet image_0328.jpg 780 | croquet image_0329.jpg 781 | polo image_0001.jpg 782 | polo image_0002.jpg 783 | polo image_0003.jpg 784 | polo image_0004.jpg 785 | polo image_0005.jpg 786 | polo image_0006.jpg 787 | polo image_0007.jpg 788 | polo image_0008.jpg 789 | polo image_0009.jpg 790 | polo image_0010.jpg 791 | polo image_0011.jpg 792 | polo image_0012.jpg 793 | polo image_0013.jpg 794 | polo image_0014.jpg 795 | polo image_0015.jpg 796 | polo image_0016.jpg 797 | polo image_0017.jpg 798 | polo image_0018.jpg 799 | polo image_0019.jpg 800 | polo image_0020.jpg 801 | polo image_0021.jpg 802 | polo image_0022.jpg 803 | polo image_0023.jpg 804 | polo image_0024.jpg 805 | polo image_0025.jpg 806 | polo image_0026.jpg 807 | polo image_0027.jpg 808 | polo image_0028.jpg 809 | polo image_0029.jpg 810 | polo image_0030.jpg 811 | polo image_0031.jpg 812 | polo image_0032.jpg 813 | polo image_0033.jpg 814 | polo image_0034.jpg 815 | polo image_0035.jpg 816 | polo image_0036.jpg 817 | polo image_0037.jpg 818 | polo image_0038.jpg 819 | polo image_0039.jpg 820 | polo image_0040.jpg 821 | polo image_0041.jpg 822 | polo image_0042.jpg 823 | polo image_0043.jpg 824 | polo image_0044.jpg 825 | polo image_0045.jpg 826 | polo image_0046.jpg 827 | polo image_0047.jpg 828 | polo image_0048.jpg 829 | polo image_0049.jpg 830 | polo image_0050.jpg 831 | polo image_0051.jpg 832 | polo image_0052.jpg 833 | polo image_0053.jpg 834 | polo image_0054.jpg 835 | polo image_0055.jpg 836 | polo image_0056.jpg 837 | polo image_0057.jpg 838 | polo image_0058.jpg 839 | polo image_0059.jpg 840 | polo image_0060.jpg 841 | polo image_0061.jpg 842 | polo image_0062.jpg 843 | polo image_0063.jpg 844 | polo image_0064.jpg 845 | polo image_0065.jpg 846 | polo image_0066.jpg 847 | polo image_0067.jpg 848 | polo image_0068.jpg 849 | polo image_0069.jpg 850 | polo image_0070.jpg 851 | polo image_0071.jpg 852 | polo image_0072.jpg 853 | polo image_0073.jpg 854 | polo image_0074.jpg 855 | polo image_0075.jpg 856 | polo image_0076.jpg 857 | polo image_0077.jpg 858 | polo image_0078.jpg 859 | polo image_0079.jpg 860 | polo image_0080.jpg 861 | polo image_0081.jpg 862 | polo image_0082.jpg 863 | polo image_0083.jpg 864 | polo image_0084.jpg 865 | polo image_0085.jpg 866 | polo image_0086.jpg 867 | polo image_0087.jpg 868 | polo image_0088.jpg 869 | polo image_0089.jpg 870 | polo image_0090.jpg 871 | polo image_0091.jpg 872 | polo image_0092.jpg 873 | polo image_0093.jpg 874 | polo image_0094.jpg 875 | polo image_0095.jpg 876 | polo image_0096.jpg 877 | polo image_0097.jpg 878 | polo image_0098.jpg 879 | polo image_0099.jpg 880 | polo image_0100.jpg 881 | polo image_0101.jpg 882 | polo image_0102.jpg 883 | polo image_0103.jpg 884 | polo image_0104.jpg 885 | polo image_0105.jpg 886 | polo image_0106.jpg 887 | polo image_0107.jpg 888 | polo image_0108.jpg 889 | polo image_0109.jpg 890 | polo image_0110.jpg 891 | polo image_0111.jpg 892 | polo image_0112.jpg 893 | polo image_0113.jpg 894 | polo image_0114.jpg 895 | polo image_0115.jpg 896 | polo image_0116.jpg 897 | polo image_0117.jpg 898 | polo image_0118.jpg 899 | polo image_0119.jpg 900 | polo image_0120.jpg 901 | polo image_0121.jpg 902 | polo image_0122.jpg 903 | polo image_0123.jpg 904 | polo image_0124.jpg 905 | polo image_0125.jpg 906 | polo image_0126.jpg 907 | polo image_0127.jpg 908 | polo image_0128.jpg 909 | polo image_0129.jpg 910 | polo image_0130.jpg 911 | polo image_0131.jpg 912 | polo image_0132.jpg 913 | polo image_0133.jpg 914 | polo image_0134.jpg 915 | polo image_0135.jpg 916 | polo image_0136.jpg 917 | polo image_0137.jpg 918 | polo image_0138.jpg 919 | polo image_0139.jpg 920 | polo image_0140.jpg 921 | polo image_0141.jpg 922 | polo image_0142.jpg 923 | polo image_0143.jpg 924 | polo image_0144.jpg 925 | polo image_0145.jpg 926 | polo image_0146.jpg 927 | polo image_0147.jpg 928 | polo image_0148.jpg 929 | polo image_0149.jpg 930 | polo image_0150.jpg 931 | polo image_0151.jpg 932 | polo image_0152.jpg 933 | polo image_0153.jpg 934 | polo image_0154.jpg 935 | polo image_0155.jpg 936 | polo image_0156.jpg 937 | polo image_0157.jpg 938 | polo image_0158.jpg 939 | polo image_0159.jpg 940 | polo image_0160.jpg 941 | polo image_0161.jpg 942 | polo image_0162.jpg 943 | polo image_0163.jpg 944 | polo image_0164.jpg 945 | polo image_0165.jpg 946 | polo image_0166.jpg 947 | polo image_0167.jpg 948 | polo image_0168.jpg 949 | polo image_0169.jpg 950 | polo image_0170.jpg 951 | polo image_0171.jpg 952 | polo image_0172.jpg 953 | polo image_0173.jpg 954 | polo image_0174.jpg 955 | polo image_0175.jpg 956 | polo image_0176.jpg 957 | polo image_0177.jpg 958 | polo image_0178.jpg 959 | polo image_0179.jpg 960 | polo image_0180.jpg 961 | polo image_0181.jpg 962 | polo image_0182.jpg 963 | polo image_0183.jpg 964 | rockclimbing image_0001.jpg 965 | rockclimbing image_0002.jpg 966 | rockclimbing image_0003.jpg 967 | rockclimbing image_0004.jpg 968 | rockclimbing image_0005.jpg 969 | rockclimbing image_0006.jpg 970 | rockclimbing image_0007.jpg 971 | rockclimbing image_0008.jpg 972 | rockclimbing image_0009.jpg 973 | rockclimbing image_0010.jpg 974 | rockclimbing image_0011.jpg 975 | rockclimbing image_0012.jpg 976 | rockclimbing image_0013.jpg 977 | rockclimbing image_0014.jpg 978 | rockclimbing image_0015.jpg 979 | rockclimbing image_0016.jpg 980 | rockclimbing image_0017.jpg 981 | rockclimbing image_0018.jpg 982 | rockclimbing image_0019.jpg 983 | rockclimbing image_0020.jpg 984 | rockclimbing image_0021.jpg 985 | rockclimbing image_0022.jpg 986 | rockclimbing image_0023.jpg 987 | rockclimbing image_0024.jpg 988 | rockclimbing image_0025.jpg 989 | rockclimbing image_0026.jpg 990 | rockclimbing image_0027.jpg 991 | rockclimbing image_0028.jpg 992 | rockclimbing image_0029.jpg 993 | rockclimbing image_0030.jpg 994 | rockclimbing image_0031.jpg 995 | rockclimbing image_0032.jpg 996 | rockclimbing image_0033.jpg 997 | rockclimbing image_0034.jpg 998 | rockclimbing image_0035.jpg 999 | rockclimbing image_0036.jpg 1000 | rockclimbing image_0037.jpg 1001 | rockclimbing image_0038.jpg 1002 | rockclimbing image_0039.jpg 1003 | rockclimbing image_0040.jpg 1004 | rockclimbing image_0041.jpg 1005 | rockclimbing image_0042.jpg 1006 | rockclimbing image_0043.jpg 1007 | rockclimbing image_0044.jpg 1008 | rockclimbing image_0045.jpg 1009 | rockclimbing image_0046.jpg 1010 | rockclimbing image_0047.jpg 1011 | rockclimbing image_0048.jpg 1012 | rockclimbing image_0049.jpg 1013 | rockclimbing image_0050.jpg 1014 | rockclimbing image_0051.jpg 1015 | rockclimbing image_0052.jpg 1016 | rockclimbing image_0053.jpg 1017 | rockclimbing image_0054.jpg 1018 | rockclimbing image_0055.jpg 1019 | rockclimbing image_0056.jpg 1020 | rockclimbing image_0057.jpg 1021 | rockclimbing image_0058.jpg 1022 | rockclimbing image_0059.jpg 1023 | rockclimbing image_0060.jpg 1024 | rockclimbing image_0061.jpg 1025 | rockclimbing image_0062.jpg 1026 | rockclimbing image_0063.jpg 1027 | rockclimbing image_0064.jpg 1028 | rockclimbing image_0065.jpg 1029 | rockclimbing image_0066.jpg 1030 | rockclimbing image_0067.jpg 1031 | rockclimbing image_0068.jpg 1032 | rockclimbing image_0069.jpg 1033 | rockclimbing image_0070.jpg 1034 | rockclimbing image_0071.jpg 1035 | rockclimbing image_0072.jpg 1036 | rockclimbing image_0073.jpg 1037 | rockclimbing image_0074.jpg 1038 | rockclimbing image_0075.jpg 1039 | rockclimbing image_0076.jpg 1040 | rockclimbing image_0077.jpg 1041 | rockclimbing image_0078.jpg 1042 | rockclimbing image_0079.jpg 1043 | rockclimbing image_0080.jpg 1044 | rockclimbing image_0081.jpg 1045 | rockclimbing image_0082.jpg 1046 | rockclimbing image_0083.jpg 1047 | rockclimbing image_0084.jpg 1048 | rockclimbing image_0085.jpg 1049 | rockclimbing image_0086.jpg 1050 | rockclimbing image_0087.jpg 1051 | rockclimbing image_0088.jpg 1052 | rockclimbing image_0089.jpg 1053 | rockclimbing image_0090.jpg 1054 | rockclimbing image_0091.jpg 1055 | rockclimbing image_0092.jpg 1056 | rockclimbing image_0093.jpg 1057 | rockclimbing image_0094.jpg 1058 | rockclimbing image_0095.jpg 1059 | rockclimbing image_0096.jpg 1060 | rockclimbing image_0097.jpg 1061 | rockclimbing image_0098.jpg 1062 | rockclimbing image_0099.jpg 1063 | rockclimbing image_0100.jpg 1064 | rockclimbing image_0101.jpg 1065 | rockclimbing image_0102.jpg 1066 | rockclimbing image_0103.jpg 1067 | rockclimbing image_0104.jpg 1068 | rockclimbing image_0105.jpg 1069 | rockclimbing image_0106.jpg 1070 | rockclimbing image_0107.jpg 1071 | rockclimbing image_0108.jpg 1072 | rockclimbing image_0109.jpg 1073 | rockclimbing image_0110.jpg 1074 | rockclimbing image_0111.jpg 1075 | rockclimbing image_0112.jpg 1076 | rockclimbing image_0113.jpg 1077 | rockclimbing image_0114.jpg 1078 | rockclimbing image_0115.jpg 1079 | rockclimbing image_0116.jpg 1080 | rockclimbing image_0117.jpg 1081 | rockclimbing image_0118.jpg 1082 | rockclimbing image_0119.jpg 1083 | rockclimbing image_0120.jpg 1084 | rockclimbing image_0121.jpg 1085 | rockclimbing image_0122.jpg 1086 | rockclimbing image_0123.jpg 1087 | rockclimbing image_0124.jpg 1088 | rockclimbing image_0125.jpg 1089 | rockclimbing image_0126.jpg 1090 | rockclimbing image_0127.jpg 1091 | rockclimbing image_0128.jpg 1092 | rockclimbing image_0129.jpg 1093 | rockclimbing image_0130.jpg 1094 | rockclimbing image_0131.jpg 1095 | rockclimbing image_0132.jpg 1096 | rockclimbing image_0133.jpg 1097 | rockclimbing image_0134.jpg 1098 | rockclimbing image_0135.jpg 1099 | rockclimbing image_0136.jpg 1100 | rockclimbing image_0137.jpg 1101 | rockclimbing image_0138.jpg 1102 | rockclimbing image_0139.jpg 1103 | rockclimbing image_0140.jpg 1104 | rockclimbing image_0141.jpg 1105 | rockclimbing image_0142.jpg 1106 | rockclimbing image_0143.jpg 1107 | rockclimbing image_0144.jpg 1108 | rockclimbing image_0145.jpg 1109 | rockclimbing image_0146.jpg 1110 | rockclimbing image_0147.jpg 1111 | rockclimbing image_0148.jpg 1112 | rockclimbing image_0149.jpg 1113 | rockclimbing image_0150.jpg 1114 | rockclimbing image_0151.jpg 1115 | rockclimbing image_0152.jpg 1116 | rockclimbing image_0153.jpg 1117 | rockclimbing image_0154.jpg 1118 | rockclimbing image_0155.jpg 1119 | rockclimbing image_0156.jpg 1120 | rockclimbing image_0157.jpg 1121 | rockclimbing image_0158.jpg 1122 | rockclimbing image_0159.jpg 1123 | rockclimbing image_0160.jpg 1124 | rockclimbing image_0161.jpg 1125 | rockclimbing image_0162.jpg 1126 | rockclimbing image_0163.jpg 1127 | rockclimbing image_0164.jpg 1128 | rockclimbing image_0165.jpg 1129 | rockclimbing image_0166.jpg 1130 | rockclimbing image_0167.jpg 1131 | rockclimbing image_0168.jpg 1132 | rockclimbing image_0169.jpg 1133 | rockclimbing image_0170.jpg 1134 | rockclimbing image_0171.jpg 1135 | rockclimbing image_0172.jpg 1136 | rockclimbing image_0173.jpg 1137 | rockclimbing image_0174.jpg 1138 | rockclimbing image_0175.jpg 1139 | rockclimbing image_0176.jpg 1140 | rockclimbing image_0177.jpg 1141 | rockclimbing image_0178.jpg 1142 | rockclimbing image_0179.jpg 1143 | rockclimbing image_0180.jpg 1144 | rockclimbing image_0181.jpg 1145 | rockclimbing image_0182.jpg 1146 | rockclimbing image_0183.jpg 1147 | rockclimbing image_0184.jpg 1148 | rockclimbing image_0185.jpg 1149 | rockclimbing image_0186.jpg 1150 | rockclimbing image_0187.jpg 1151 | rockclimbing image_0188.jpg 1152 | rockclimbing image_0189.jpg 1153 | rockclimbing image_0190.jpg 1154 | rockclimbing image_0191.jpg 1155 | rockclimbing image_0192.jpg 1156 | rockclimbing image_0193.jpg 1157 | rockclimbing image_0194.jpg 1158 | rowing image_0001.jpg 1159 | rowing image_0002.jpg 1160 | rowing image_0003.jpg 1161 | rowing image_0004.jpg 1162 | rowing image_0005.jpg 1163 | rowing image_0006.jpg 1164 | rowing image_0007.jpg 1165 | rowing image_0008.jpg 1166 | rowing image_0009.jpg 1167 | rowing image_0010.jpg 1168 | rowing image_0011.jpg 1169 | rowing image_0012.jpg 1170 | rowing image_0013.jpg 1171 | rowing image_0014.jpg 1172 | rowing image_0015.jpg 1173 | rowing image_0016.jpg 1174 | rowing image_0017.jpg 1175 | rowing image_0018.jpg 1176 | rowing image_0019.jpg 1177 | rowing image_0020.jpg 1178 | rowing image_0021.jpg 1179 | rowing image_0022.jpg 1180 | rowing image_0023.jpg 1181 | rowing image_0024.jpg 1182 | rowing image_0025.jpg 1183 | rowing image_0026.jpg 1184 | rowing image_0027.jpg 1185 | rowing image_0028.jpg 1186 | rowing image_0029.jpg 1187 | rowing image_0030.jpg 1188 | rowing image_0031.jpg 1189 | rowing image_0032.jpg 1190 | rowing image_0033.jpg 1191 | rowing image_0034.jpg 1192 | rowing image_0035.jpg 1193 | rowing image_0036.jpg 1194 | rowing image_0037.jpg 1195 | rowing image_0038.jpg 1196 | rowing image_0039.jpg 1197 | rowing image_0040.jpg 1198 | rowing image_0041.jpg 1199 | rowing image_0042.jpg 1200 | rowing image_0043.jpg 1201 | rowing image_0044.jpg 1202 | rowing image_0045.jpg 1203 | rowing image_0046.jpg 1204 | rowing image_0047.jpg 1205 | rowing image_0048.jpg 1206 | rowing image_0049.jpg 1207 | rowing image_0050.jpg 1208 | rowing image_0051.jpg 1209 | rowing image_0052.jpg 1210 | rowing image_0053.jpg 1211 | rowing image_0054.jpg 1212 | rowing image_0055.jpg 1213 | rowing image_0056.jpg 1214 | rowing image_0057.jpg 1215 | rowing image_0058.jpg 1216 | rowing image_0059.jpg 1217 | rowing image_0060.jpg 1218 | rowing image_0061.jpg 1219 | rowing image_0062.jpg 1220 | rowing image_0063.jpg 1221 | rowing image_0064.jpg 1222 | rowing image_0065.jpg 1223 | rowing image_0066.jpg 1224 | rowing image_0067.jpg 1225 | rowing image_0068.jpg 1226 | rowing image_0069.jpg 1227 | rowing image_0070.jpg 1228 | rowing image_0071.jpg 1229 | rowing image_0072.jpg 1230 | rowing image_0073.jpg 1231 | rowing image_0074.jpg 1232 | rowing image_0075.jpg 1233 | rowing image_0076.jpg 1234 | rowing image_0077.jpg 1235 | rowing image_0078.jpg 1236 | rowing image_0079.jpg 1237 | rowing image_0080.jpg 1238 | rowing image_0081.jpg 1239 | rowing image_0082.jpg 1240 | rowing image_0083.jpg 1241 | rowing image_0084.jpg 1242 | rowing image_0085.jpg 1243 | rowing image_0086.jpg 1244 | rowing image_0087.jpg 1245 | rowing image_0088.jpg 1246 | rowing image_0089.jpg 1247 | rowing image_0090.jpg 1248 | rowing image_0091.jpg 1249 | rowing image_0092.jpg 1250 | rowing image_0093.jpg 1251 | rowing image_0094.jpg 1252 | rowing image_0095.jpg 1253 | rowing image_0096.jpg 1254 | rowing image_0097.jpg 1255 | rowing image_0098.jpg 1256 | rowing image_0099.jpg 1257 | rowing image_0100.jpg 1258 | rowing image_0101.jpg 1259 | rowing image_0102.jpg 1260 | rowing image_0103.jpg 1261 | rowing image_0104.jpg 1262 | rowing image_0105.jpg 1263 | rowing image_0106.jpg 1264 | rowing image_0107.jpg 1265 | rowing image_0108.jpg 1266 | rowing image_0109.jpg 1267 | rowing image_0110.jpg 1268 | rowing image_0111.jpg 1269 | rowing image_0112.jpg 1270 | rowing image_0113.jpg 1271 | rowing image_0114.jpg 1272 | rowing image_0115.jpg 1273 | rowing image_0116.jpg 1274 | rowing image_0117.jpg 1275 | rowing image_0118.jpg 1276 | rowing image_0119.jpg 1277 | rowing image_0120.jpg 1278 | rowing image_0121.jpg 1279 | rowing image_0122.jpg 1280 | rowing image_0123.jpg 1281 | rowing image_0124.jpg 1282 | rowing image_0125.jpg 1283 | rowing image_0126.jpg 1284 | rowing image_0127.jpg 1285 | rowing image_0128.jpg 1286 | rowing image_0129.jpg 1287 | rowing image_0130.jpg 1288 | rowing image_0131.jpg 1289 | rowing image_0132.jpg 1290 | rowing image_0133.jpg 1291 | rowing image_0134.jpg 1292 | rowing image_0135.jpg 1293 | rowing image_0136.jpg 1294 | rowing image_0137.jpg 1295 | rowing image_0138.jpg 1296 | rowing image_0139.jpg 1297 | rowing image_0140.jpg 1298 | rowing image_0141.jpg 1299 | rowing image_0142.jpg 1300 | rowing image_0143.jpg 1301 | rowing image_0144.jpg 1302 | rowing image_0145.jpg 1303 | rowing image_0146.jpg 1304 | rowing image_0147.jpg 1305 | rowing image_0148.jpg 1306 | rowing image_0149.jpg 1307 | rowing image_0150.jpg 1308 | rowing image_0151.jpg 1309 | rowing image_0152.jpg 1310 | rowing image_0153.jpg 1311 | rowing image_0154.jpg 1312 | rowing image_0155.jpg 1313 | rowing image_0156.jpg 1314 | rowing image_0157.jpg 1315 | rowing image_0158.jpg 1316 | rowing image_0159.jpg 1317 | rowing image_0160.jpg 1318 | rowing image_0161.jpg 1319 | rowing image_0162.jpg 1320 | rowing image_0163.jpg 1321 | rowing image_0164.jpg 1322 | rowing image_0165.jpg 1323 | rowing image_0166.jpg 1324 | rowing image_0167.jpg 1325 | rowing image_0168.jpg 1326 | rowing image_0169.jpg 1327 | rowing image_0170.jpg 1328 | rowing image_0171.jpg 1329 | rowing image_0172.jpg 1330 | rowing image_0173.jpg 1331 | rowing image_0174.jpg 1332 | rowing image_0175.jpg 1333 | rowing image_0176.jpg 1334 | rowing image_0177.jpg 1335 | rowing image_0178.jpg 1336 | rowing image_0179.jpg 1337 | rowing image_0180.jpg 1338 | rowing image_0181.jpg 1339 | rowing image_0182.jpg 1340 | rowing image_0183.jpg 1341 | rowing image_0184.jpg 1342 | rowing image_0185.jpg 1343 | rowing image_0186.jpg 1344 | rowing image_0187.jpg 1345 | rowing image_0188.jpg 1346 | rowing image_0189.jpg 1347 | rowing image_0190.jpg 1348 | rowing image_0191.jpg 1349 | rowing image_0192.jpg 1350 | rowing image_0193.jpg 1351 | rowing image_0194.jpg 1352 | rowing image_0195.jpg 1353 | rowing image_0196.jpg 1354 | rowing image_0197.jpg 1355 | rowing image_0198.jpg 1356 | rowing image_0199.jpg 1357 | rowing image_0200.jpg 1358 | rowing image_0201.jpg 1359 | rowing image_0202.jpg 1360 | rowing image_0203.jpg 1361 | rowing image_0204.jpg 1362 | rowing image_0205.jpg 1363 | rowing image_0206.jpg 1364 | rowing image_0207.jpg 1365 | rowing image_0208.jpg 1366 | rowing image_0209.jpg 1367 | rowing image_0210.jpg 1368 | rowing image_0211.jpg 1369 | rowing image_0212.jpg 1370 | rowing image_0213.jpg 1371 | rowing image_0214.jpg 1372 | rowing image_0215.jpg 1373 | rowing image_0216.jpg 1374 | rowing image_0217.jpg 1375 | rowing image_0218.jpg 1376 | rowing image_0219.jpg 1377 | rowing image_0220.jpg 1378 | rowing image_0221.jpg 1379 | rowing image_0222.jpg 1380 | rowing image_0223.jpg 1381 | rowing image_0224.jpg 1382 | rowing image_0225.jpg 1383 | rowing image_0226.jpg 1384 | rowing image_0227.jpg 1385 | rowing image_0228.jpg 1386 | rowing image_0229.jpg 1387 | rowing image_0230.jpg 1388 | rowing image_0231.jpg 1389 | rowing image_0232.jpg 1390 | rowing image_0233.jpg 1391 | rowing image_0234.jpg 1392 | rowing image_0235.jpg 1393 | rowing image_0236.jpg 1394 | rowing image_0237.jpg 1395 | rowing image_0238.jpg 1396 | rowing image_0239.jpg 1397 | rowing image_0240.jpg 1398 | rowing image_0241.jpg 1399 | rowing image_0242.jpg 1400 | rowing image_0243.jpg 1401 | rowing image_0244.jpg 1402 | rowing image_0245.jpg 1403 | rowing image_0246.jpg 1404 | rowing image_0247.jpg 1405 | rowing image_0248.jpg 1406 | rowing image_0249.jpg 1407 | rowing image_0250.jpg 1408 | rowing image_0251.jpg 1409 | rowing image_0252.jpg 1410 | rowing image_0253.jpg 1411 | rowing image_0254.jpg 1412 | rowing image_0255.jpg 1413 | sailing image_0001.jpg 1414 | sailing image_0002.jpg 1415 | sailing image_0003.jpg 1416 | sailing image_0004.jpg 1417 | sailing image_0005.jpg 1418 | sailing image_0006.jpg 1419 | sailing image_0007.jpg 1420 | sailing image_0008.jpg 1421 | sailing image_0009.jpg 1422 | sailing image_0010.jpg 1423 | sailing image_0011.jpg 1424 | sailing image_0012.jpg 1425 | sailing image_0013.jpg 1426 | sailing image_0014.jpg 1427 | sailing image_0015.jpg 1428 | sailing image_0016.jpg 1429 | sailing image_0017.jpg 1430 | sailing image_0018.jpg 1431 | sailing image_0019.jpg 1432 | sailing image_0020.jpg 1433 | sailing image_0021.jpg 1434 | sailing image_0022.jpg 1435 | sailing image_0023.jpg 1436 | sailing image_0024.jpg 1437 | sailing image_0025.jpg 1438 | sailing image_0026.jpg 1439 | sailing image_0027.jpg 1440 | sailing image_0028.jpg 1441 | sailing image_0029.jpg 1442 | sailing image_0030.jpg 1443 | sailing image_0031.jpg 1444 | sailing image_0032.jpg 1445 | sailing image_0033.jpg 1446 | sailing image_0034.jpg 1447 | sailing image_0035.jpg 1448 | sailing image_0036.jpg 1449 | sailing image_0037.jpg 1450 | sailing image_0038.jpg 1451 | sailing image_0039.jpg 1452 | sailing image_0040.jpg 1453 | sailing image_0041.jpg 1454 | sailing image_0042.jpg 1455 | sailing image_0043.jpg 1456 | sailing image_0044.jpg 1457 | sailing image_0045.jpg 1458 | sailing image_0046.jpg 1459 | sailing image_0047.jpg 1460 | sailing image_0048.jpg 1461 | sailing image_0049.jpg 1462 | sailing image_0050.jpg 1463 | sailing image_0051.jpg 1464 | sailing image_0052.jpg 1465 | sailing image_0053.jpg 1466 | sailing image_0054.jpg 1467 | sailing image_0055.jpg 1468 | sailing image_0056.jpg 1469 | sailing image_0057.jpg 1470 | sailing image_0058.jpg 1471 | sailing image_0059.jpg 1472 | sailing image_0060.jpg 1473 | sailing image_0061.jpg 1474 | sailing image_0062.jpg 1475 | sailing image_0063.jpg 1476 | sailing image_0064.jpg 1477 | sailing image_0065.jpg 1478 | sailing image_0066.jpg 1479 | sailing image_0067.jpg 1480 | sailing image_0068.jpg 1481 | sailing image_0069.jpg 1482 | sailing image_0070.jpg 1483 | sailing image_0071.jpg 1484 | sailing image_0072.jpg 1485 | sailing image_0073.jpg 1486 | sailing image_0074.jpg 1487 | sailing image_0075.jpg 1488 | sailing image_0076.jpg 1489 | sailing image_0077.jpg 1490 | sailing image_0078.jpg 1491 | sailing image_0079.jpg 1492 | sailing image_0080.jpg 1493 | sailing image_0081.jpg 1494 | sailing image_0082.jpg 1495 | sailing image_0083.jpg 1496 | sailing image_0084.jpg 1497 | sailing image_0085.jpg 1498 | sailing image_0086.jpg 1499 | sailing image_0087.jpg 1500 | sailing image_0088.jpg 1501 | sailing image_0089.jpg 1502 | sailing image_0090.jpg 1503 | sailing image_0091.jpg 1504 | sailing image_0092.jpg 1505 | sailing image_0093.jpg 1506 | sailing image_0094.jpg 1507 | sailing image_0095.jpg 1508 | sailing image_0096.jpg 1509 | sailing image_0097.jpg 1510 | sailing image_0098.jpg 1511 | sailing image_0099.jpg 1512 | sailing image_0100.jpg 1513 | sailing image_0101.jpg 1514 | sailing image_0102.jpg 1515 | sailing image_0103.jpg 1516 | sailing image_0104.jpg 1517 | sailing image_0105.jpg 1518 | sailing image_0106.jpg 1519 | sailing image_0107.jpg 1520 | sailing image_0108.jpg 1521 | sailing image_0109.jpg 1522 | sailing image_0110.jpg 1523 | sailing image_0111.jpg 1524 | sailing image_0112.jpg 1525 | sailing image_0113.jpg 1526 | sailing image_0114.jpg 1527 | sailing image_0115.jpg 1528 | sailing image_0116.jpg 1529 | sailing image_0117.jpg 1530 | sailing image_0118.jpg 1531 | sailing image_0119.jpg 1532 | sailing image_0120.jpg 1533 | sailing image_0121.jpg 1534 | sailing image_0122.jpg 1535 | sailing image_0123.jpg 1536 | sailing image_0124.jpg 1537 | sailing image_0125.jpg 1538 | sailing image_0126.jpg 1539 | sailing image_0127.jpg 1540 | sailing image_0128.jpg 1541 | sailing image_0129.jpg 1542 | sailing image_0130.jpg 1543 | sailing image_0131.jpg 1544 | sailing image_0132.jpg 1545 | sailing image_0133.jpg 1546 | sailing image_0134.jpg 1547 | sailing image_0135.jpg 1548 | sailing image_0136.jpg 1549 | sailing image_0137.jpg 1550 | sailing image_0138.jpg 1551 | sailing image_0139.jpg 1552 | sailing image_0140.jpg 1553 | sailing image_0141.jpg 1554 | sailing image_0142.jpg 1555 | sailing image_0143.jpg 1556 | sailing image_0144.jpg 1557 | sailing image_0145.jpg 1558 | sailing image_0146.jpg 1559 | sailing image_0147.jpg 1560 | sailing image_0148.jpg 1561 | sailing image_0149.jpg 1562 | sailing image_0150.jpg 1563 | sailing image_0151.jpg 1564 | sailing image_0152.jpg 1565 | sailing image_0153.jpg 1566 | sailing image_0154.jpg 1567 | sailing image_0155.jpg 1568 | sailing image_0156.jpg 1569 | sailing image_0157.jpg 1570 | sailing image_0158.jpg 1571 | sailing image_0159.jpg 1572 | sailing image_0160.jpg 1573 | sailing image_0161.jpg 1574 | sailing image_0162.jpg 1575 | sailing image_0163.jpg 1576 | sailing image_0164.jpg 1577 | sailing image_0165.jpg 1578 | sailing image_0166.jpg 1579 | sailing image_0167.jpg 1580 | sailing image_0168.jpg 1581 | sailing image_0169.jpg 1582 | sailing image_0170.jpg 1583 | sailing image_0171.jpg 1584 | sailing image_0172.jpg 1585 | sailing image_0173.jpg 1586 | sailing image_0174.jpg 1587 | sailing image_0175.jpg 1588 | sailing image_0176.jpg 1589 | sailing image_0177.jpg 1590 | sailing image_0178.jpg 1591 | sailing image_0179.jpg 1592 | sailing image_0180.jpg 1593 | sailing image_0181.jpg 1594 | sailing image_0182.jpg 1595 | sailing image_0183.jpg 1596 | sailing image_0184.jpg 1597 | sailing image_0185.jpg 1598 | sailing image_0186.jpg 1599 | sailing image_0187.jpg 1600 | sailing image_0188.jpg 1601 | sailing image_0189.jpg 1602 | sailing image_0190.jpg 1603 | snowboarding image_0001.jpg 1604 | snowboarding image_0002.jpg 1605 | snowboarding image_0003.jpg 1606 | snowboarding image_0004.jpg 1607 | snowboarding image_0005.jpg 1608 | snowboarding image_0006.jpg 1609 | snowboarding image_0007.jpg 1610 | snowboarding image_0008.jpg 1611 | snowboarding image_0009.jpg 1612 | snowboarding image_0010.jpg 1613 | snowboarding image_0011.jpg 1614 | snowboarding image_0012.jpg 1615 | snowboarding image_0013.jpg 1616 | snowboarding image_0014.jpg 1617 | snowboarding image_0015.jpg 1618 | snowboarding image_0016.jpg 1619 | snowboarding image_0017.jpg 1620 | snowboarding image_0018.jpg 1621 | snowboarding image_0019.jpg 1622 | snowboarding image_0020.jpg 1623 | snowboarding image_0021.jpg 1624 | snowboarding image_0022.jpg 1625 | snowboarding image_0023.jpg 1626 | snowboarding image_0024.jpg 1627 | snowboarding image_0025.jpg 1628 | snowboarding image_0026.jpg 1629 | snowboarding image_0027.jpg 1630 | snowboarding image_0028.jpg 1631 | snowboarding image_0029.jpg 1632 | snowboarding image_0030.jpg 1633 | snowboarding image_0031.jpg 1634 | snowboarding image_0032.jpg 1635 | snowboarding image_0033.jpg 1636 | snowboarding image_0034.jpg 1637 | snowboarding image_0035.jpg 1638 | snowboarding image_0036.jpg 1639 | snowboarding image_0037.jpg 1640 | snowboarding image_0038.jpg 1641 | snowboarding image_0039.jpg 1642 | snowboarding image_0040.jpg 1643 | snowboarding image_0041.jpg 1644 | snowboarding image_0042.jpg 1645 | snowboarding image_0043.jpg 1646 | snowboarding image_0044.jpg 1647 | snowboarding image_0045.jpg 1648 | snowboarding image_0046.jpg 1649 | snowboarding image_0047.jpg 1650 | snowboarding image_0048.jpg 1651 | snowboarding image_0049.jpg 1652 | snowboarding image_0050.jpg 1653 | snowboarding image_0051.jpg 1654 | snowboarding image_0052.jpg 1655 | snowboarding image_0053.jpg 1656 | snowboarding image_0054.jpg 1657 | snowboarding image_0055.jpg 1658 | snowboarding image_0056.jpg 1659 | snowboarding image_0057.jpg 1660 | snowboarding image_0058.jpg 1661 | snowboarding image_0059.jpg 1662 | snowboarding image_0060.jpg 1663 | snowboarding image_0061.jpg 1664 | snowboarding image_0062.jpg 1665 | snowboarding image_0063.jpg 1666 | snowboarding image_0064.jpg 1667 | snowboarding image_0065.jpg 1668 | snowboarding image_0066.jpg 1669 | snowboarding image_0067.jpg 1670 | snowboarding image_0068.jpg 1671 | snowboarding image_0069.jpg 1672 | snowboarding image_0070.jpg 1673 | snowboarding image_0071.jpg 1674 | snowboarding image_0072.jpg 1675 | snowboarding image_0073.jpg 1676 | snowboarding image_0074.jpg 1677 | snowboarding image_0075.jpg 1678 | snowboarding image_0076.jpg 1679 | snowboarding image_0077.jpg 1680 | snowboarding image_0078.jpg 1681 | snowboarding image_0079.jpg 1682 | snowboarding image_0080.jpg 1683 | snowboarding image_0081.jpg 1684 | snowboarding image_0082.jpg 1685 | snowboarding image_0083.jpg 1686 | snowboarding image_0084.jpg 1687 | snowboarding image_0085.jpg 1688 | snowboarding image_0086.jpg 1689 | snowboarding image_0087.jpg 1690 | snowboarding image_0088.jpg 1691 | snowboarding image_0089.jpg 1692 | snowboarding image_0090.jpg 1693 | snowboarding image_0091.jpg 1694 | snowboarding image_0092.jpg 1695 | snowboarding image_0093.jpg 1696 | snowboarding image_0094.jpg 1697 | snowboarding image_0095.jpg 1698 | snowboarding image_0096.jpg 1699 | snowboarding image_0097.jpg 1700 | snowboarding image_0098.jpg 1701 | snowboarding image_0099.jpg 1702 | snowboarding image_0100.jpg 1703 | snowboarding image_0101.jpg 1704 | snowboarding image_0102.jpg 1705 | snowboarding image_0103.jpg 1706 | snowboarding image_0104.jpg 1707 | snowboarding image_0105.jpg 1708 | snowboarding image_0106.jpg 1709 | snowboarding image_0107.jpg 1710 | snowboarding image_0108.jpg 1711 | snowboarding image_0109.jpg 1712 | snowboarding image_0110.jpg 1713 | snowboarding image_0111.jpg 1714 | snowboarding image_0112.jpg 1715 | snowboarding image_0113.jpg 1716 | snowboarding image_0114.jpg 1717 | snowboarding image_0115.jpg 1718 | snowboarding image_0116.jpg 1719 | snowboarding image_0117.jpg 1720 | snowboarding image_0118.jpg 1721 | snowboarding image_0119.jpg 1722 | snowboarding image_0120.jpg 1723 | snowboarding image_0121.jpg 1724 | snowboarding image_0122.jpg 1725 | snowboarding image_0123.jpg 1726 | snowboarding image_0124.jpg 1727 | snowboarding image_0125.jpg 1728 | snowboarding image_0126.jpg 1729 | snowboarding image_0127.jpg 1730 | snowboarding image_0128.jpg 1731 | snowboarding image_0129.jpg 1732 | snowboarding image_0130.jpg 1733 | snowboarding image_0131.jpg 1734 | snowboarding image_0132.jpg 1735 | snowboarding image_0133.jpg 1736 | snowboarding image_0134.jpg 1737 | snowboarding image_0135.jpg 1738 | snowboarding image_0136.jpg 1739 | snowboarding image_0137.jpg 1740 | snowboarding image_0138.jpg 1741 | snowboarding image_0139.jpg 1742 | snowboarding image_0140.jpg 1743 | snowboarding image_0141.jpg 1744 | snowboarding image_0142.jpg 1745 | snowboarding image_0143.jpg 1746 | snowboarding image_0144.jpg 1747 | snowboarding image_0145.jpg 1748 | snowboarding image_0146.jpg 1749 | snowboarding image_0147.jpg 1750 | snowboarding image_0148.jpg 1751 | snowboarding image_0149.jpg 1752 | snowboarding image_0150.jpg 1753 | snowboarding image_0151.jpg 1754 | snowboarding image_0152.jpg 1755 | snowboarding image_0153.jpg 1756 | snowboarding image_0154.jpg 1757 | snowboarding image_0155.jpg 1758 | snowboarding image_0156.jpg 1759 | snowboarding image_0157.jpg 1760 | snowboarding image_0158.jpg 1761 | snowboarding image_0159.jpg 1762 | snowboarding image_0160.jpg 1763 | snowboarding image_0161.jpg 1764 | snowboarding image_0162.jpg 1765 | snowboarding image_0163.jpg 1766 | snowboarding image_0164.jpg 1767 | snowboarding image_0165.jpg 1768 | snowboarding image_0166.jpg 1769 | snowboarding image_0167.jpg 1770 | snowboarding image_0168.jpg 1771 | snowboarding image_0169.jpg 1772 | snowboarding image_0170.jpg 1773 | snowboarding image_0171.jpg 1774 | snowboarding image_0172.jpg 1775 | snowboarding image_0173.jpg 1776 | snowboarding image_0174.jpg 1777 | snowboarding image_0175.jpg 1778 | snowboarding image_0176.jpg 1779 | snowboarding image_0177.jpg 1780 | snowboarding image_0178.jpg 1781 | snowboarding image_0179.jpg 1782 | snowboarding image_0180.jpg 1783 | snowboarding image_0181.jpg 1784 | snowboarding image_0182.jpg 1785 | snowboarding image_0183.jpg 1786 | snowboarding image_0184.jpg 1787 | snowboarding image_0185.jpg 1788 | snowboarding image_0186.jpg 1789 | snowboarding image_0187.jpg 1790 | snowboarding image_0188.jpg 1791 | snowboarding image_0189.jpg 1792 | snowboarding image_0190.jpg 1793 | -------------------------------------------------------------------------------- /settings.h: -------------------------------------------------------------------------------- 1 | // (C) Copyright 2009, Chong Wang, David Blei and Li Fei-Fei 2 | 3 | // written by Chong Wang, chongw@cs.princeton.edu 4 | 5 | // This file is part of slda. 6 | 7 | // slda is free software; you can redistribute it and/or modify it under 8 | // the terms of the GNU General Public License as published by the Free 9 | // Software Foundation; either version 2 of the License, or (at your 10 | // option) any later version. 11 | 12 | // slda is distributed in the hope that it will be useful, but WITHOUT 13 | // ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 14 | // FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License 15 | // for more details. 16 | 17 | // You should have received a copy of the GNU General Public License 18 | // along with this program; if not, write to the Free Software 19 | // Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 20 | // USA 21 | #ifndef SETTINGS_H 22 | #define SETTINGS_H 23 | #include 24 | #include 25 | 26 | struct settings 27 | { 28 | float VAR_CONVERGED; 29 | int VAR_MAX_ITER; 30 | float EM_CONVERGED; 31 | int EM_MAX_ITER; 32 | int ESTIMATE_ALPHA; 33 | float PENALTY; 34 | 35 | void read_settings(char* filename) 36 | { 37 | FILE * fileptr; 38 | char alpha_action[100]; 39 | 40 | fileptr = fopen(filename, "r"); 41 | fscanf(fileptr, "var max iter %d\n", &this->VAR_MAX_ITER); 42 | fscanf(fileptr, "var convergence %f\n", &this->VAR_CONVERGED); 43 | fscanf(fileptr, "em max iter %d\n", &this->EM_MAX_ITER); 44 | fscanf(fileptr, "em convergence %f\n", &this->EM_CONVERGED); 45 | fscanf(fileptr, "L2 penalty %f\n", &this->PENALTY); 46 | 47 | fscanf(fileptr, "alpha %s", alpha_action); 48 | if (strcmp(alpha_action, "fixed") == 0) 49 | { 50 | this->ESTIMATE_ALPHA = 0; 51 | printf("alpha is fixed ...\n"); 52 | } 53 | else 54 | { 55 | this->ESTIMATE_ALPHA = 1; 56 | printf("alpha is esimated ...\n"); 57 | } 58 | fclose(fileptr); 59 | printf("var max iter %d\n", this->VAR_MAX_ITER); 60 | printf("var convergence %.2E\n", this->VAR_CONVERGED); 61 | printf("em max iter %d\n", this->EM_MAX_ITER); 62 | printf("em convergence %.2E\n", this->EM_CONVERGED); 63 | printf("L2 penalty %.2E\n", this->PENALTY); 64 | } 65 | }; 66 | 67 | #endif // SETTINGS_H 68 | 69 | -------------------------------------------------------------------------------- /settings.txt: -------------------------------------------------------------------------------- 1 | var max iter 20 2 | var convergence 1e-3 3 | em max iter 50 4 | em convergence 1e-4 5 | L2 penalty 0.01 6 | alpha fixed 7 | -------------------------------------------------------------------------------- /slda.cpp: -------------------------------------------------------------------------------- 1 | // (C) Copyright 2009, Chong Wang, David Blei and Li Fei-Fei 2 | 3 | // written by Chong Wang, chongw@cs.princeton.edu 4 | 5 | // This file is part of slda. 6 | 7 | // slda is free software; you can redistribute it and/or modify it under 8 | // the terms of the GNU General Public License as published by the Free 9 | // Software Foundation; either version 2 of the License, or (at your 10 | // option) any later version. 11 | 12 | // slda is distributed in the hope that it will be useful, but WITHOUT 13 | // ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 14 | // FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License 15 | // for more details. 16 | 17 | // You should have received a copy of the GNU General Public License 18 | // along with this program; if not, write to the Free Software 19 | // Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 20 | // USA 21 | 22 | #include "slda.h" 23 | #include 24 | #include "utils.h" 25 | #include "assert.h" 26 | #include "opt.h" 27 | #include 28 | #include 29 | #include 30 | 31 | const int NUM_INIT = 50; 32 | const int LAG = 10; 33 | const int LDA_INIT_MAX = 0; 34 | const int MSTEP_MAX_ITER = 50; 35 | 36 | slda::slda() 37 | { 38 | //ctor 39 | alpha = 1.0; 40 | num_topics = 0; 41 | num_classes = 0; 42 | size_vocab = 0; 43 | 44 | log_prob_w = NULL; 45 | eta = NULL; 46 | } 47 | 48 | slda::~slda() 49 | { 50 | free_model(); 51 | } 52 | 53 | /* 54 | * init the model 55 | */ 56 | 57 | void slda::init(double alpha_, int num_topics_, 58 | const corpus * c) 59 | { 60 | alpha = alpha_; 61 | num_topics = num_topics_; 62 | size_vocab = c->size_vocab; 63 | num_classes = c->num_classes; 64 | 65 | log_prob_w = new double * [num_topics]; 66 | for (int k = 0; k < num_topics; k++) 67 | { 68 | log_prob_w[k] = new double [size_vocab]; 69 | memset(log_prob_w[k], 0, sizeof(double)*size_vocab); 70 | } 71 | //no need to train slda if we only have on class 72 | if (num_classes > 1) 73 | { 74 | eta = new double * [num_classes-1]; 75 | for (int i = 0; i < num_classes-1; i ++) 76 | { 77 | eta[i] = new double [num_topics]; 78 | memset(eta[i], 0, sizeof(double)*num_topics); 79 | } 80 | } 81 | } 82 | 83 | /* 84 | * free the model 85 | */ 86 | 87 | void slda::free_model() 88 | { 89 | if (log_prob_w != NULL) 90 | { 91 | for (int k = 0; k < num_topics; k++) 92 | { 93 | delete [] log_prob_w[k]; 94 | } 95 | delete [] log_prob_w; 96 | log_prob_w = NULL; 97 | } 98 | if (eta != NULL) 99 | { 100 | for (int i = 0; i < num_classes-1; i ++) 101 | { 102 | delete [] eta[i]; 103 | } 104 | delete [] eta; 105 | eta = NULL; 106 | } 107 | } 108 | 109 | /* 110 | * save the model in the binary format 111 | */ 112 | 113 | void slda::save_model(const char * filename) 114 | { 115 | FILE * file = NULL; 116 | file = fopen(filename, "wb"); 117 | fwrite(&alpha, sizeof (double), 1, file); 118 | fwrite(&num_topics, sizeof (int), 1, file); 119 | fwrite(&size_vocab, sizeof (int), 1, file); 120 | fwrite(&num_classes, sizeof (int), 1, file); 121 | 122 | for (int k = 0; k < num_topics; k++) 123 | { 124 | fwrite(log_prob_w[k], sizeof(double), size_vocab, file); 125 | } 126 | if (num_classes > 1) 127 | { 128 | for (int i = 0; i < num_classes-1; i ++) 129 | { 130 | fwrite(eta[i], sizeof(double), num_topics, file); 131 | } 132 | } 133 | 134 | fflush(file); 135 | fclose(file); 136 | } 137 | 138 | /* 139 | * load the model in the binary format 140 | */ 141 | 142 | void slda::load_model(const char * filename) 143 | { 144 | FILE * file = NULL; 145 | file = fopen(filename, "rb"); 146 | fread(&alpha, sizeof (double), 1, file); 147 | fread(&num_topics, sizeof (int), 1, file); 148 | fread(&size_vocab, sizeof (int), 1, file); 149 | fread(&num_classes, sizeof (int), 1, file); 150 | 151 | log_prob_w = new double * [num_topics]; 152 | for (int k = 0; k < num_topics; k++) 153 | { 154 | log_prob_w[k] = new double [size_vocab]; 155 | fread(log_prob_w[k], sizeof(double), size_vocab, file); 156 | } 157 | if (num_classes > 1) 158 | { 159 | eta = new double * [num_classes-1]; 160 | for (int i = 0; i < num_classes-1; i ++) 161 | { 162 | eta[i] = new double [num_topics]; 163 | fread(eta[i], sizeof(double), num_topics, file); 164 | } 165 | } 166 | 167 | fflush(file); 168 | fclose(file); 169 | } 170 | 171 | /* 172 | * save the model in the text format 173 | */ 174 | 175 | void slda::save_model_text(const char * filename) 176 | { 177 | FILE * file = NULL; 178 | file = fopen(filename, "w"); 179 | fprintf(file, "alpha: %lf\n", alpha); 180 | fprintf(file, "number of topics: %d\n", num_topics); 181 | fprintf(file, "size of vocab: %d\n", size_vocab); 182 | fprintf(file, "number of classes: %d\n", num_classes); 183 | 184 | fprintf(file, "betas: \n"); // in log space 185 | for (int k = 0; k < num_topics; k++) 186 | { 187 | for (int j = 0; j < size_vocab; j ++) 188 | { 189 | fprintf(file, "%lf ", log_prob_w[k][j]); 190 | } 191 | fprintf(file, "\n"); 192 | } 193 | if (num_classes > 1) 194 | { 195 | fprintf(file, "etas: \n"); 196 | for (int i = 0; i < num_classes-1; i ++) 197 | { 198 | for (int j = 0; j < num_topics; j ++) 199 | { 200 | fprintf(file, "%lf ", eta[i][j]); 201 | } 202 | fprintf(file, "\n"); 203 | } 204 | } 205 | 206 | fflush(file); 207 | fclose(file); 208 | } 209 | 210 | /* 211 | * create the data structure for sufficient statistic 212 | */ 213 | 214 | suffstats * slda::new_suffstats(int num_docs) 215 | { 216 | suffstats * ss = new suffstats; 217 | ss->num_docs = num_docs; 218 | ss->word_total_ss = new double [num_topics]; 219 | memset(ss->word_total_ss, 0, sizeof(double)*num_topics); 220 | ss->word_ss = new double * [num_topics]; 221 | for (int k = 0; k < num_topics; k ++) 222 | { 223 | ss->word_ss[k] = new double [size_vocab]; 224 | memset(ss->word_ss[k], 0, sizeof(double)*size_vocab); 225 | } 226 | 227 | int num_var_entries = num_topics*(num_topics+1)/2; 228 | ss->z_bar = new z_stat [num_docs]; 229 | for (int d = 0; d < num_docs; d ++) 230 | { 231 | ss->z_bar[d].z_bar_m = new double [num_topics]; 232 | ss->z_bar[d].z_bar_var = new double [num_var_entries]; 233 | memset(ss->z_bar[d].z_bar_m, 0, sizeof(double)*num_topics); 234 | memset(ss->z_bar[d].z_bar_var, 0, sizeof(double)*num_var_entries); 235 | } 236 | ss->labels = new int [num_docs]; 237 | memset(ss->labels, 0, sizeof(int)*(num_docs)); 238 | ss->tot_labels = new int [num_classes]; 239 | memset(ss->tot_labels, 0, sizeof(int)*(num_classes)); 240 | 241 | return(ss); 242 | } 243 | 244 | 245 | /* 246 | * initialize the sufficient statistics with zeros 247 | */ 248 | 249 | void slda::zero_initialize_ss(suffstats * ss) 250 | { 251 | memset(ss->word_total_ss, 0, sizeof(double)*num_topics); 252 | for (int k = 0; k < num_topics; k ++) 253 | { 254 | memset(ss->word_ss[k], 0, sizeof(double)*size_vocab); 255 | } 256 | 257 | int num_var_entries = num_topics*(num_topics+1)/2; 258 | for (int d = 0; d < ss->num_docs; d ++) 259 | { 260 | memset(ss->z_bar[d].z_bar_m, 0, sizeof(double)*num_topics); 261 | memset(ss->z_bar[d].z_bar_var, 0, sizeof(double)*num_var_entries); 262 | } 263 | ss->num_docs = 0; 264 | } 265 | 266 | 267 | /* 268 | * initialize the sufficient statistics with random numbers 269 | */ 270 | 271 | void slda::random_initialize_ss(suffstats * ss, corpus* c) 272 | { 273 | int num_docs = ss->num_docs; 274 | gsl_rng * rng = gsl_rng_alloc(gsl_rng_taus); 275 | time_t seed; 276 | time(&seed); 277 | gsl_rng_set(rng, (long) seed); 278 | int k, w, d, j, idx; 279 | for (k = 0; k < num_topics; k++) 280 | { 281 | for (w = 0; w < size_vocab; w++) 282 | { 283 | ss->word_ss[k][w] = 1.0/size_vocab + 0.1*gsl_rng_uniform(rng); 284 | ss->word_total_ss[k] += ss->word_ss[k][w]; 285 | } 286 | } 287 | 288 | for (d = 0; d < num_docs; d ++) 289 | { 290 | document * doc = c->docs[d]; 291 | ss->labels[d] = doc->label; 292 | ss->tot_labels[doc->label] ++; 293 | 294 | double total = 0.0; 295 | for (k = 0; k < num_topics; k ++) 296 | { 297 | ss->z_bar[d].z_bar_m[k] = gsl_rng_uniform(rng); 298 | total += ss->z_bar[d].z_bar_m[k]; 299 | } 300 | for (k = 0; k < num_topics; k ++) 301 | { 302 | ss->z_bar[d].z_bar_m[k] /= total; 303 | } 304 | for (k = 0; k < num_topics; k ++) 305 | { 306 | for (j = k; j < num_topics; j ++) 307 | { 308 | idx = map_idx(k, j, num_topics); 309 | if (j == k) 310 | ss->z_bar[d].z_bar_var[idx] = ss->z_bar[d].z_bar_m[k] / (double)(doc->total); 311 | else 312 | ss->z_bar[d].z_bar_var[idx] = 0.0; 313 | 314 | ss->z_bar[d].z_bar_var[idx] -= 315 | ss->z_bar[d].z_bar_m[k] * ss->z_bar[d].z_bar_m[j] / (double)(doc->total); 316 | } 317 | } 318 | } 319 | 320 | gsl_rng_free(rng); 321 | } 322 | 323 | void slda::corpus_initialize_ss(suffstats* ss, corpus* c) 324 | { 325 | int num_docs = ss->num_docs; 326 | gsl_rng * rng = gsl_rng_alloc(gsl_rng_taus); 327 | time_t seed; 328 | time(&seed); 329 | gsl_rng_set(rng, (long) seed); 330 | int k, n, d, j, idx, i, w; 331 | 332 | for (k = 0; k < num_topics; k++) 333 | { 334 | for (i = 0; i < NUM_INIT; i++) 335 | { 336 | d = (int)(floor(gsl_rng_uniform(rng) * num_docs)); 337 | printf("initialized with document %d\n", d); 338 | document * doc = c->docs[d]; 339 | for (n = 0; n < doc->length; n++) 340 | { 341 | ss->word_ss[k][doc->words[n]] += doc->counts[n]; 342 | } 343 | } 344 | for (w = 0; w < size_vocab; w++) 345 | { 346 | ss->word_ss[k][w] = 2*ss->word_ss[k][w] + 5 + gsl_rng_uniform(rng); 347 | ss->word_total_ss[k] = ss->word_total_ss[k] + ss->word_ss[k][w]; 348 | } 349 | } 350 | 351 | for (d = 0; d < num_docs; d ++) 352 | { 353 | document * doc = c->docs[d]; 354 | ss->labels[d] = doc->label; 355 | ss->tot_labels[doc->label] ++; 356 | 357 | double total = 0.0; 358 | for (k = 0; k < num_topics; k ++) 359 | { 360 | ss->z_bar[d].z_bar_m[k] = gsl_rng_uniform(rng); 361 | total += ss->z_bar[d].z_bar_m[k]; 362 | } 363 | for (k = 0; k < num_topics; k ++) 364 | { 365 | ss->z_bar[d].z_bar_m[k] /= total; 366 | } 367 | for (k = 0; k < num_topics; k ++) 368 | { 369 | for (j = k; j < num_topics; j ++) 370 | { 371 | idx = map_idx(k, j, num_topics); 372 | if (j == k) 373 | ss->z_bar[d].z_bar_var[idx] = ss->z_bar[d].z_bar_m[k] / (double)(doc->total); 374 | else 375 | ss->z_bar[d].z_bar_var[idx] = 0.0; 376 | 377 | ss->z_bar[d].z_bar_var[idx] -= 378 | ss->z_bar[d].z_bar_m[k] * ss->z_bar[d].z_bar_m[j] / (double)(doc->total); 379 | } 380 | } 381 | } 382 | gsl_rng_free(rng); 383 | } 384 | 385 | void slda::load_model_initialize_ss(suffstats* ss, corpus * c) 386 | { 387 | int num_docs = ss->num_docs; 388 | for (int d = 0; d < num_docs; d ++) 389 | { 390 | document * doc = c->docs[d]; 391 | ss->labels[d] = doc->label; 392 | ss->tot_labels[doc->label] ++; 393 | } 394 | } 395 | 396 | void slda::free_suffstats(suffstats * ss) 397 | { 398 | delete [] ss->word_total_ss; 399 | 400 | for (int k = 0; k < num_topics; k ++) 401 | { 402 | delete [] ss->word_ss[k]; 403 | } 404 | delete [] ss->word_ss; 405 | 406 | for (int d = 0; d < ss->num_docs; d ++) 407 | { 408 | delete [] ss->z_bar[d].z_bar_m; 409 | delete [] ss->z_bar[d].z_bar_var; 410 | } 411 | delete [] ss->z_bar; 412 | delete [] ss->labels; 413 | delete [] ss->tot_labels; 414 | 415 | delete ss; 416 | } 417 | 418 | void slda::v_em(corpus * c, const settings * setting, 419 | const char * start, const char * directory) 420 | { 421 | char filename[100]; 422 | int max_length = c->max_corpus_length(); 423 | double **var_gamma, **phi, **lambda; 424 | double likelihood, likelihood_old = 0, converged = 1; 425 | int d, n, i; 426 | double L2penalty = setting->PENALTY; 427 | // allocate variational parameters 428 | var_gamma = new double * [c->num_docs]; 429 | for (d = 0; d < c->num_docs; d++) 430 | var_gamma[d] = new double [num_topics]; 431 | 432 | phi = new double * [max_length]; 433 | for (n = 0; n < max_length; n++) 434 | phi[n] = new double [num_topics]; 435 | 436 | printf("initializing ...\n"); 437 | suffstats * ss = new_suffstats(c->num_docs); 438 | if (strcmp(start, "seeded") == 0) 439 | { 440 | corpus_initialize_ss(ss, c); 441 | mle(ss, 0, setting); 442 | } 443 | else if (strcmp(start, "random") == 0) 444 | { 445 | random_initialize_ss(ss, c); 446 | mle(ss, 0, setting); 447 | } 448 | else 449 | { 450 | load_model(start); 451 | load_model_initialize_ss(ss, c); 452 | } 453 | 454 | FILE * likelihood_file = NULL; 455 | sprintf(filename, "%s/likelihood.dat", directory); 456 | likelihood_file = fopen(filename, "w"); 457 | 458 | int ETA_UPDATE = 0; 459 | 460 | i = 0; 461 | while (((converged < 0) || (converged > setting->EM_CONVERGED) || (i <= LDA_INIT_MAX+2)) && (i <= setting->EM_MAX_ITER)) 462 | { 463 | printf("**** em iteration %d ****\n", ++i); 464 | likelihood = 0; 465 | zero_initialize_ss(ss); 466 | if (i > LDA_INIT_MAX) ETA_UPDATE = 1; 467 | // e-step 468 | printf("**** e-step ****\n"); 469 | for (d = 0; d < c->num_docs; d++) 470 | { 471 | if ((d % 100) == 0) printf("document %d\n", d); 472 | likelihood += doc_e_step(c->docs[d], var_gamma[d], phi, ss, ETA_UPDATE, setting); 473 | } 474 | 475 | printf("likelihood: %10.10f\n", likelihood); 476 | // m-step 477 | printf("**** m-step ****\n"); 478 | mle(ss, ETA_UPDATE, setting); 479 | 480 | // check for convergence 481 | converged = fabs((likelihood_old - likelihood) / (likelihood_old)); 482 | //if (converged < 0) VAR_MAX_ITER = VAR_MAX_ITER * 2; 483 | likelihood_old = likelihood; 484 | 485 | // output model and likelihood 486 | fprintf(likelihood_file, "%10.10f\t%5.5e\n", likelihood, converged); 487 | fflush(likelihood_file); 488 | if ((i % LAG) == 0) 489 | { 490 | sprintf(filename, "%s/%03d.model", directory, i); 491 | save_model(filename); 492 | sprintf(filename, "%s/%03d.model.text", directory, i); 493 | save_model_text(filename); 494 | sprintf(filename, "%s/%03d.gamma", directory, i); 495 | save_gamma(filename, var_gamma, c->num_docs); 496 | } 497 | } 498 | 499 | // output the final model 500 | sprintf(filename, "%s/final.model", directory); 501 | save_model(filename); 502 | sprintf(filename, "%s/final.model.text", directory); 503 | save_model_text(filename); 504 | sprintf(filename, "%s/final.gamma", directory); 505 | save_gamma(filename, var_gamma, c->num_docs); 506 | 507 | 508 | fclose(likelihood_file); 509 | FILE * w_asgn_file = NULL; 510 | sprintf(filename, "%s/word-assignments.dat", directory); 511 | w_asgn_file = fopen(filename, "w"); 512 | for (d = 0; d < c->num_docs; d ++) 513 | { 514 | //final inference 515 | if ((d % 100) == 0) printf("final e step document %d\n", d); 516 | likelihood += slda_inference(c->docs[d], var_gamma[d], phi, setting); 517 | write_word_assignment(w_asgn_file, c->docs[d], phi); 518 | 519 | } 520 | fclose(w_asgn_file); 521 | 522 | free_suffstats(ss); 523 | for (d = 0; d < c->num_docs; d++) 524 | delete [] var_gamma[d]; 525 | delete [] var_gamma; 526 | 527 | for (n = 0; n < max_length; n++) 528 | delete [] phi[n]; 529 | delete [] phi; 530 | } 531 | void slda::mle(suffstats * ss, int eta_update, const settings * setting) 532 | { 533 | int k, w; 534 | 535 | for (k = 0; k < num_topics; k++) 536 | { 537 | for (w = 0; w < size_vocab; w++) 538 | { 539 | if (ss->word_ss[k][w] > 0) 540 | log_prob_w[k][w] = log(ss->word_ss[k][w]) - log(ss->word_total_ss[k]); 541 | else 542 | log_prob_w[k][w] = -100.0; 543 | } 544 | } 545 | if (eta_update == 0) return; 546 | 547 | //the label part goes here 548 | printf("maximizing ...\n"); 549 | double f = 0.0; 550 | int status; 551 | int opt_iter; 552 | int opt_size = (num_classes-1) * num_topics; 553 | int l; 554 | 555 | opt_parameter param; 556 | param.ss = ss; 557 | param.model = this; 558 | param.PENALTY = setting->PENALTY; 559 | 560 | const gsl_multimin_fdfminimizer_type * T; 561 | gsl_multimin_fdfminimizer * s; 562 | gsl_vector * x; 563 | gsl_multimin_function_fdf opt_fun; 564 | opt_fun.f = &softmax_f; 565 | opt_fun.df = &softmax_df; 566 | opt_fun.fdf = &softmax_fdf; 567 | opt_fun.n = opt_size; 568 | opt_fun.params = (void*)(¶m); 569 | x = gsl_vector_alloc(opt_size); 570 | 571 | 572 | for (l = 0; l < num_classes-1; l ++) 573 | { 574 | for (k = 0; k < num_topics; k ++) 575 | { 576 | gsl_vector_set(x, l*num_topics + k, eta[l][k]); 577 | } 578 | } 579 | 580 | T = gsl_multimin_fdfminimizer_vector_bfgs; 581 | s = gsl_multimin_fdfminimizer_alloc(T, opt_size); 582 | gsl_multimin_fdfminimizer_set(s, &opt_fun, x, 0.02, 1e-4); 583 | 584 | opt_iter = 0; 585 | do 586 | { 587 | opt_iter ++; 588 | status = gsl_multimin_fdfminimizer_iterate(s); 589 | if (status) 590 | break; 591 | status = gsl_multimin_test_gradient(s->gradient, 1e-3); 592 | if (status == GSL_SUCCESS) 593 | break; 594 | f = -s->f; 595 | if ((opt_iter-1) % 10 == 0) 596 | printf("step: %02d -> f: %f\n", opt_iter-1, f); 597 | } while (status == GSL_CONTINUE && opt_iter < MSTEP_MAX_ITER); 598 | 599 | for (l = 0; l < num_classes-1; l ++) 600 | { 601 | for (k = 0; k < num_topics; k ++) 602 | { 603 | eta[l][k] = gsl_vector_get(s->x, l*num_topics + k); 604 | } 605 | } 606 | 607 | gsl_multimin_fdfminimizer_free (s); 608 | gsl_vector_free (x); 609 | 610 | printf("final f: %f\n", f); 611 | } 612 | 613 | double slda::doc_e_step(document* doc, double* gamma, double** phi, 614 | suffstats * ss, int eta_update, const settings * setting) 615 | { 616 | double likelihood = 0.0; 617 | if (eta_update == 1) 618 | likelihood = slda_inference(doc, gamma, phi, setting); 619 | else 620 | likelihood = lda_inference(doc, gamma, phi, setting); 621 | 622 | int d = ss->num_docs; 623 | 624 | int n, k, i, idx, m; 625 | 626 | // update sufficient statistics 627 | 628 | for (n = 0; n < doc->length; n++) 629 | { 630 | for (k = 0; k < num_topics; k++) 631 | { 632 | ss->word_ss[k][doc->words[n]] += doc->counts[n]*phi[n][k]; 633 | ss->word_total_ss[k] += doc->counts[n]*phi[n][k]; 634 | 635 | //statistics for each document of the supervised part 636 | ss->z_bar[d].z_bar_m[k] += doc->counts[n] * phi[n][k]; //mean 637 | for (i = k; i < num_topics; i ++) //variance 638 | { 639 | idx = map_idx(k, i, num_topics); 640 | if (i == k) 641 | ss->z_bar[d].z_bar_var[idx] += 642 | doc->counts[n] * doc->counts[n] * phi[n][k]; 643 | 644 | ss->z_bar[d].z_bar_var[idx] -= 645 | doc->counts[n] * doc->counts[n] * phi[n][k] * phi[n][i]; 646 | } 647 | } 648 | } 649 | for (k = 0; k < num_topics; k++) 650 | { 651 | ss->z_bar[d].z_bar_m[k] /= (double)(doc->total); 652 | } 653 | for (i = 0; i < num_topics*(num_topics+1)/2; i ++) 654 | { 655 | ss->z_bar[d].z_bar_var[i] /= (double)(doc->total * doc->total); 656 | } 657 | 658 | ss->num_docs = ss->num_docs + 1; //because we need it for store statistics for each docs 659 | 660 | return (likelihood); 661 | } 662 | 663 | double slda::lda_inference(document* doc, double* var_gamma, double** phi, const settings * setting) 664 | { 665 | int k, n, var_iter; 666 | double converged = 1, phisum = 0, likelihood = 0, likelihood_old = 0; 667 | 668 | double *oldphi = new double [num_topics]; 669 | double *digamma_gam = new double [num_topics]; 670 | 671 | // compute posterior dirichlet 672 | for (k = 0; k < num_topics; k++) 673 | { 674 | var_gamma[k] = alpha + (doc->total/((double) num_topics)); 675 | digamma_gam[k] = digamma(var_gamma[k]); 676 | for (n = 0; n < doc->length; n++) 677 | phi[n][k] = 1.0/num_topics; 678 | } 679 | var_iter = 0; 680 | 681 | while (converged > setting->VAR_CONVERGED && (var_iter < setting->VAR_MAX_ITER || setting->VAR_MAX_ITER == -1)) 682 | { 683 | var_iter++; 684 | for (n = 0; n < doc->length; n++) 685 | { 686 | phisum = 0; 687 | for (k = 0; k < num_topics; k++) 688 | { 689 | oldphi[k] = phi[n][k]; 690 | phi[n][k] = digamma_gam[k] + log_prob_w[k][doc->words[n]]; 691 | 692 | if (k > 0) 693 | phisum = log_sum(phisum, phi[n][k]); 694 | else 695 | phisum = phi[n][k]; // note, phi is in log space 696 | } 697 | 698 | for (k = 0; k < num_topics; k++) 699 | { 700 | phi[n][k] = exp(phi[n][k] - phisum); 701 | var_gamma[k] = var_gamma[k] + doc->counts[n]*(phi[n][k] - oldphi[k]); 702 | digamma_gam[k] = digamma(var_gamma[k]); 703 | } 704 | } 705 | 706 | likelihood = lda_compute_likelihood(doc, phi, var_gamma); 707 | assert(!isnan(likelihood)); 708 | converged = (likelihood_old - likelihood) / likelihood_old; 709 | likelihood_old = likelihood; 710 | } 711 | 712 | delete [] oldphi; 713 | delete [] digamma_gam; 714 | 715 | return likelihood; 716 | } 717 | 718 | double slda::lda_compute_likelihood(document* doc, double** phi, double* var_gamma) 719 | { 720 | double likelihood = 0, digsum = 0, var_gamma_sum = 0; 721 | double *dig = new double [num_topics]; 722 | int k, n; 723 | double alpha_sum = num_topics * alpha; 724 | for (k = 0; k < num_topics; k++) 725 | { 726 | dig[k] = digamma(var_gamma[k]); 727 | var_gamma_sum += var_gamma[k]; 728 | } 729 | digsum = digamma(var_gamma_sum); 730 | 731 | likelihood = lgamma(alpha_sum) - lgamma(var_gamma_sum); 732 | 733 | for (k = 0; k < num_topics; k++) 734 | { 735 | likelihood += - lgamma(alpha) + (alpha - 1)*(dig[k] - digsum) + 736 | lgamma(var_gamma[k]) - (var_gamma[k] - 1)*(dig[k] - digsum); 737 | 738 | for (n = 0; n < doc->length; n++) 739 | { 740 | if (phi[n][k] > 0) 741 | { 742 | likelihood += doc->counts[n]*(phi[n][k]*((dig[k] - digsum) - 743 | log(phi[n][k]) + log_prob_w[k][doc->words[n]])); 744 | } 745 | } 746 | } 747 | 748 | delete [] dig; 749 | return likelihood; 750 | } 751 | 752 | double slda::slda_compute_likelihood(document* doc, double** phi, double* var_gamma) 753 | { 754 | double likelihood = 0, digsum = 0, var_gamma_sum = 0, t = 0.0, t1 = 0.0, t2 = 0.0; 755 | double * dig = new double [num_topics]; 756 | int k, n, l; 757 | int flag; 758 | double alpha_sum = num_topics * alpha; 759 | for (k = 0; k < num_topics; k++) 760 | { 761 | dig[k] = digamma(var_gamma[k]); 762 | var_gamma_sum += var_gamma[k]; 763 | } 764 | digsum = digamma(var_gamma_sum); 765 | 766 | likelihood = lgamma(alpha_sum) - lgamma(var_gamma_sum); 767 | t = 0.0; 768 | for (k = 0; k < num_topics; k++) 769 | { 770 | likelihood += -lgamma(alpha) + (alpha - 1)*(dig[k] - digsum) + lgamma(var_gamma[k]) - (var_gamma[k] - 1)*(dig[k] - digsum); 771 | 772 | for (n = 0; n < doc->length; n++) 773 | { 774 | if (phi[n][k] > 0) 775 | { 776 | likelihood += doc->counts[n]*(phi[n][k]*((dig[k] - digsum) - log(phi[n][k]) + log_prob_w[k][doc->words[n]])); 777 | if (doc->label < num_classes-1) 778 | t += eta[doc->label][k] * doc->counts[n] * phi[n][k]; 779 | } 780 | } 781 | } 782 | likelihood += t / (double)(doc->total); //eta_k*\bar{\phi} 783 | 784 | t = 1.0; //the class model->num_classes-1 785 | for (l = 0; l < num_classes-1; l ++) 786 | { 787 | t1 = 1.0; 788 | for (n = 0; n < doc->length; n ++) 789 | { 790 | t2 = 0.0; 791 | for (k = 0; k < num_topics; k ++) 792 | { 793 | t2 += phi[n][k] * exp(eta[l][k] * doc->counts[n]/(double)(doc->total)); 794 | } 795 | t1 *= t2; 796 | } 797 | t += t1; 798 | } 799 | likelihood -= log(t); 800 | delete [] dig; 801 | //printf("%lf\n", likelihood); 802 | return likelihood; 803 | } 804 | 805 | double slda::slda_inference(document* doc, double* var_gamma, double** phi, const settings * setting) 806 | { 807 | int k, n, var_iter, l; 808 | int FP_MAX_ITER = 10; 809 | int fp_iter = 0; 810 | double converged = 1, phisum = 0, likelihood = 0, likelihood_old = 0; 811 | double * oldphi = new double [num_topics]; 812 | double * digamma_gam = new double [num_topics]; 813 | double * sf_params = new double [num_topics]; 814 | double * sf_aux = new double [num_classes-1]; 815 | double sf_val = 0.0; 816 | 817 | // compute posterior dirichlet 818 | for (k = 0; k < num_topics; k++) 819 | { 820 | var_gamma[k] = alpha + (doc->total/((double) num_topics)); 821 | digamma_gam[k] = digamma(var_gamma[k]); 822 | for (n = 0; n < doc->length; n++) 823 | phi[n][k] = 1.0/(double)(num_topics); 824 | } 825 | 826 | double t = 0.0; 827 | for (l = 0; l < num_classes-1; l ++) 828 | { 829 | sf_aux[l] = 1.0; // the quantity for equation 6 of each class 830 | for (n = 0; n < doc->length; n ++) 831 | { 832 | t = 0.0; 833 | for (k = 0; k < num_topics; k ++) 834 | { 835 | t += phi[n][k] * exp(eta[l][k] * doc->counts[n]/(double)(doc->total)); 836 | } 837 | sf_aux[l] *= t; 838 | } 839 | } 840 | 841 | var_iter = 0; 842 | 843 | while ((converged > setting->VAR_CONVERGED) && ((var_iter < setting->VAR_MAX_ITER) || (setting->VAR_MAX_ITER == -1))) 844 | { 845 | var_iter++; 846 | for (n = 0; n < doc->length; n++) 847 | { 848 | //compute sf_params 849 | memset(sf_params, 0, sizeof(double)*num_topics); //in log space 850 | for (l = 0; l < num_classes-1; l ++) 851 | { 852 | t = 0.0; 853 | for (k = 0; k < num_topics; k ++) 854 | { 855 | t += phi[n][k] * exp(eta[l][k] * doc->counts[n]/(double)(doc->total)); 856 | } 857 | sf_aux[l] /= t; //take out word n 858 | 859 | for (k = 0; k < num_topics; k ++) 860 | { 861 | //h in the paper 862 | sf_params[k] += sf_aux[l]*exp(eta[l][k] * doc->counts[n]/(double)(doc->total)); 863 | } 864 | } 865 | // 866 | for (k = 0; k < num_topics; k++) 867 | { 868 | oldphi[k] = phi[n][k]; 869 | } 870 | for (fp_iter = 0; fp_iter < FP_MAX_ITER; fp_iter ++) //fixed point update 871 | { 872 | sf_val = 1.0; // the base class, in log space 873 | for (k = 0; k < num_topics; k++) 874 | { 875 | sf_val += sf_params[k]*phi[n][k]; 876 | } 877 | 878 | phisum = 0; 879 | for (k = 0; k < num_topics; k++) 880 | { 881 | phi[n][k] = digamma_gam[k] + log_prob_w[k][doc->words[n]]; 882 | 883 | //added softmax parts 884 | if (doc->label < num_classes-1) 885 | phi[n][k] += eta[doc->label][k]/(double)(doc->total); 886 | phi[n][k] -= sf_params[k]/(sf_val*(double)(doc->counts[n])); 887 | 888 | if (k > 0) 889 | phisum = log_sum(phisum, phi[n][k]); 890 | else 891 | phisum = phi[n][k]; // note, phi is in log space 892 | } 893 | for (k = 0; k < num_topics; k++) 894 | { 895 | phi[n][k] = exp(phi[n][k] - phisum); //normalize 896 | } 897 | } 898 | //back to sf_aux value 899 | for (l = 0; l < num_classes-1; l ++) 900 | { 901 | t = 0.0; 902 | for (k = 0; k < num_topics; k ++) 903 | { 904 | t += phi[n][k] * exp(eta[l][k] * doc->counts[n]/(double)(doc->total)); 905 | } 906 | sf_aux[l] *= t; 907 | } 908 | for (k = 0; k < num_topics; k++) 909 | { 910 | var_gamma[k] = var_gamma[k] + doc->counts[n]*(phi[n][k] - oldphi[k]); 911 | digamma_gam[k] = digamma(var_gamma[k]); 912 | } 913 | } 914 | 915 | likelihood = slda_compute_likelihood(doc, phi, var_gamma); 916 | assert(!isnan(likelihood)); 917 | converged = fabs((likelihood_old - likelihood) / likelihood_old); 918 | likelihood_old = likelihood; 919 | } 920 | 921 | delete [] oldphi; 922 | delete [] digamma_gam; 923 | delete [] sf_params; 924 | delete [] sf_aux; 925 | return likelihood; 926 | } 927 | 928 | void slda::infer_only(corpus * c, const settings * setting, const char * directory) 929 | { 930 | int i, k, d, n; 931 | double **var_gamma, likelihood, **phi; 932 | double* phi_m; 933 | char filename[100]; 934 | double base_score, score; 935 | int label; 936 | int num_correct = 0; 937 | int max_length = c->max_corpus_length(); 938 | 939 | 940 | var_gamma = new double * [c->num_docs]; 941 | for (i = 0; i < c->num_docs; i++) 942 | var_gamma[i] = new double [num_topics]; 943 | 944 | 945 | phi = new double * [max_length]; 946 | for (n = 0; n < max_length; n++) 947 | phi[n] = new double [num_topics]; 948 | 949 | phi_m = new double [num_topics]; 950 | 951 | FILE * likelihood_file = NULL; 952 | sprintf(filename, "%s/inf-likelihood.dat", directory); 953 | likelihood_file = fopen(filename, "w"); 954 | FILE * inf_label_file = NULL; 955 | sprintf(filename, "%s/inf-labels.dat", directory); 956 | inf_label_file = fopen(filename, "w"); 957 | 958 | for (d = 0; d < c->num_docs; d++) 959 | { 960 | if ((d % 100) == 0) 961 | printf("document %d\n", d); 962 | 963 | document * doc = c->docs[d]; 964 | likelihood = lda_inference(doc, var_gamma[d], phi, setting); 965 | 966 | memset(phi_m, 0, sizeof(double)*num_topics); //zero_initialize 967 | for (n = 0; n < doc->length; n++) 968 | { 969 | for (k = 0; k < num_topics; k ++) 970 | { 971 | phi_m[k] += doc->counts[n] * phi[n][k]; 972 | } 973 | } 974 | for (k = 0; k < num_topics; k ++) 975 | { 976 | phi_m[k] /= (double)(doc->total); 977 | } 978 | 979 | //do classification 980 | label = num_classes-1; 981 | base_score = 0.0; 982 | for (i = 0; i < num_classes-1; i ++) 983 | { 984 | score = 0.0; 985 | for (k = 0; k < num_topics; k ++) 986 | { 987 | score += eta[i][k] * phi_m[k]; 988 | } 989 | if (score > base_score) 990 | { 991 | base_score = score; 992 | label = i; 993 | } 994 | } 995 | if (label == doc->label) 996 | num_correct ++; 997 | 998 | fprintf(likelihood_file, "%5.5f\n", likelihood); 999 | fprintf(inf_label_file, "%d\n", label); 1000 | } 1001 | 1002 | printf("average accuracy: %.3f\n", (double)num_correct / (double) c->num_docs); 1003 | 1004 | sprintf(filename, "%s/inf-gamma.dat", directory); 1005 | save_gamma(filename, var_gamma, c->num_docs); 1006 | 1007 | for (d = 0; d < c->num_docs; d++) 1008 | delete [] var_gamma[d]; 1009 | delete [] var_gamma; 1010 | 1011 | for (n = 0; n < max_length; n++) 1012 | delete [] phi[n]; 1013 | delete [] phi; 1014 | 1015 | delete [] phi_m; 1016 | } 1017 | 1018 | void slda::save_gamma(char* filename, double** gamma, int num_docs) 1019 | { 1020 | int d, k; 1021 | 1022 | FILE* fileptr = fopen(filename, "w"); 1023 | for (d = 0; d < num_docs; d++) 1024 | { 1025 | fprintf(fileptr, "%5.10f", gamma[d][0]); 1026 | for (k = 1; k < num_topics; k++) 1027 | fprintf(fileptr, " %5.10f", gamma[d][k]); 1028 | fprintf(fileptr, "\n"); 1029 | } 1030 | fclose(fileptr); 1031 | } 1032 | 1033 | void slda::write_word_assignment(FILE* f, document* doc, double** phi) 1034 | { 1035 | int n; 1036 | 1037 | fprintf(f, "%03d", doc->length); 1038 | for (n = 0; n < doc->length; n++) 1039 | { 1040 | fprintf(f, " %04d:%02d", doc->words[n], argmax(phi[n], num_topics)); 1041 | } 1042 | fprintf(f, "\n"); 1043 | fflush(f); 1044 | } 1045 | -------------------------------------------------------------------------------- /slda.h: -------------------------------------------------------------------------------- 1 | // (C) Copyright 2009, Chong Wang, David Blei and Li Fei-Fei 2 | 3 | // written by Chong Wang, chongw@cs.princeton.edu 4 | 5 | // This file is part of slda. 6 | 7 | // slda is free software; you can redistribute it and/or modify it under 8 | // the terms of the GNU General Public License as published by the Free 9 | // Software Foundation; either version 2 of the License, or (at your 10 | // option) any later version. 11 | 12 | // slda is distributed in the hope that it will be useful, but WITHOUT 13 | // ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 14 | // FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License 15 | // for more details. 16 | 17 | // You should have received a copy of the GNU General Public License 18 | // along with this program; if not, write to the Free Software 19 | // Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 20 | // USA 21 | 22 | #ifndef SLDA_H 23 | #define SLDA_H 24 | #include "settings.h" 25 | #include "corpus.h" 26 | 27 | typedef struct { 28 | double * z_bar_m; 29 | double * z_bar_var; 30 | } z_stat; 31 | 32 | typedef struct { 33 | double ** word_ss; 34 | double * word_total_ss; 35 | int num_docs; 36 | z_stat * z_bar; 37 | int * labels; 38 | int * tot_labels; 39 | } suffstats; 40 | 41 | class slda 42 | { 43 | public: 44 | slda(); 45 | ~slda(); 46 | void free_model(); 47 | void init(double alpha_, int num_topics_, const corpus * c); 48 | void v_em(corpus * c, const settings * setting, 49 | const char * start, const char * directory); 50 | 51 | void save_model(const char * filename); 52 | void save_model_text(const char * filename); 53 | void load_model(const char * model_filename); 54 | void infer_only(corpus * c, const settings * setting, 55 | const char * directory); 56 | 57 | suffstats * new_suffstats(int num_docs); 58 | void free_suffstats(suffstats * ss); 59 | void zero_initialize_ss(suffstats * ss); 60 | void random_initialize_ss(suffstats * ss, corpus * c); 61 | void corpus_initialize_ss(suffstats* ss, corpus * c); 62 | void load_model_initialize_ss(suffstats* ss, corpus * c); 63 | void mle(suffstats * ss, int eta_update, const settings * setting); 64 | 65 | double doc_e_step(document* doc, double* gamma, double** phi, suffstats * ss, int eta_update, const settings * setting); 66 | 67 | double lda_inference(document* doc, double* var_gamma, double** phi, const settings * setting); 68 | double lda_compute_likelihood(document* doc, double** phi, double* var_gamma); 69 | double slda_inference(document* doc, double* var_gamma, double** phi, const settings * setting); 70 | double slda_compute_likelihood(document* doc, double** phi, double* var_gamma); 71 | 72 | void save_gamma(char* filename, double** gamma, int num_docs); 73 | void write_word_assignment(FILE* f, document* doc, double** phi); 74 | 75 | 76 | public: 77 | double alpha; // the parameter for the dirichlet 78 | int num_topics; 79 | int num_classes; 80 | int size_vocab; 81 | 82 | double ** log_prob_w; //the log of the topic distribution 83 | double ** eta; //softmax regression, in general, there are num_classes-1 etas, we don't need a intercept here, since \sum_i \bar{z_i} = 1 84 | }; 85 | 86 | #endif // SLDA_H 87 | 88 | -------------------------------------------------------------------------------- /utils.cpp: -------------------------------------------------------------------------------- 1 | #include "utils.h" 2 | 3 | /* 4 | * given log(a) and log(b), return log(a + b) 5 | * 6 | */ 7 | 8 | double log_sum(double log_a, double log_b) 9 | { 10 | double v; 11 | 12 | if (log_a < log_b) 13 | v = log_b+log(1 + exp(log_a-log_b)); 14 | else 15 | v = log_a+log(1 + exp(log_b-log_a)); 16 | 17 | return v; 18 | } 19 | 20 | /** 21 | * Proc to calculate the value of the trigamma, the second 22 | * derivative of the loggamma function. Accepts positive matrices. 23 | * From Abromowitz and Stegun. Uses formulas 6.4.11 and 6.4.12 with 24 | * recurrence formula 6.4.6. Each requires workspace at least 5 25 | * times the size of X. 26 | * 27 | **/ 28 | 29 | double trigamma(double x) 30 | { 31 | double p; 32 | int i; 33 | 34 | x = x+6; 35 | p = 1/(x*x); 36 | p = (((((0.075757575757576*p-0.033333333333333)*p+0.0238095238095238)*p-0.033333333333333)*p+0.166666666666667)*p+1)/x+0.5*p; 37 | for (i=0; i<6 ;i++) 38 | { 39 | x = x-1; 40 | p = 1/(x*x)+p; 41 | } 42 | return p; 43 | } 44 | 45 | 46 | /* 47 | * taylor approximation of first derivative of the log gamma function 48 | * 49 | */ 50 | 51 | double digamma(double x) 52 | { 53 | double p; 54 | x = x+6; 55 | p = 1/(x*x); 56 | p = (((0.004166666666667*p-0.003968253986254)*p+0.008333333333333)*p-0.083333333333333)*p; 57 | p = p+log(x)-0.5/x-1/(x-1)-1/(x-2)-1/(x-3)-1/(x-4)-1/(x-5)-1/(x-6); 58 | return p; 59 | } 60 | 61 | /* 62 | * this log gamma function has the implementation of this function 63 | * 64 | */ 65 | 66 | /* double lgamma(double x) 67 | * { 68 | * double x0,x2,xp,gl,gl0; 69 | * int n,k; 70 | * static double a[] = { 71 | * 8.333333333333333e-02, 72 | * -2.777777777777778e-03, 73 | * 7.936507936507937e-04, 74 | * -5.952380952380952e-04, 75 | * 8.417508417508418e-04, 76 | * -1.917526917526918e-03, 77 | * 6.410256410256410e-03, 78 | * -2.955065359477124e-02, 79 | * 1.796443723688307e-01, 80 | * -1.39243221690590 81 | * }; 82 | * 83 | * x0 = x; 84 | * if (x <= 0.0) return 1e308; 85 | * else if ((x == 1.0) || (x == 2.0)) return 0.0; 86 | * else if (x <= 7.0) { 87 | * n = (int)(7-x); 88 | * x0 = x+n; 89 | * } 90 | * x2 = 1.0/(x0*x0); 91 | * xp = 2.0*M_PI; 92 | * gl0 = a[9]; 93 | * for (k=8;k>=0;k--) { 94 | * gl0 = gl0*x2 + a[k]; 95 | * } 96 | * gl = gl0/x0+0.5*log(xp)+(x0-0.5)*log(x0)-x0; 97 | * if (x <= 7.0) { 98 | * for (k=1;k<=n;k++) { 99 | * gl -= log(x0-1.0); 100 | * x0 -= 1.0; 101 | * } 102 | * } 103 | * return gl; 104 | * } 105 | */ 106 | 107 | 108 | 109 | /* 110 | * make directory 111 | * 112 | */ 113 | 114 | void make_directory(char* name) 115 | { 116 | mkdir(name, S_IRUSR|S_IWUSR|S_IXUSR); 117 | } 118 | 119 | 120 | /* 121 | * argmax 122 | * 123 | */ 124 | 125 | int argmax(double* x, int n) 126 | { 127 | int i, argmax = 0; 128 | double max = x[0]; 129 | 130 | for (i = 1; i < n; i++) 131 | { 132 | if (x[i] > max) 133 | { 134 | max = x[i]; 135 | argmax = i; 136 | } 137 | } 138 | return argmax; 139 | } 140 | 141 | /* 142 | * return the correponding index in the n(n+1)/2 given row and col 143 | * this is a upper triangle matrix, we can do this since this is 144 | * a symmetric matrix 145 | * 146 | */ 147 | 148 | int map_idx(int row, int col, int dim) 149 | { 150 | int swap, idx; 151 | if (row > col) 152 | { 153 | swap = row; 154 | row = col; 155 | col = swap; 156 | } 157 | //now row <= col 158 | idx = (2*dim - row + 1)*row/2 + col - row; 159 | return idx; 160 | } 161 | 162 | -------------------------------------------------------------------------------- /utils.h: -------------------------------------------------------------------------------- 1 | #ifndef UTILS_H 2 | #define UTILS_H 3 | 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | #include 10 | 11 | double log_sum(double log_a, double log_b); 12 | double trigamma(double x); 13 | double digamma(double x); 14 | //double lgamma(double x); 15 | void make_directory(char* name); 16 | int argmax(double* x, int n); 17 | int map_idx(int row, int col, int dim); 18 | 19 | #endif 20 | 21 | --------------------------------------------------------------------------------