├── .gitignore
├── CONTRIBUTION_GUIDE.md
├── CONTRIBUTORS.md
├── COPYRIGHT.txt
├── Contents.m
├── INSTALL.md
├── LICENSE.txt
├── README.md
├── cstep.m
├── cvsrch.m
├── docs
├── README.md
├── examples_doc.m
├── gradientcheck_doc.m
├── helptoc.xml
├── html
│ ├── examples_doc.html
│ ├── examples_doc_eq00543772986039519777.png
│ ├── examples_doc_eq02586487823804834451.png
│ ├── examples_doc_eq02619125812656254059.png
│ ├── examples_doc_eq02739270504201626537.png
│ ├── examples_doc_eq02978325375087395648.png
│ ├── examples_doc_eq03421469680577370400.png
│ ├── examples_doc_eq03672095713503266041.png
│ ├── examples_doc_eq05147331747641807187.png
│ ├── examples_doc_eq05495199406804600545.png
│ ├── examples_doc_eq05508344529756732484.png
│ ├── examples_doc_eq06327244962903926757.png
│ ├── examples_doc_eq07741076879175499892.png
│ ├── examples_doc_eq11316155984473799093.png
│ ├── examples_doc_eq11475201221685588019.png
│ ├── examples_doc_eq11638118408029626293.png
│ ├── examples_doc_eq12267211811494638899.png
│ ├── examples_doc_eq12534387414789875675.png
│ ├── examples_doc_eq13029888333516263543.png
│ ├── examples_doc_eq15636846968047188835.png
│ ├── examples_doc_eq17937321231062034804.png
│ ├── gradientcheck_doc.html
│ ├── gradientcheck_doc_eq00827914115064801059.png
│ ├── gradientcheck_doc_eq05671228016298599287.png
│ ├── gradientcheck_doc_eq08062630503172331818.png
│ ├── gradientcheck_doc_eq12428413953531653171.png
│ ├── gradientcheck_doc_eq13924783238156832738.png
│ ├── gradientcheck_doc_eq15869567321110343067.png
│ ├── gradientcheck_doc_eq16537821098686504121.png
│ ├── images
│ │ ├── banner-background.jpg
│ │ └── logo.gif
│ ├── lbfgs_doc.html
│ ├── lbfgs_doc_eq00345914025344442474.png
│ ├── lbfgs_doc_eq02499409918622348759.png
│ ├── lbfgs_doc_eq02757236760158446565.png
│ ├── lbfgs_doc_eq05671228016298599287.png
│ ├── lbfgs_doc_eq07165327077797634723.png
│ ├── lbfgs_doc_eq17452142193152507074.png
│ ├── methods.html
│ ├── ncg_doc.html
│ ├── ncg_doc_eq00202142981986870057.png
│ ├── ncg_doc_eq00232388002207087152.png
│ ├── ncg_doc_eq00345914025344442474.png
│ ├── ncg_doc_eq00906644648645161877.png
│ ├── ncg_doc_eq02535271716144487441.png
│ ├── ncg_doc_eq05857437662927211826.png
│ ├── ncg_doc_eq07165327077797634723.png
│ ├── ncg_doc_eq08047493418710975323.png
│ ├── ncg_doc_eq11960540003268942809.png
│ ├── ncg_doc_eq12955286862837271730.png
│ ├── ncg_doc_eq16009458126034669820.png
│ ├── ncg_doc_eq16519649717996589748.png
│ ├── ncg_doc_eq17452142193152507074.png
│ ├── overview_doc.html
│ ├── overview_doc_eq00884557479967417095.png
│ ├── overview_doc_eq07500982203449410655.png
│ ├── overview_doc_eq08097522164358817053.png
│ ├── overview_doc_eq13721435717236182146.png
│ ├── overview_doc_eq13754963546198947976.png
│ ├── overview_doc_eq16485556778937288589.png
│ ├── parameters.html
│ ├── parameters_input_doc.html
│ ├── parameters_output_doc.html
│ ├── poblano.css
│ ├── tn_doc.html
│ ├── tn_doc_eq00345914025344442474.png
│ ├── tn_doc_eq03158747792916826732.png
│ ├── tn_doc_eq06834187619862737782.png
│ ├── tn_doc_eq07165327077797634723.png
│ ├── tn_doc_eq08610859800392802260.png
│ ├── tn_doc_eq11373214381793991308.png
│ ├── tn_doc_eq13505831761284237672.png
│ ├── tn_doc_eq13549337764080820674.png
│ ├── tn_doc_eq15653174921336884406.png
│ ├── tn_doc_eq17016985893546760461.png
│ └── tn_doc_eq17452142193152507074.png
├── index.html
├── lbfgs_doc.m
├── mxdom2simplehtml_poblano.xsl
├── ncg_doc.m
├── overview_doc.m
├── parameters_input_doc.m
├── parameters_output_doc.m
└── tn_doc.m
├── example1.m
├── example2.m
├── example2_extract.m
├── example2_init.m
├── gradientcheck.m
├── hessvec_fd.m
├── info.xml
├── install_poblano.m
├── lbfgs.m
├── ncg.m
├── poblano_linesearch.m
├── poblano_out.m
├── poblano_params.m
├── tests
├── README.md
├── matlab.mat
├── poblano_coverage.m
├── test_cvsrch.m
├── test_gradientcheck.m
├── test_hessvec_fd.m
├── test_lbfgs.m
├── test_ncg.m
├── test_poblano_linsearch.m
├── test_poblano_out.m
├── test_tn.m
├── unittest_sum_sin.m
└── unittest_sum_squares.m
└── tn.m
/.gitignore:
--------------------------------------------------------------------------------
1 | *~
2 |
--------------------------------------------------------------------------------
/CONTRIBUTION_GUIDE.md:
--------------------------------------------------------------------------------
1 | # Poblano Toolbox Contribution Guide
2 |
3 | ## Checklist
4 |
5 | - [ ] **Issue** Before the pull request, submit an issue for the change, providing as much detailed information as possible. For bug reports, please provide enough information to reproduce the problem.
6 |
7 | - [ ] **Fork** Create a branch or fork of the code and make your changes.
8 |
9 | - [ ] **Help Comments** Create or update comments for the m-files, following the style of the existing files. Be sure to explain all code options.
10 |
11 | - [ ] **Documentation** For any major new functionality, please follow the following steps.
12 | - [ ] Add MATLAB script documentation in the `doc` directory with the name `XXX_doc.m`
13 | - [ ] Use the MATLAB `publish` command on `XXX_doc.m` and the `mxdom2simplehtml_poblano.xsl` as the stylesheet to create a new file in `doc\html`
14 | - [ ] Add a pointer to this documentation file in `doc\html\helptoc.xml`
15 | - [ ] Add pointers in any related higher-level files, e.g., a new method for be referenced in the `html/methods.html` file
16 | - [ ] Add link to HTML documentation from help comments in function
17 |
18 | - [ ] **Tests** Create or update tests in the `tests` directory, especially for bug fixes or strongly encouraged for new code.
19 |
20 | - [ ] **Contents** If new functions were added at top level, go to `maintenance` and run `update_topcontents` to update the Contents.m file at the top level.
21 |
22 | - [ ] **Release Notes** Include release notes in this checklist in the pull request associated with this contribution. Include any significant bug fixes or additions.
23 |
24 | - [ ] **Contributors List** Update `CONTRIBUTORS.md` with your name and a brief description of the contributions.
25 |
26 | - [ ] **Pass All Tests** Confirm that all tests (including existing tests) pass in `tests` directory.
27 |
28 | - [ ] **Create Pull Request** At any point, create a work-in-progress pull request, referencing the issue number and with this checklist and `WIP` in the header.
29 |
30 | - [ ] **Finalize Pull Request** Once all of the items of this checklist in the pull request have been completed, remove `WIP` from the title of the merge requests and add the Comment, "Completed, ready to submit", and check this item as completed.
31 |
32 |
--------------------------------------------------------------------------------
/CONTRIBUTORS.md:
--------------------------------------------------------------------------------
1 | # Primary POC
2 | * Daniel M. Dunlavy, dmdunla@sandia.gov
3 |
4 | # Contributors
5 | * Evrim Acar - Original author
6 | * Daniel M. Dunlavy - Original author, tests
7 | * Tamara G. Kolda - Original author
8 | * Dianne P. O'Leary - ```cstep.m```, ```cvsrch.m```
9 |
10 | # Other Contributions
11 | * This product includes software developed by the University of
12 | Chicago, as Operator of Argonne National Laboratory. (```cstep.m```,
13 | ```cvsrch.m```)
14 |
15 |
16 |
--------------------------------------------------------------------------------
/COPYRIGHT.txt:
--------------------------------------------------------------------------------
1 | Poblano Toolbox for MATLAB
2 |
3 | Copyright
4 |
5 | Copyright 2009 National Technology & Engineering Solutions of Sandia,
6 | LLC (NTESS). Under the terms of Contract DE-NA0003525 with NTESS, the
7 | U.S. Government retains certain rights in this software.
8 |
9 | NOTICE:
10 |
11 | For five (5) years from 06/20/2018 the United States Government is
12 | granted for itself and others acting on its behalf a paid-up,
13 | nonexclusive, irrevocable worldwide license in this data to reproduce,
14 | prepare derivative works, and perform publicly and display publicly,
15 | by or on behalf of the Government. There is provision for the possible
16 | extension of the term of this license. Subsequent to that period or
17 | any extension granted, the United States Government is granted for
18 | itself and others acting on its behalf a paid-up, nonexclusive,
19 | irrevocable worldwide license in this data to reproduce, prepare
20 | derivative works, distribute copies to the public, perform publicly
21 | and display publicly, and to permit others to do so. The specific term
22 | of the license can be identified by inquiry made to National
23 | Technology and Engineering Solutions of Sandia, LLC or DOE.
24 |
25 | NEITHER THE UNITED STATES GOVERNMENT, NOR THE UNITED STATES DEPARTMENT
26 | OF ENERGY, NOR NATIONAL TECHNOLOGY AND ENGINEERING SOLUTIONS OF
27 | SANDIA, LLC, NOR ANY OF THEIR EMPLOYEES, MAKES ANY WARRANTY, EXPRESS
28 | OR IMPLIED, OR ASSUMES ANY LEGAL RESPONSIBILITY FOR THE ACCURACY,
29 | COMPLETENESS, OR USEFULNESS OF ANY INFORMATION, APPARATUS, PRODUCT, OR
30 | PROCESS DISCLOSED, OR REPRESENTS THAT ITS USE WOULD NOT INFRINGE
31 | PRIVATELY OWNED RIGHTS.
32 |
33 | Any licensee of this software has the obligation and responsibility to
34 | abide by the applicable export control laws, regulations, and general
35 | prohibitions relating to the export of technical data. Failure to
36 | obtain an export control license or other authority from the
37 | Government may result in criminal liability under U.S. laws.
38 |
39 |
40 |
--------------------------------------------------------------------------------
/Contents.m:
--------------------------------------------------------------------------------
1 | % Poblano Toolbox (Sandia National Labs)
2 | % Version 1.2 26-APR-2019
3 | %
4 | % Poblano Toolbox for large-scale nonlinear optimization.
5 | %
6 | % cstep - More-Thuente line search step from MINPACK.
7 | % cvsrch - More-Thuente line search from MINPACK.
8 | % example1 - Example function for Poblano Toolbox routines.
9 | % example2 - Matrix factorization.
10 | % example2_extract - Matrix factorization example: extraction.
11 | % example2_init - Matrix factorization example: initialization.
12 | % gradientcheck - Finite difference verification of analytic gradients.
13 | % hessvec_fd - Hessian vector product finite difference approximation.
14 | % install_poblano - Script to install Poblano path.
15 | % lbfgs - Limited-memory BFGS minimization (vector-based).
16 | % ncg - Nonlinear conjugate gradient minimization.
17 | % poblano_linesearch - Line search methods in the Poblano Toolbox.
18 | % poblano_out - Standard output parameters for the Poblano Toolbox.
19 | % poblano_params - Standard input parameters for the Poblano Toolbox.
20 | % tn - Truncated Newton minimization.
21 | %
22 | %Poblano Toolbox for MATLAB
23 | %
24 | %Copyright 2009 National Technology & Engineering Solutions of Sandia,
25 | %LLC (NTESS). Under the terms of Contract DE-NA0003525 with NTESS, the
26 | %U.S. Government retains certain rights in this software.
27 |
--------------------------------------------------------------------------------
/INSTALL.md:
--------------------------------------------------------------------------------
1 | ## Installing Poblano Toolbox for MATLAB
2 |
3 | You can obtain a copy of the Poblano Toolbox for MATLAB as follows:
4 | * Clone or download the `master` branch of the git repository at https://github.com/sandialabs/poblano_toolbox
5 | * Download a released version at https://github.com/sandialabs/poblano_toolbox/releases
6 |
7 | Once you have a version of the Poblano Toolbox for MATLAB:
8 | * Start MATLAB.
9 | * Within MATLAB, change to the `poblano_toolbox` directory from above and execute the following commands:
10 |
11 | addpath(pwd) %<-- Add the Poblano toolbox to the MATLAB path
12 | savepath %<-- Save for future MATLAB sessions
13 | or enter the following command:
14 |
15 | install_poblano
16 |
--------------------------------------------------------------------------------
/LICENSE.txt:
--------------------------------------------------------------------------------
1 | Copyright 2009 National Technology & Engineering Solutions of Sandia,
2 | LLC (NTESS). Under the terms of Contract DE-NA0003525 with NTESS, the
3 | U.S. Government retains certain rights in this software.
4 |
5 | Redistribution and use in source and binary forms, with or without
6 | modification, are permitted provided that the following conditions are
7 | met:
8 |
9 | 1. Redistributions of source code must retain the above copyright
10 | notice, this list of conditions and the following disclaimer.
11 |
12 | 2. Redistributions in binary form must reproduce the above copyright
13 | notice, this list of conditions and the following disclaimer in the
14 | documentation and/or other materials provided with the distribution.
15 |
16 | 3. Neither the name of the copyright holder nor the names of its
17 | contributors may be used to endorse or promote products derived from
18 | this software without specific prior written permission.
19 |
20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
21 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
22 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
23 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
24 | HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
25 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
26 | LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
27 | DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
28 | THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
29 | (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
30 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
31 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Poblano Toolbox for MATLAB
2 |
3 | ```
4 | Copyright 2009 National Technology & Engineering Solutions of Sandia,
5 | LLC (NTESS). Under the terms of Contract DE-NA0003525 with NTESS, the
6 | U.S. Government retains certain rights in this software.
7 | ```
8 |
9 | Poblano is a Matlab toolbox of large-scale algorithms for
10 | unconstrained nonlinear optimization problems. The algorithms in
11 | Poblano require only first-order derivative information (e.g.,
12 | gradients for scalar-valued objective functions), and therefore can
13 | scale to very large problems. The driving application for Poblano
14 | development has been tensor decompositions in data analysis
15 | applications (bibliometric analysis, social network analysis,
16 | chemometrics, etc.).
17 |
18 | Poblano optimizers find local minimizers of scalar-valued objective
19 | functions taking vector inputs. The gradient (i.e., first derivative)
20 | of the objective function is required for all Poblano optimizers. The
21 | optimizers converge to a stationary point where the gradient is
22 | approximately zero. A line search satisfying the strong Wolfe
23 | conditions is used to guarantee global convergence of the Poblano
24 | optimizers. The optimization methods in Poblano include several
25 | nonlinear conjugate gradient methods (Fletcher-Reeves, Polak-Ribiere,
26 | Hestenes-Stiefel), a limited-memory quasi-Newton method using BFGS
27 | updates to approximate second-order derivative information, and a
28 | truncated Newton method using finite differences to approximate
29 | second-order derivative information.
30 |
--------------------------------------------------------------------------------
/cstep.m:
--------------------------------------------------------------------------------
1 | function [stx,fx,dx,sty,fy,dy,stp,fp,dp,brackt,info] ...
2 | = cstep(stx,fx,dx,sty,fy,dy,stp,fp,dp,brackt,stpmin,stpmax)
3 | %CSTEP More-Thuente line search step from MINPACK.
4 | %
5 | % Translation of minpack subroutine cstep
6 | % Dianne O'Leary July 1991
7 | % **********
8 | %
9 | % Subroutine cstep
10 | %
11 | % The purpose of cstep is to compute a safeguarded step for
12 | % a linesearch and to update an interval of uncertainty for
13 | % a minimizer of the function.
14 | %
15 | % The parameter stx contains the step with the least function
16 | % value. The parameter stp contains the current step. It is
17 | % assumed that the derivative at stx is negative in the
18 | % direction of the step. If brackt is set true then a
19 | % minimizer has been bracketed in an interval of uncertainty
20 | % with endpoints stx and sty.
21 | %
22 | % The subroutine statement is
23 | %
24 | % subroutine cstep(stx,fx,dx,sty,fy,dy,stp,fp,dp,brackt,
25 | % stpmin,stpmax,info)
26 | %
27 | % where
28 | %
29 | % stx, fx, and dx are variables which specify the step,
30 | % the function, and the derivative at the best step obtained
31 | % so far. The derivative must be negative in the direction
32 | % of the step, that is, dx and stp-stx must have opposite
33 | % signs. On output these parameters are updated appropriately.
34 | %
35 | % sty, fy, and dy are variables which specify the step,
36 | % the function, and the derivative at the other endpoint of
37 | % the interval of uncertainty. On output these parameters are
38 | % updated appropriately.
39 | %
40 | % stp, fp, and dp are variables which specify the step,
41 | % the function, and the derivative at the current step.
42 | % If brackt is set true then on input stp must be
43 | % between stx and sty. On output stp is set to the new step.
44 | %
45 | % brackt is a logical variable which specifies if a minimizer
46 | % has been bracketed. If the minimizer has not been bracketed
47 | % then on input brackt must be set false. If the minimizer
48 | % is bracketed then on output brackt is set true.
49 | %
50 | % stpmin and stpmax are input variables which specify lower
51 | % and upper bounds for the step.
52 | %
53 | % info is an integer output variable set as follows:
54 | % If info = 1,2,3,4,5, then the step has been computed
55 | % according to one of the five cases below. Otherwise
56 | % info = 0, and this indicates improper input parameters.
57 | %
58 | % Subprograms called
59 | %
60 | % FORTRAN-supplied ... abs,max,min,sqrt
61 | % ... dble
62 | %
63 | % Argonne National Laboratory. MINPACK Project. June 1983
64 | % Jorge J. More', David J. Thuente
65 | %
66 | % **********
67 |
68 |
69 | % Redistribution and use in source and binary forms, with or
70 | % without modification, are permitted provided that the
71 | % following conditions are met:
72 | %
73 | % 1. Redistributions of source code must retain the above
74 | % copyright notice, this list of conditions and the following
75 | % disclaimer.
76 | %
77 | % 2. Redistributions in binary form must reproduce the above
78 | % copyright notice, this list of conditions and the following
79 | % disclaimer in the documentation and/or other materials
80 | % provided with the distribution.
81 | %
82 | % 3. The end-user documentation included with the
83 | % redistribution, if any, must include the following
84 | % acknowledgment:
85 | %
86 | % "This product includes software developed by the
87 | % University of Chicago, as Operator of Argonne National
88 | % Laboratory.
89 | %
90 | % Alternately, this acknowledgment may appear in the software
91 | % itself, if and wherever such third-party acknowledgments
92 | % normally appear.
93 | %
94 | % 4. WARRANTY DISCLAIMER. THE SOFTWARE IS SUPPLIED "AS IS"
95 | % WITHOUT WARRANTY OF ANY KIND. THE COPYRIGHT HOLDER, THE
96 | % UNITED STATES, THE UNITED STATES DEPARTMENT OF ENERGY, AND
97 | % THEIR EMPLOYEES: (1) DISCLAIM ANY WARRANTIES, EXPRESS OR
98 | % IMPLIED, INCLUDING BUT NOT LIMITED TO ANY IMPLIED WARRANTIES
99 | % OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE
100 | % OR NON-INFRINGEMENT, (2) DO NOT ASSUME ANY LEGAL LIABILITY
101 | % OR RESPONSIBILITY FOR THE ACCURACY, COMPLETENESS, OR
102 | % USEFULNESS OF THE SOFTWARE, (3) DO NOT REPRESENT THAT USE OF
103 | % THE SOFTWARE WOULD NOT INFRINGE PRIVATELY OWNED RIGHTS, (4)
104 | % DO NOT WARRANT THAT THE SOFTWARE WILL FUNCTION
105 | % UNINTERRUPTED, THAT IT IS ERROR-FREE OR THAT ANY ERRORS WILL
106 | % BE CORRECTED.
107 | %
108 | % 5. LIMITATION OF LIABILITY. IN NO EVENT WILL THE COPYRIGHT
109 | % HOLDER, THE UNITED STATES, THE UNITED STATES DEPARTMENT OF
110 | % ENERGY, OR THEIR EMPLOYEES: BE LIABLE FOR ANY INDIRECT,
111 | % INCIDENTAL, CONSEQUENTIAL, SPECIAL OR PUNITIVE DAMAGES OF
112 | % ANY KIND OR NATURE, INCLUDING BUT NOT LIMITED TO LOSS OF
113 | % PROFITS OR LOSS OF DATA, FOR ANY REASON WHATSOEVER, WHETHER
114 | % SUCH LIABILITY IS ASSERTED ON THE BASIS OF CONTRACT, TORT
115 | % (INCLUDING NEGLIGENCE OR STRICT LIABILITY), OR OTHERWISE,
116 | % EVEN IF ANY OF SAID PARTIES HAS BEEN WARNED OF THE
117 | % POSSIBILITY OF SUCH LOSS OR DAMAGES.
118 | %
119 |
120 | p66 = 0.66;
121 | info = 0;
122 | %
123 | % Check the input parameters for errors.
124 | %
125 | if ((brackt & (stp <= min(stx,sty) | ...
126 | stp >= max(stx,sty))) | ...
127 | dx*(stp-stx) >= 0.0 | stpmax < stpmin)
128 | return
129 | end
130 | %
131 | % Determine if the derivatives have opposite sign.
132 | %
133 | sgnd = dp*(dx/abs(dx));
134 | %
135 | % First case. A higher function value.
136 | % The minimum is bracketed. If the cubic step is closer
137 | % to stx than the quadratic step, the cubic step is taken,
138 | % else the average of the cubic and quadratic steps is taken.
139 | %
140 | if (fp > fx)
141 | info = 1;
142 | bound = 1;
143 | theta = 3*(fx - fp)/(stp - stx) + dx + dp;
144 | s = norm([theta,dx,dp],inf);
145 | gamma = s*sqrt((theta/s)^2 - (dx/s)*(dp/s));
146 | if (stp < stx)
147 | gamma = -gamma;
148 | end
149 | p = (gamma - dx) + theta;
150 | q = ((gamma - dx) + gamma) + dp;
151 | r = p/q;
152 | stpc = stx + r*(stp - stx);
153 | stpq = stx + ((dx/((fx-fp)/(stp-stx)+dx))/2)*(stp - stx);
154 | if (abs(stpc-stx) < abs(stpq-stx))
155 | stpf = stpc;
156 | else
157 | stpf = stpc + (stpq - stpc)/2;
158 | end
159 | brackt = 1;
160 | %
161 | % Second case. A lower function value and derivatives of
162 | % opposite sign. The minimum is bracketed. If the cubic
163 | % step is closer to stx than the quadratic (secant) step,
164 | % the cubic step is taken, else the quadratic step is taken.
165 | %
166 | elseif (sgnd < 0.0)
167 | info = 2;
168 | bound = 0;
169 | theta = 3*(fx - fp)/(stp - stx) + dx + dp;
170 | s = norm([theta,dx,dp],inf);
171 | gamma = s*sqrt((theta/s)^2 - (dx/s)*(dp/s));
172 | if (stp > stx)
173 | gamma = -gamma;
174 | end
175 | p = (gamma - dp) + theta;
176 | q = ((gamma - dp) + gamma) + dx;
177 | r = p/q;
178 | stpc = stp + r*(stx - stp);
179 | stpq = stp + (dp/(dp-dx))*(stx - stp);
180 | if (abs(stpc-stp) > abs(stpq-stp))
181 | stpf = stpc;
182 | else
183 | stpf = stpq;
184 | end
185 | brackt = 1;
186 | %
187 | % Third case. A lower function value, derivatives of the
188 | % same sign, and the magnitude of the derivative decreases.
189 | % The cubic step is only used if the cubic tends to infinity
190 | % in the direction of the step or if the minimum of the cubic
191 | % is beyond stp. Otherwise the cubic step is defined to be
192 | % either stpmin or stpmax. The quadratic (secant) step is also
193 | % computed and if the minimum is bracketed then the the step
194 | % closest to stx is taken, else the step farthest away is taken.
195 | %
196 | elseif (abs(dp) < abs(dx))
197 | info = 3;
198 | bound = 1;
199 | theta = 3*(fx - fp)/(stp - stx) + dx + dp;
200 | s = norm([theta,dx,dp],inf);
201 | %
202 | % The case gamma = 0 only arises if the cubic does not tend
203 | % to infinity in the direction of the step.
204 | %
205 | gamma = s*sqrt(max(0.,(theta/s)^2 - (dx/s)*(dp/s)));
206 | if (stp > stx)
207 | gamma = -gamma;
208 | end
209 | p = (gamma - dp) + theta;
210 | q = (gamma + (dx - dp)) + gamma;
211 | r = p/q;
212 | if (r < 0.0 & gamma ~= 0.0)
213 | stpc = stp + r*(stx - stp);
214 | elseif (stp > stx)
215 | stpc = stpmax;
216 | else
217 | stpc = stpmin;
218 | end
219 | stpq = stp + (dp/(dp-dx))*(stx - stp);
220 | if (brackt)
221 | if (abs(stp-stpc) < abs(stp-stpq))
222 | stpf = stpc;
223 | else
224 | stpf = stpq;
225 | end
226 | else
227 | if (abs(stp-stpc) > abs(stp-stpq))
228 | stpf = stpc;
229 | else
230 | stpf = stpq;
231 | end
232 | end
233 | %
234 | % Fourth case. A lower function value, derivatives of the
235 | % same sign, and the magnitude of the derivative does
236 | % not decrease. If the minimum is not bracketed, the step
237 | % is either stpmin or stpmax, else the cubic step is taken.
238 | %
239 | else
240 | info = 4;
241 | bound = 0;
242 | if (brackt)
243 | theta = 3*(fp - fy)/(sty - stp) + dy + dp;
244 | s = norm([theta,dy,dp],inf);
245 | gamma = s*sqrt((theta/s)^2 - (dy/s)*(dp/s));
246 | if (stp > sty)
247 | gamma = -gamma;
248 | end
249 | p = (gamma - dp) + theta;
250 | q = ((gamma - dp) + gamma) + dy;
251 | r = p/q;
252 | stpc = stp + r*(sty - stp);
253 | stpf = stpc;
254 | elseif (stp > stx)
255 | stpf = stpmax;
256 | else
257 | stpf = stpmin;
258 | end
259 | end
260 | %
261 | % Update the interval of uncertainty. This update does not
262 | % depend on the new step or the case analysis above.
263 | %
264 | if (fp > fx)
265 | sty = stp;
266 | fy = fp;
267 | dy = dp;
268 | else
269 | if (sgnd < 0.0)
270 | sty = stx;
271 | fy = fx;
272 | dy = dx;
273 | end
274 | stx = stp;
275 | fx = fp;
276 | dx = dp;
277 | end
278 | %
279 | % Compute the new step and safeguard it.
280 | %
281 | stpf = min(stpmax,stpf);
282 | stpf = max(stpmin,stpf);
283 | stp = stpf;
284 | if (brackt & bound)
285 | if (sty > stx)
286 | stp = min(stx+p66*(sty-stx),stp);
287 | else
288 | stp = max(stx+p66*(sty-stx),stp);
289 | end
290 | end
291 | return
292 | %
293 | % Last card of subroutine cstep.
294 | %
295 |
296 |
297 |
298 |
299 |
--------------------------------------------------------------------------------
/cvsrch.m:
--------------------------------------------------------------------------------
1 | function [x,f,g,stp,info,nfev] = cvsrch(fcn,x,f,g,stp,s,params)
2 | %CVSRCH More-Thuente line search from MINPACK.
3 | %
4 | % Re-factor of API for use in the Poblano Toolbox,
5 | % Daniel M. Dunlavy, August 2008, March 2009
6 | %
7 | % Translation of minpack subroutine cvsrch
8 | % Dianne O'Leary July 1991
9 | % **********
10 | %
11 | % Subroutine cvsrch
12 | %
13 | % The purpose of cvsrch is to find a step which satisfies
14 | % a sufficient decrease condition and a curvature condition.
15 | % The user must provide a subroutine which calculates the
16 | % function and the gradient.
17 | %
18 | % At each stage the subroutine updates an interval of
19 | % uncertainty with endpoints stx and sty. The interval of
20 | % uncertainty is initially chosen so that it contains a
21 | % minimizer of the modified function
22 | %
23 | % f(x+stp*s) - f(x) - ftol*stp*(gradf(x)'s).
24 | %
25 | % If a step is obtained for which the modified function
26 | % has a nonpositive function value and nonnegative derivative,
27 | % then the interval of uncertainty is chosen so that it
28 | % contains a minimizer of f(x+stp*s).
29 | %
30 | % The algorithm is designed to find a step which satisfies
31 | % the sufficient decrease condition
32 | %
33 | % f(x+stp*s) <= f(x) + ftol*stp*(gradf(x)'s),
34 | %
35 | % and the curvature condition
36 | %
37 | % abs(gradf(x+stp*s)'s)) <= gtol*abs(gradf(x)'s).
38 | %
39 | % If ftol is less than gtol and if, for example, the function
40 | % is bounded below, then there is always a step which satisfies
41 | % both conditions. If no step can be found which satisfies both
42 | % conditions, then the algorithm usually stops when rounding
43 | % errors prevent further progress. In this case stp only
44 | % satisfies the sufficient decrease condition.
45 | %
46 | % The subroutine statement is
47 | %
48 | % subroutine cvsrch(fcn,n,x,f,g,s,stp,ftol,gtol,xtol,
49 | % stpmin,stpmax,maxfev,info,nfev,wa)
50 | % where
51 | %
52 | % fcn is the name of the user-supplied subroutine which
53 | % calculates the function and the gradient. fcn must
54 | % be declared in an external statement in the user
55 | % calling program, and should be written as follows.
56 | %
57 | % subroutine fcn(n,x,f,g)
58 | % integer n
59 | % f
60 | % x(n),g(n)
61 | % ----------
62 | % Calculate the function at x and
63 | % return this value in the variable f.
64 | % Calculate the gradient at x and
65 | % return this vector in g.
66 | % ----------
67 | % return
68 | % end
69 | %
70 | % n is a positive integer input variable set to the number
71 | % of variables.
72 | %
73 | % x is an array of length n. On input it must contain the
74 | % base point for the line search. On output it contains
75 | % x + stp*s.
76 | %
77 | % f is a variable. On input it must contain the value of f
78 | % at x. On output it contains the value of f at x + stp*s.
79 | %
80 | % g is an array of length n. On input it must contain the
81 | % gradient of f at x. On output it contains the gradient
82 | % of f at x + stp*s.
83 | %
84 | % s is an input array of length n which specifies the
85 | % search direction.
86 | %
87 | % stp is a nonnegative variable. On input stp contains an
88 | % initial estimate of a satisfactory step. On output
89 | % stp contains the final estimate.
90 | %
91 | % ftol and gtol are nonnegative input variables. Termination
92 | % occurs when the sufficient decrease condition and the
93 | % directional derivative condition are satisfied.
94 | %
95 | % xtol is a nonnegative input variable. Termination occurs
96 | % when the relative width of the interval of uncertainty
97 | % is at most xtol.
98 | %
99 | % stpmin and stpmax are nonnegative input variables which
100 | % specify lower and upper bounds for the step.
101 | %
102 | % maxfev is a positive integer input variable. Termination
103 | % occurs when the number of calls to fcn is at least
104 | % maxfev by the end of an iteration.
105 | %
106 | % info is an integer output variable set as follows:
107 | %
108 | % info = 0 Improper input parameters.
109 | %
110 | % info = 1 The sufficient decrease condition and the
111 | % directional derivative condition hold.
112 | %
113 | % info = 2 Relative width of the interval of uncertainty
114 | % is at most xtol.
115 | %
116 | % info = 3 Number of calls to fcn has reached maxfev.
117 | %
118 | % info = 4 The step is at the lower bound stpmin.
119 | %
120 | % info = 5 The step is at the upper bound stpmax.
121 | %
122 | % info = 6 Rounding errors prevent further progress.
123 | % There may not be a step which satisfies the
124 | % sufficient decrease and curvature conditions.
125 | % Tolerances may be too small.
126 | %
127 | % nfev is an integer output variable set to the number of
128 | % calls to fcn.
129 | %
130 | % wa is a work array of length n.
131 | %
132 | % Subprograms called
133 | %
134 | % user-supplied......fcn
135 | %
136 | % MINPACK-supplied...cstep
137 | %
138 | % FORTRAN-supplied...abs,max,min
139 | %
140 | % Argonne National Laboratory. MINPACK Project. June 1983
141 | % Jorge J. More', David J. Thuente
142 | %
143 | % **********
144 |
145 | % Redistribution and use in source and binary forms, with or
146 | % without modification, are permitted provided that the
147 | % following conditions are met:
148 | %
149 | % 1. Redistributions of source code must retain the above
150 | % copyright notice, this list of conditions and the following
151 | % disclaimer.
152 | %
153 | % 2. Redistributions in binary form must reproduce the above
154 | % copyright notice, this list of conditions and the following
155 | % disclaimer in the documentation and/or other materials
156 | % provided with the distribution.
157 | %
158 | % 3. The end-user documentation included with the
159 | % redistribution, if any, must include the following
160 | % acknowledgment:
161 | %
162 | % "This product includes software developed by the
163 | % University of Chicago, as Operator of Argonne National
164 | % Laboratory.
165 | %
166 | % Alternately, this acknowledgment may appear in the software
167 | % itself, if and wherever such third-party acknowledgments
168 | % normally appear.
169 | %
170 | % 4. WARRANTY DISCLAIMER. THE SOFTWARE IS SUPPLIED "AS IS"
171 | % WITHOUT WARRANTY OF ANY KIND. THE COPYRIGHT HOLDER, THE
172 | % UNITED STATES, THE UNITED STATES DEPARTMENT OF ENERGY, AND
173 | % THEIR EMPLOYEES: (1) DISCLAIM ANY WARRANTIES, EXPRESS OR
174 | % IMPLIED, INCLUDING BUT NOT LIMITED TO ANY IMPLIED WARRANTIES
175 | % OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, TITLE
176 | % OR NON-INFRINGEMENT, (2) DO NOT ASSUME ANY LEGAL LIABILITY
177 | % OR RESPONSIBILITY FOR THE ACCURACY, COMPLETENESS, OR
178 | % USEFULNESS OF THE SOFTWARE, (3) DO NOT REPRESENT THAT USE OF
179 | % THE SOFTWARE WOULD NOT INFRINGE PRIVATELY OWNED RIGHTS, (4)
180 | % DO NOT WARRANT THAT THE SOFTWARE WILL FUNCTION
181 | % UNINTERRUPTED, THAT IT IS ERROR-FREE OR THAT ANY ERRORS WILL
182 | % BE CORRECTED.
183 | %
184 | % 5. LIMITATION OF LIABILITY. IN NO EVENT WILL THE COPYRIGHT
185 | % HOLDER, THE UNITED STATES, THE UNITED STATES DEPARTMENT OF
186 | % ENERGY, OR THEIR EMPLOYEES: BE LIABLE FOR ANY INDIRECT,
187 | % INCIDENTAL, CONSEQUENTIAL, SPECIAL OR PUNITIVE DAMAGES OF
188 | % ANY KIND OR NATURE, INCLUDING BUT NOT LIMITED TO LOSS OF
189 | % PROFITS OR LOSS OF DATA, FOR ANY REASON WHATSOEVER, WHETHER
190 | % SUCH LIABILITY IS ASSERTED ON THE BASIS OF CONTRACT, TORT
191 | % (INCLUDING NEGLIGENCE OR STRICT LIABILITY), OR OTHERWISE,
192 | % EVEN IF ANY OF SAID PARTIES HAS BEEN WARNED OF THE
193 | % POSSIBILITY OF SUCH LOSS OR DAMAGES.
194 | %
195 |
196 | % Additions from D. Dunlavy
197 | n = length(x);
198 |
199 | % Initialize params
200 | xtol = params.LineSearch_xtol;
201 | ftol = params.LineSearch_ftol;
202 | gtol = params.LineSearch_gtol;
203 | stpmin = params.LineSearch_stpmin;
204 | stpmax = params.LineSearch_stpmax;
205 | maxfev = params.LineSearch_maxfev;
206 |
207 | % move up in case of early termination (before this was initialized
208 | nfev = 0;
209 |
210 | % Start of D. O'Leary translation
211 | p5 = .5;
212 | p66 = .66;
213 | xtrapf = 4;
214 | info = 0;
215 | infoc = 1;
216 | %
217 | % Check the input parameters for errors.
218 | %
219 | if (n <= 0 | stp <= 0.0 | ftol < 0.0 | ...
220 | gtol < 0.0 | xtol < 0.0 | stpmin < 0.0 ...
221 | | stpmax < stpmin | maxfev <= 0)
222 | return
223 | end
224 | %
225 | % Compute the initial gradient in the search direction
226 | % and check that s is a descent direction.
227 | %
228 | dginit = g'*s;
229 | if (dginit >= 0.0)
230 | return
231 | end
232 | %
233 | % Initialize local variables.
234 | %
235 | brackt = 0;
236 | stage1 = 1;
237 | % moved up to initialize before any potential return
238 | % nfev = 0;
239 | finit = f;
240 | dgtest = ftol*dginit;
241 | width = stpmax - stpmin;
242 | width1 = 2*width;
243 | wa = x;
244 | %
245 | % The variables stx, fx, dgx contain the values of the step,
246 | % function, and directional derivative at the best step.
247 | % The variables sty, fy, dgy contain the value of the step,
248 | % function, and derivative at the other endpoint of
249 | % the interval of uncertainty.
250 | % The variables stp, f, dg contain the values of the step,
251 | % function, and derivative at the current step.
252 | %
253 | stx = 0.0;
254 | fx = finit;
255 | dgx = dginit;
256 | sty = 0.0;
257 | fy = finit;
258 | dgy = dginit;
259 | %
260 | % Start of iteration.
261 | %
262 | while (1)
263 | %
264 | % Set the minimum and maximum steps to correspond
265 | % to the present interval of uncertainty.
266 | %
267 | if (brackt)
268 | stmin = min(stx,sty);
269 | stmax = max(stx,sty);
270 | else
271 | stmin = stx;
272 | stmax = stp + xtrapf*(stp - stx);
273 | end
274 | %
275 | % Force the step to be within the bounds stpmax and stpmin.
276 | %
277 | stp = max(stp,stpmin);
278 | stp = min(stp,stpmax);
279 | %
280 | % If an unusual termination is to occur then let
281 | % stp be the lowest point obtained so far.
282 | %
283 | if ((brackt & (stp <= stmin | stp >= stmax)) ...
284 | | nfev >= maxfev-1 | infoc == 0 ...
285 | | (brackt & stmax-stmin <= xtol*stmax))
286 | stp = stx;
287 | end
288 | %
289 | % Evaluate the function and gradient at stp
290 | % and compute the directional derivative.
291 | %
292 | x = wa + stp * s;
293 | [f,g] = feval(fcn,x);
294 | nfev = nfev + 1;
295 | dg = g' * s;
296 | ftest1 = finit + stp*dgtest;
297 | %
298 | % Test for convergence.
299 | %
300 | if ((brackt & (stp <= stmin | stp >= stmax)) | infoc == 0)
301 | info = 6;
302 | end
303 | if (stp == stpmax & f <= ftest1 & dg <= dgtest)
304 | info = 5;
305 | end
306 | if (stp == stpmin & (f > ftest1 | dg >= dgtest))
307 | info = 4;
308 | end
309 | if (nfev >= maxfev)
310 | info = 3;
311 | end
312 | if (brackt & stmax-stmin <= xtol*stmax)
313 | info = 2;
314 | end
315 | if (f <= ftest1 & abs(dg) <= gtol*(-dginit))
316 | info = 1;
317 | end
318 | %
319 | % Check for termination.
320 | %
321 | if (info ~= 0)
322 | return
323 | end
324 | %
325 | % In the first stage we seek a step for which the modified
326 | % function has a nonpositive value and nonnegative derivative.
327 | %
328 | if (stage1 & f <= ftest1 & dg >= min(ftol,gtol)*dginit)
329 | stage1 = 0;
330 | end
331 | %
332 | % A modified function is used to predict the step only if
333 | % we have not obtained a step for which the modified
334 | % function has a nonpositive function value and nonnegative
335 | % derivative, and if a lower function value has been
336 | % obtained but the decrease is not sufficient.
337 | %
338 | if (stage1 & f <= fx & f > ftest1)
339 | %
340 | % Define the modified function and derivative values.
341 | %
342 | fm = f - stp*dgtest;
343 | fxm = fx - stx*dgtest;
344 | fym = fy - sty*dgtest;
345 | dgm = dg - dgtest;
346 | dgxm = dgx - dgtest;
347 | dgym = dgy - dgtest;
348 | %
349 | % Call cstep to update the interval of uncertainty
350 | % and to compute the new step.
351 | %
352 | [stx,fxm,dgxm,sty,fym,dgym,stp,fm,dgm,brackt,infoc] ...
353 | = cstep(stx,fxm,dgxm,sty,fym,dgym,stp,fm,dgm, ...
354 | brackt,stmin,stmax);
355 | %
356 | % Reset the function and gradient values for f.
357 | %
358 | fx = fxm + stx*dgtest;
359 | fy = fym + sty*dgtest;
360 | dgx = dgxm + dgtest;
361 | dgy = dgym + dgtest;
362 | else
363 | %
364 | % Call cstep to update the interval of uncertainty
365 | % and to compute the new step.
366 | %
367 | [stx,fx,dgx,sty,fy,dgy,stp,f,dg,brackt,infoc] ...
368 | = cstep(stx,fx,dgx,sty,fy,dgy,stp,f,dg, ...
369 | brackt,stmin,stmax);
370 | end
371 | %
372 | % Force a sufficient decrease in the size of the
373 | % interval of uncertainty.
374 | %
375 | if (brackt)
376 | if (abs(sty-stx) >= p66*width1)
377 | stp = stx + p5*(sty - stx);
378 | end
379 | width1 = width;
380 | width = abs(sty-stx);
381 | end
382 | %
383 | % End of iteration.
384 | %
385 | end
386 | %
387 | % Last card of subroutine cvsrch.
388 | %
389 |
390 |
391 |
--------------------------------------------------------------------------------
/docs/README.md:
--------------------------------------------------------------------------------
1 | # Poblano Toolbox for MATLAB
2 |
3 | ## Documentation
4 |
5 | To generate the documentation in HTML that will appear in the MATLAB
6 | Dcoumentation, you can run the following MATALB command from within
7 | this directory:
8 |
9 | ```
10 | opts.stylesheet = 'mxdom2simplehtml_poblano.xsl'; D = dir('*_doc.m'); for i = 1:length(D), publish(D(i).name, opts); end
11 | ```
12 |
13 | All of the HTML files will be placed in the ```html``` subdirectory.
14 |
15 | The file ```html/helptoc.xml``` specifies the order of the
16 | documentation as it will appear in the MATLAB Documentation.
--------------------------------------------------------------------------------
/docs/examples_doc.m:
--------------------------------------------------------------------------------
1 | %% Poblano Toolbox for MATLAB: Examples
2 | % This section describes the examples provided with Poblano. These examples
3 | % can be used as templates for other function/gradient M-files.
4 | %%
5 | %
6 | %
7 | %% Solving a Problem Using Poblano
8 | % To solve a problem using Poblano, the following steps are performed:
9 | %
10 | % * *Step 1: Create M-file for objective and gradient.* An M-file which
11 | % takes a vector as input and provides a scalar function value and gradient
12 | % vector (the same size as the input) must be provided to the Poblano
13 | % optimizers.
14 | % * *Step 2: Call the optimizer.* One of the Poblano optimizers is called,
15 | % taking an anonymous function handle to the function to be minimized, a
16 | % starting point, and an optional set of optimizer parameters as inputs.
17 | %
18 | % Poblano provides two example function/gradient M-files:
19 | %
20 | % * |example1|: simple multivariate function
21 | % * |example2|: more complicated function of a matrix variable
22 | %%
23 | %
24 | %
25 | %% Example 1: Multivariate Sum
26 | %
27 | % The following example is a simple multivariate function, $f_1 : R^N \to R$,
28 | % that can be minimized using Poblano:
29 | %%
30 | %
31 | % $$f_1(x) = \sum_{n=1}^N \sin(ax_i)$$
32 | %%
33 | %
34 | % $$\left(\nabla f_1(x)\right)_i = a \cos(ax_i)\; , \quad i=1,\dots,N$
35 | %%
36 | %
37 | % where $a$ is a scalar parameter.
38 | %%
39 | %
40 | % Listed below are the contents of the |example1.m| M-file
41 | % distributed with the Poblano code. This is an example of a self-contained
42 | % function requiring a vector of independent variables and an
43 | % _optional_ scalar input parameter for $a$.
44 | %
45 | % function [f,g]=example1(x,a)
46 | % if nargin < 2, a = 1; end
47 | % f = sum(sin(a*x));
48 | % g = a*cos(a*x);
49 | %%
50 | %
51 | % The following presents a call to |ncg| optimizer using the default
52 | % parameters (see documentation for more
53 | % details) along with the information displayed and output return by
54 | % Poblano. By default, at each iteration Poblano displays the number of
55 | % function evaluations (|FuncEvals|), the function value at the current
56 | % iterate (|F(X)|), and the Euclidean norm of the scaled gradient at the
57 | % current iterate (|||||G(X)||||||/N|, where |N| is the size of |X|) for
58 | % each iteration. The output of Poblano optimizers is a Matlab structure
59 | % containing the solution. See the documentation for more details.
61 | out = ncg(@(x) example1(x,3), pi/4)
62 | %%
63 | % The problem dimension, $N$, of |example1| is determined by the size of the
64 | % initial guess. For example, to solve a problem with $N=10^6$, simply use
65 | % an initial guess vector of size $N=10^6$ when calling the Poblano optimizer:
66 | randn('state',0);
67 | x0 = randn(1e6,1);
68 | out = ncg(@(x) example1(x,3), x0, 'Display', 'final')
69 | %% Example 2: Matrix Decomposition
70 | %
71 | % The following example is a more complicated function involving matrix
72 | % variables. As the Poblano methods require scalar functions with vector
73 | % inputs, variable matrices must be reshaped into vectors first. The
74 | % problem in this example is to find a rank-$k$ decomposition, $UV^T$, of
75 | % a $m \times n$ matrix, $A$, which minimizes the Frobenius norm of the fit
76 | %%
77 | %
78 | % $$f_2 = \frac12\| A - UV^T \|_F^2$$
79 | %%
80 | %
81 | % $$\nabla_U f_2 = -(A-UV^T)V$$
82 | %%
83 | %
84 | % $$\nabla_V f_2 = -(A-UV^T)^T U$$
85 | %%
86 | %
87 | % where $A \in R^{m \times n}$ is a matrix with rank $\geq k$, $U
88 | % \in R^{m \times k}$, and $V \in R^{n \times k}$. This problem can be
89 | % solved using Poblano by providing an M-file that computes the function
90 | % and gradient shown above but that takes $U$ and $V$ as input in
91 | % vectorized form.
92 | %
93 | % This problem can be solved using Poblano by providing an M-?le that
94 | % computes the function and gradient shown above but that takes U and V as
95 | % input in vectorized form.
96 | %%
97 | %
98 | % Listed below are the contents of the |example2.m| M-file
99 | % distributed with the Poblano code. Note that the input |Data| is
100 | % required and is a structure containing the matrix to be decomposed, $A$,
101 | % and the desired rank, $k$. This example also illustrates how the
102 | % vectorized form of the factor matrices, $U$ and $V$, are converted to
103 | % matrix form for the function and gradient computations.
104 | %
105 | % function [f,g]=example2(x,Data)
106 | % % Data setup
107 | % [m,n] = size(Data.A);
108 | % k = Data.rank;
109 | % U = reshape(x(1:m*k),m,k);
110 | % V = reshape(x(m*k+1:m*k+n*k),n,k);
111 | %
112 | % % Function value (residual)
113 | % AmUVt = Data.A-U*V';
114 | % f = 0.5*norm(AmUVt,'fro')^2;
115 | %
116 | % % First derivatives computed in matrix form
117 | % g = zeros((m+n)*k,1);
118 | % g(1:m*k) = -reshape(AmUVt*V,m*k,1);
119 | % g(m*k+1:end) = -reshape(AmUVt'*U,n*k,1);
120 | %%
121 | %
122 | % Included with Poblano are two helper functions and which can be used to
123 | % generate problems instances along with starting points
124 | % (|example2_init.m|) and extract the factors $U$ and $V$ from a solution
125 | % vector (|example2_extract.m|). We show an example of their use below.
126 | randn('state',0);
127 | m = 4; n = 3; k = 2;
128 | [x0,Data] = example2_init(m,n,k)
129 | out = ncg(@(x) example2(x,Data), x0, 'RelFuncTol', 1e-16, 'StopTol', 1e-8, ...
130 | 'MaxFuncEvals',1000,'Display','final')
131 | %%
132 | %
133 | % Extracting the factors from the solution, we see that we have found a
134 | % solution, since the the Euclidean norm of the difference between the
135 | % matrix and the approximate solution is equal to the the $k+1$ singular
136 | % value of $A$ [1, Theorem 2.5.3].
137 | [U,V] = example2_extract(4,3,2,out.X);
138 | norm_diff = norm(Data.A-U*V')
139 | sv = svd(Data.A);
140 | sv_k_plus_1 = sv(k+1)
141 | %%
142 | %
143 | %
144 | %% References
145 | %
146 | % [1] Golub, G. H. and Loan, C. F. V. (1996). _Matrix Computations_.
147 | % Johns Hopkins University Press.
148 | %
149 |
--------------------------------------------------------------------------------
/docs/gradientcheck_doc.m:
--------------------------------------------------------------------------------
1 | %% Poblano Toolbox for MATLAB: Checking Gradient Calculations
2 | % Analytic gradients can be checked using finite difference approximations.
3 | % The Poblano function |gradientcheck| computes the gradient approximations
4 | % and compares the results to the analytic gradient using a user-supplied
5 | % objective function/gradient M-file. The user can choose one of several
6 | % difference formulas as well as the difference step used in the
7 | % computations.
8 | %%
9 | %
10 | %
11 | %% Difference Formulas
12 | % The difference formulas for approximating the gradients in Poblano are
13 | % listed below. For more details on the different formulas, see [1].
14 | %
15 | % _Forward Differences_
16 | %
17 | % $$\frac{\partial f}{\partial x_i}(x) \approx \frac{f(x+he_i)-f(x)}{h}$$
18 | %
19 | % _Backward Differences_
20 | %
21 | % $$\frac{\partial f}{\partial x_i}(x) \approx \frac{f(x)-f(x-he_i)}{h}$$
22 | %
23 | % _Centered Differences_
24 | %
25 | % $$\frac{\partial f}{\partial x_i}(x) \approx \frac{f(x+he_i)-f(x-he_i)}{2h}$$
26 | %
27 | % Note that $e_i$ is a vector the same size as $x$ with a 1 in element $i$
28 | % and zeros elsewhere. The type of finite differences to use is specified
29 | % using the |DifferenceType| input parameter, and the value of $h$ is
30 | % specified using the |DifferenceStep| input parameter.
31 | %
32 | % For a detailed discussion on the impact of the choice of $h$ on the
33 | % quality of the approximation, see [2].
34 | %
35 | %%
36 | %
37 | %
38 | %% Gradient Check Input Parameters
39 | %
40 | % The input parameters available for the |gradientcheck| function
41 | % are presented below.
42 | %
43 | % DifferenceType Difference formula to use {'forward'}
44 | % 'forward' g_i = (f(x+he_i)-f(x)) / h
45 | % 'backward' g_i = (f(x)-f(x-he_i)) / h
46 | % 'centered' g_i = (f(x+he_i)-f(x-he_i)) / (2h)
47 | %
48 | % DifferenceStep Value of h in difference formulae {1e-8}
49 | %%
50 | %
51 | %
52 | %% Gradient Check Output Parameters
53 | %
54 | % The fields in the structure of output parameters generated by the
55 | % |gradientcheck| function are presented below.
56 | %
57 | % G Analytic gradient
58 | %
59 | % GFD FD approximation of gradient
60 | %
61 | % MaxDiff Maximum difference between G and GFD
62 | %
63 | % MaxDiffInd Index of maximum difference between G and GFD
64 | %
65 | % NormGradientDiffs 2-norm of G - GFD
66 | %
67 | % GradientDiffs G - GFD
68 | %
69 | % Params Parameters used to compute FD approximations
70 | %%
71 | %
72 | %
73 | %% Examples
74 | %%
75 | % We use |example1| (described in detail in the
76 | % section) to illustrate
77 | % how to use the |gradientcheck| function to check user-supplied
78 | % gradients. The user provides a function handle to the M-file containing
79 | % their function and gradient computations, a point at which to check the
80 | % gradients, and the type of difference formula to use. Below is an example
81 | % of running the gradient check using each of the difference formulas.
82 | %%
83 | outFD = gradientcheck(@(x) example1(x,3), pi./[4 5 6]','DifferenceType','forward')
84 | outBD = gradientcheck(@(x) example1(x,3), pi./[4 5 6]','DifferenceType','backward')
85 | outCD = gradientcheck(@(x) example1(x,3), pi./[4 5 6]','DifferenceType','centered')
86 | %%
87 | %
88 | % Note the different gradients produced using the various differencing
89 | % formulas:
90 | format long
91 | [outFD.G outFD.GFD outBD.GFD outCD.GFD]
92 | format short
93 | %% References
94 | %
95 | % [1] Nocedal, J. and Wright S. J. (1999).
96 | % _Numerical Optimization_. Springer.
97 | %
98 | % [2] Overton, M. L. (2001).
99 | % _Numerical Computing with IEEE Floating Point Arithmetic_.
100 | % Society for Industrial and Applied Mathematics.
101 |
--------------------------------------------------------------------------------
/docs/helptoc.xml:
--------------------------------------------------------------------------------
1 |
2 |
3 |
4 |
5 | Poblano Toolbox
6 |
7 | Overview
8 |
9 |
10 | Parameters
11 | Input
12 | Output
13 |
14 |
15 | Methods
16 | Nonlinear Conjugate Gradient (ncg)
17 | Limited-Memory BFGS (lbfgs)
18 | Truncated Newton (tn)
19 |
20 |
21 | Examples
22 |
23 |
24 | Checking Gradient Calculations
25 |
26 |
27 |
28 |
29 |
30 |
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq00543772986039519777.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq00543772986039519777.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq02586487823804834451.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq02586487823804834451.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq02619125812656254059.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq02619125812656254059.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq02739270504201626537.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq02739270504201626537.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq02978325375087395648.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq02978325375087395648.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq03421469680577370400.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq03421469680577370400.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq03672095713503266041.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq03672095713503266041.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq05147331747641807187.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq05147331747641807187.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq05495199406804600545.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq05495199406804600545.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq05508344529756732484.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq05508344529756732484.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq06327244962903926757.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq06327244962903926757.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq07741076879175499892.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq07741076879175499892.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq11316155984473799093.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq11316155984473799093.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq11475201221685588019.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq11475201221685588019.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq11638118408029626293.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq11638118408029626293.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq12267211811494638899.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq12267211811494638899.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq12534387414789875675.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq12534387414789875675.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq13029888333516263543.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq13029888333516263543.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq15636846968047188835.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq15636846968047188835.png
--------------------------------------------------------------------------------
/docs/html/examples_doc_eq17937321231062034804.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/sandialabs/poblano_toolbox/2bc59e21132d51eac7336777a2fc634b391af648/docs/html/examples_doc_eq17937321231062034804.png
--------------------------------------------------------------------------------
/docs/html/gradientcheck_doc.html:
--------------------------------------------------------------------------------
1 |
2 |
4 |
5 |
6 | Poblano Toolbox for MATLAB: Checking Gradient Calculations
Poblano Toolbox for MATLAB: Checking Gradient Calculations
Analytic gradients can be checked using finite difference approximations. The Poblano function gradientcheck computes the gradient approximations and compares the results to the analytic gradient using a user-supplied objective function/gradient M-file. The user can choose one of several difference formulas as well as the difference step used in the computations.
The difference formulas for approximating the gradients in Poblano are listed below. For more details on the different formulas, see [1].
Forward Differences
Backward Differences
Centered Differences
Note that is a vector the same size as with a 1 in element and zeros elsewhere. The type of finite differences to use is specified using the DifferenceType input parameter, and the value of is specified using the DifferenceStep input parameter.
For a detailed discussion on the impact of the choice of on the quality of the approximation, see [2].
Gradient Check Input Parameters
The input parameters available for the gradientcheck function are presented below.
DifferenceType Difference formula to use {'forward'}
72 | 'forward' g_i = (f(x+he_i)-f(x)) / h
73 | 'backward' g_i = (f(x)-f(x-he_i)) / h
74 | 'centered' g_i = (f(x+he_i)-f(x-he_i)) / (2h)
DifferenceStep Value of h in difference formulae {1e-8}
Gradient Check Output Parameters
The fields in the structure of output parameters generated by the gradientcheck function are presented below.
G Analytic gradient
GFD FD approximation of gradient
MaxDiff Maximum difference between G and GFD
MaxDiffInd Index of maximum difference between G and GFD
NormGradientDiffs 2-norm of G - GFD
GradientDiffs G - GFD
Params Parameters used to compute FD approximations
Examples
We use example1 (described in detail in the Poblano Examples section) to illustrate how to use the gradientcheck function to check user-supplied gradients. The user provides a function handle to the M-file containing their function and gradient computations, a point at which to check the gradients, and the type of difference formula to use. Below is an example of running the gradient check using each of the difference formulas.
Poblano Toolbox for MATLAB: Optimization Output Parameters
Each of the optimization methods in Poblano outputs a single structure containing fields for the approximate solution, function and gradient values at the solution, and various information about the optimization run (e.g., number of function evaluations, etc.). The Poblano function poblano_out is used by the optimization methods to set the output parameters.
Each of the optimization methods in Poblano outputs a single structure containing fields described below.
X Final iterate
F Function value at X
G Gradient at X
Params Input parameters used for the minimization method (as
72 | parsed Matlab inputParser object)
FuncEvals Number of function evaluations performed
Iters Number of iterations performed (see individual minimization
73 | routines for details on what each iteration consists of
ExitFlag Termination flag, with one of the following values
74 | 0 : scaled gradient norm < StopTol input parameter)
75 | 1 : maximum number of iterations exceeded
76 | 2 : maximum number of function values exceeded
77 | 3 : relative change in function value < RelFuncTol input parameter
78 | 4 : NaNs found in F, G, or ||G||
ExitDescription Text description of the termination flag
Optional Trace Output Parameters
Additional output parameters returned by the Poblano optimization methods are presented below. The histories (i.e., traces) of different variables and parameters at each iteration are returned as output parameters if the corresponding input parameters are set to true (see the Optimization Input Parameters documentation for more details on the input parameters).
TraceX History of X (iterates)
TraceFunc History of the function values of the iterates
TraceRelFunc History of the relative difference between the function
79 | values at the current and previous iterates
TraceGrad History of the gradients of the iterates
TraceGradNorm History of the norm of the gradients of the iterates
TraceFuncEvals History of the number of function evaluations performed
80 | at each iteration
Example Output
The following example shows the output produced when the default input parameters are used.
We can also see the values of X (current iterate) and its gradient G for each iteration (including iteration 0, which just computes the function and gradient values of the initial point):
The Poblano Toolbox for MATLAB enables large-scale optimization of
19 | scalar-valued functions using first-order derivative information only.
20 |
21 |
Functionality
22 |
23 |
The Poblano Toolbox provides the following capabilities for
24 | large-scale optimization. This documention is provided via the
25 | MATLAB help interface under "Supplemental Software".
26 |
27 |
28 |
29 |
Overview - An overview of
30 | the entire toolbox.
31 |
Parameters - Input and
32 | output parameters for the optimziation methods are described
33 | in detail here.
34 |
Methods - The optimization
35 | methods available in Poblano are described here. Current
36 | methods include Nonlinear Conjugate Gradient, Limited Memory
37 | BFGS, and Truncated Newton.
38 |
Examples - Several
39 | examples are provided to illustrate how Poblano methods can be
40 | used in practice.
41 |
Checking Gradient
42 | Calculations - To validate gradient functions, a function
43 | is provided to check the analytic gradients provided by the
44 | user with finite difference approximations.
45 |
46 |
This is an open-source project hosted on GITLAB at
62 | http://github.com/sandialabs/poblano_toolbox.
63 | Visit this website to submit bug reports and suggestions for
64 | improvement.
65 |
Please email dmdunla@sandia.gov
79 | with any questions about the toolbox that cannot be resolved via issue reporting.
80 | Stories of its usefulness are especially welcome.
81 | We will try to respond to every email may not always be successful due to the volume of emails.
82 |
83 |
86 |
87 |
88 |
--------------------------------------------------------------------------------
/docs/lbfgs_doc.m:
--------------------------------------------------------------------------------
1 | %% Poblano Toolbox for MATLAB: Limited-Memory BFGS Optimization
2 | % Limited-memory quasi-Newton methods [2] are a class of methods that
3 | % compute and/or maintain simple, compact approximations of the Hessian
4 | % matrices of second derivatives, which are used determining search
5 | % directions. Poblano includes the limited-memory BFGS (L-BFGS) method, a
6 | % variant of these methods whose Hessian approximations are based on the
7 | % BFGS method (see [1] for more details).
8 | %
9 | % The Poblano function for the L-BFGS method is called |lbfgs|.
10 | %%
11 | %
12 | %
13 | %% Introduction
14 | % The general steps of L-BFGS methods are given below in high-level
15 | % pseudo-code [2]:
16 | %%
17 | %
18 | %
19 | % $$
20 | % \begin{tabular}{ll}
21 | % \emph{1.} & Input: $x_0$, a starting point; $M > 0$, an integer\\
22 | % \emph{2.} & Evaluate $f_0 = f(x_0)$, $g_0 = \nabla f(x_0)$\\
23 | % \emph{3.} & Set $p_0 = -g_0$, $\gamma_0 = 1$, $i = 0$\\
24 | % \emph{4.} & \textbf{while} $\|g_i\| > 0$\\
25 | % \emph{5.} & \qquad Choose an initial Hessian approximation: $H_i^0 = \gamma_i I$\\
26 | % \emph{6.} & \qquad Compute a step direction $p_i = -r$ using {\tt TwoLoopRecursion} method\\
27 | % \emph{7.} & \qquad Compute a step length $\alpha_i$ and set $x_{i+1} = x_i + \alpha_i p_i$\\
28 | % \emph{8.} & \qquad Set $g_i = \nabla f(x_{i+1})$\\
29 | % \emph{9.} & \qquad \textbf{if} $i > M$\\
30 | % \emph{10.} & \qquad \qquad Discard vectors $\left\{s_{i-m}, y_{i-m}\right\}$ from storage\\
31 | % \emph{11.} & \qquad \textbf{end if}\\
32 | % \emph{12.} & \qquad \qquad Set and store $s_i = x_{i+1}-x_i$ and $y_i = g_{i+1} - g_i$\\
33 | % \emph{13.} & \qquad Set $i = i + 1$\\
34 | % \emph{14.} & \textbf{end while}\\
35 | % \emph{15.} & Output: $x_i \approx x^*$\\
36 | % \end{tabular}
37 | % $$
38 | %
39 | %%
40 | % *Computing the step direction*
41 | %
42 | % In Step 6 in the above method, the computation of the step direction is
43 | % performed using the following method (assume we are at iteration $i$) [2]:
44 | %%
45 | %
46 | % $$
47 | % \begin{tabular}{ll}
48 | % \multicolumn{2}{l}{\tt TwoLoopRecursion}\\
49 | % \emph{1.} & $q = g_i$\\
50 | % \emph{2.} & \textbf{for} $k = i-1, i-2, \dots, i-m$\\
51 | % \emph{3.} & \qquad $a_k = (s_k^Tq)/(y_k^Ts_k)$\\
52 | % \emph{4.} & \qquad $q = q - a_k y_k$\\
53 | % \emph{5.} & \textbf{end for}\\
54 | % \emph{6.} & $r = H_i^0 q$\\
55 | % \emph{7.} & \textbf{for} $k = i-m, i-m+1, \dots, i-1$\\
56 | % \emph{8.} & \qquad $b = (y_k^Tr) / (y_k^Ts_k)$\\
57 | % \emph{9.} & \qquad $r = r + (a_k - b)s_k$\\
58 | % \emph{10.} & \textbf{end for}\\
59 | % \emph{11.} & Output: $r = H_ig_i$\\
60 | % \end{tabular}
61 | % $$
62 | %%
63 | %
64 | %
65 | %% Method Specific Input Parameters
66 | %
67 | % The input parameters specific to the |lbfgs| method are presented below.
68 | % See the
69 | % documentation for more details on the Poblano parameters shared across
70 | % all methods.
71 | %
72 | % M Limited memory parameter {5}
73 | %%
74 | %
75 | %
76 | %% Default Input Parameters
77 | % The default input parameters are returned with the following call to
78 | % |lbfgs|:
79 | params = lbfgs('defaults')
80 | %%
81 | %
82 | % See the
83 | % documentationfor more details on the Poblano parameters shared across
84 | % all methods.
85 | %%
86 | %
87 | %
88 | %% Examples
89 | % Below are the results of using the |lbfgs| method in Poblano to solve
90 | % example problems solved using the |ncg| method in the documentation. Note that the
92 | % different methods lead to slightly different solutions and a different
93 | % number of function evaluations.
94 | %%
95 | % *Example 1* (from )
96 | %
97 | % In this example, we have $x \in R^{10}$ and $a = 3$, and use a random
98 | % starting point.
99 | randn('state',0);
100 | x0 = randn(10,1)
101 | out = lbfgs(@(x) example1(x,3), x0)
102 | %%
103 | % *Example 2* (from )
104 | %
105 | % In this example, we compute a rank-4 approximation to a $4 \times 4$
106 | % Pascal matrix (generated using the Matlab function |pascal(4)|). The
107 | % starting point is random vector. Note that in the interest of space,
108 | % Poblano is set to display only the final iteration is this example.
109 | m = 4; n = 4; k = 4;
110 | Data.rank = k;
111 | Data.A = pascal(m);
112 | randn('state',0);
113 | x0 = randn((m+n)*k,1);
114 | out = lbfgs(@(x) example2(x,Data), x0, 'Display', 'final')
115 | %%
116 | %
117 | % As for the |ncg| method, the fact that
118 | % |out.ExitFlag| > 0 indicates that the method did not
119 | % converge to the specified tolerance (i.e., using the default |StopTol|
120 | % input parameter value of |1e-5|). Since the maximum number of function
121 | % evaluations was exceeded, we can increasing the number of maximum numbers
122 | % of function evaluations and iterations allowed, and the optimizer
123 | % converges to a solution within the specified tolerance.
124 | out = lbfgs(@(x) example2(x,Data), x0, 'MaxIters',1000, ...
125 | 'MaxFuncEvals',10000,'Display','final')
126 | %%
127 | %
128 | % Verifying the solution, we see that we find a matrix decomposition which
129 | % fits the matrix with very small relative error (given the stopping
130 | % tolerance of |1e-5| used by the optimizer).
131 | [U,V] = example2_extract(m,n,k,out.X);
132 | norm(Data.A-U*V')/norm(Data.A)
133 | %%
134 | %
135 | % For Example 2, we see that |lbfgs| requires slightly fewer
136 | % function evaluations than |ncg| to solve the problem (143 versus
137 | % 175). Performance of the different methods in Poblano is dependent on
138 | % both the method and the particular parameters chosen. Thus, it is
139 | % recommended that several test runs on smaller problems are performed
140 | % initially using the different methods to help decide which method and set
141 | % of parameters works best for a particular class of problems.
142 | %%
143 | %
144 | %
145 | %% References
146 | %
147 | % [1] Dennis, Jr., J. E. and Schnabel, R. B. (1996). _Numerical Methods
148 | % for Unconstrained Optimization and Nonlinear Equations_. SIAM.
149 | %
150 | % [2] Nocedal, J. and Wright S. J. (1999).
151 | % _Numerical Optimization_. Springer.
152 |
153 |
--------------------------------------------------------------------------------
/docs/mxdom2simplehtml_poblano.xsl:
--------------------------------------------------------------------------------
1 |
2 |
3 |
9 |
10 | ]>
11 |
16 |
19 |
20 |
21 |
22 |
23 |
24 |
25 |
26 |
27 |
28 |
29 |
30 |
31 |
32 |
33 |
34 |
35 |
36 | This HTML was auto-generated from MATLAB code.
37 | To make changes, update the MATLAB code and republish this document.
38 |
39 |
40 | <xsl:value-of select="$title"/>
41 |
42 |
43 | MATLAB
44 |
45 |
46 |
47 |
48 |
49 |
50 | .m
51 |
52 |
53 |
54 |
55 |
56 |
57 |
58 |
59 |
60 |
61 |
is a vector the same size as 
with a 1 in element 
and zeros elsewhere. The type of finite differences to use is specified using the DifferenceType input parameter, and the value of 
is specified using the DifferenceStep input parameter.