├── README.md └── notebooks ├── MathProgBase Newton solver.ipynb ├── JuMP-Sudoku.ipynb ├── DualNumbers.ipynb ├── JuMP-NetRevMgmt.ipynb ├── JuMP-SensitivityAnalysis.ipynb ├── Shuvomoy - Benders decomposition.ipynb ├── Chiwei Yan - Cutting Stock.ipynb └── Shuvomoy - Column generation.ipynb /README.md: -------------------------------------------------------------------------------- 1 | **This repository is old and unmaintained, i.e. the examples are possibly outdated wrt the latest version of JuMP. 2 | Please use [JuMPTutorials.jl](https://github.com/JuliaOpt/JuMPTutorials.jl) instead.** 3 | -------------------------------------------------------------------------------- /notebooks/MathProgBase Newton solver.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "**Description**: An example of implementing a derivative-based nonlinear solver in Julia and hooking it into MathProgBase. MathProgBase connects solvers to modeling interfaces like JuMP and AMPL, which provide automatic computation of exact first and second-order derivatives.\n", 8 | "\n", 9 | "**Author**: Miles Lubin\n", 10 | "\n", 11 | "**License**: \"Creative
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License." 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "metadata": {}, 17 | "source": [ 18 | "## Newton's method in less than 50 lines\n", 19 | "\n", 20 | "In this notebook, we demonstrate how to implement [Newton's method](https://en.wikipedia.org/wiki/Newton%27s_method_in_optimization), by querying derivatives through the [MathProgBase nonlinear interface](http://mathprogbasejl.readthedocs.org/en/latest/nlp.html). We then demonstrate using this new solver from both JuMP and AMPL." 21 | ] 22 | }, 23 | { 24 | "cell_type": "code", 25 | "execution_count": 1, 26 | "metadata": { 27 | "collapsed": false 28 | }, 29 | "outputs": [], 30 | "source": [ 31 | "using MathProgBase" 32 | ] 33 | }, 34 | { 35 | "cell_type": "markdown", 36 | "metadata": {}, 37 | "source": [ 38 | "First we define the ``NewtonSolver`` object, which holds solver options (here there are none). Solver objects are used to instantiate a ``NewtonData`` object, which is the object which then solves the instance of the optimization problem. The ``NewtonData`` type is a subtype of the ``AbstractNonlinearModel`` type and stores all the instance data we need to run the algorithm." 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": 2, 44 | "metadata": { 45 | "collapsed": false 46 | }, 47 | "outputs": [], 48 | "source": [ 49 | "type NewtonSolver <: MathProgBase.AbstractMathProgSolver\n", 50 | "end\n", 51 | "\n", 52 | "type NewtonData <: MathProgBase.AbstractNonlinearModel\n", 53 | " numVar::Int\n", 54 | " d # NLP evaluator\n", 55 | " x::Vector{Float64}\n", 56 | " hess_I::Vector{Int} # 1st component of Hessian sparsity pattern\n", 57 | " hess_J::Vector{Int} # 2nd component of Hessian sparsity pattern\n", 58 | " status::Symbol\n", 59 | "end\n", 60 | "\n", 61 | "MathProgBase.NonlinearModel(solver::NewtonSolver) = NewtonData(0,nothing,Float64[],Int[],Int[],:Uninitialized);" 62 | ] 63 | }, 64 | { 65 | "cell_type": "markdown", 66 | "metadata": {}, 67 | "source": [ 68 | "This method is called when we're about to solve the optimization problem. It provides basic problem dimensions and lower and upper bound vectors, as well as the ``AbstractNLPEvaluator`` object, which is an oracle that can be used by the solver to query function and derivative evaluations." 69 | ] 70 | }, 71 | { 72 | "cell_type": "code", 73 | "execution_count": 3, 74 | "metadata": { 75 | "collapsed": false 76 | }, 77 | "outputs": [], 78 | "source": [ 79 | "function MathProgBase.loadproblem!(m::NewtonData, numVar, numConstr, l, u, lb, ub, sense, d::MathProgBase.AbstractNLPEvaluator)\n", 80 | " @assert numConstr == 0 # we don't handle constraints\n", 81 | " @assert all(l .== -Inf) && all(u .== Inf) # or variable bounds\n", 82 | " @assert sense == :Min # or maximization\n", 83 | " \n", 84 | " MathProgBase.initialize(d, [:Grad, :Hess]) # request gradient and hessian evaluations\n", 85 | " \n", 86 | " m.d = d\n", 87 | " m.numVar = numVar\n", 88 | " m.x = zeros(numVar)\n", 89 | " m.hess_I, m.hess_J = MathProgBase.hesslag_structure(d)\n", 90 | "end;" 91 | ] 92 | }, 93 | { 94 | "cell_type": "markdown", 95 | "metadata": {}, 96 | "source": [ 97 | "And here's the actual implementation of Newton's method. We assume:\n", 98 | "\n", 99 | "- The domain of the objective function is $\\mathbb{R}^n$\n", 100 | "- The objective function is strictly convex, so the Hessian matrix is positive definite.\n", 101 | "\n", 102 | "We also fix the step size to 1 and do not perform a line search." 103 | ] 104 | }, 105 | { 106 | "cell_type": "code", 107 | "execution_count": 4, 108 | "metadata": { 109 | "collapsed": false 110 | }, 111 | "outputs": [], 112 | "source": [ 113 | "function MathProgBase.optimize!(m::NewtonData)\n", 114 | " \n", 115 | " iteration = 0\n", 116 | "\n", 117 | " ∇f = Array(Float64,m.numVar)\n", 118 | " hess_val = Array(Float64, length(m.hess_I)) # array to store nonzeros in Hessian\n", 119 | " MathProgBase.eval_grad_f(m.d, ∇f, m.x) # writes gradient to ∇f vector\n", 120 | " ∇f_norm = norm(∇f)\n", 121 | " while ∇f_norm > 1e-5\n", 122 | " println(\"Iteration $iteration. Gradient norm $∇f_norm, x = $(m.x)\");\n", 123 | " \n", 124 | " # Evaluate the Hessian of the Lagrangian.\n", 125 | " # We don't have any constraints, so this is just the Hessian.\n", 126 | " MathProgBase.eval_hesslag(m.d, hess_val, m.x, 1.0, Float64[])\n", 127 | " \n", 128 | " # The derivative evaluator provides the Hessian matrix\n", 129 | " # in lower-triangular sparse (i,j,value) triplet format,\n", 130 | " # so now we convert this into a Julia symmetric sparse matrix object.\n", 131 | " H = Symmetric(sparse(m.hess_I,m.hess_J,hess_val),:L)\n", 132 | " \n", 133 | " m.x = m.x - H\\∇f # newton step\n", 134 | " \n", 135 | " MathProgBase.eval_grad_f(m.d, ∇f, m.x)\n", 136 | " ∇f_norm = norm(∇f)\n", 137 | " iteration += 1\n", 138 | " end\n", 139 | " \n", 140 | " println(\"Iteration $iteration. Gradient norm $∇f_norm, x = $(m.x), DONE\");\n", 141 | " \n", 142 | " m.status = :Optimal \n", 143 | "end;" 144 | ] 145 | }, 146 | { 147 | "cell_type": "markdown", 148 | "metadata": {}, 149 | "source": [ 150 | "We implement a few methods to provide starting solutions and query solution status." 151 | ] 152 | }, 153 | { 154 | "cell_type": "code", 155 | "execution_count": 5, 156 | "metadata": { 157 | "collapsed": false 158 | }, 159 | "outputs": [], 160 | "source": [ 161 | "MathProgBase.setwarmstart!(m::NewtonData,x) = (m.x = x)\n", 162 | "MathProgBase.status(m::NewtonData) = m.status\n", 163 | "MathProgBase.getsolution(m::NewtonData) = m.x\n", 164 | "MathProgBase.getconstrduals(m::NewtonData) = [] # lagrange multipliers on constraints (none)\n", 165 | "MathProgBase.getreducedcosts(m::NewtonData) = zeros(m.numVar) # lagrange multipliers on variable bounds\n", 166 | "MathProgBase.getobjval(m::NewtonData) = MathProgBase.eval_f(m.d, m.x);" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": 6, 172 | "metadata": { 173 | "collapsed": false 174 | }, 175 | "outputs": [], 176 | "source": [ 177 | "using JuMP" 178 | ] 179 | }, 180 | { 181 | "cell_type": "code", 182 | "execution_count": 7, 183 | "metadata": { 184 | "collapsed": false 185 | }, 186 | "outputs": [ 187 | { 188 | "name": "stdout", 189 | "output_type": "stream", 190 | "text": [ 191 | "Iteration 0. Gradient norm 11.661903789690601, x = [0.0,0.0]\n", 192 | "Iteration 1. Gradient norm 1.9860273225978185e-15, x = [4.999999999999999,2.9999999999999996], DONE\n" 193 | ] 194 | } 195 | ], 196 | "source": [ 197 | "jumpmodel = Model(solver=NewtonSolver())\n", 198 | "@variable(jumpmodel, x)\n", 199 | "@variable(jumpmodel, y)\n", 200 | "@NLobjective(jumpmodel, Min, (x-5)^2 + (y-3)^2)\n", 201 | "\n", 202 | "status = solve(jumpmodel);" 203 | ] 204 | }, 205 | { 206 | "cell_type": "markdown", 207 | "metadata": {}, 208 | "source": [ 209 | "Note that we've converged to the optimal solution in one step. This is expected when applying Newton's method to a strictly convex quadratic objective function." 210 | ] 211 | }, 212 | { 213 | "cell_type": "markdown", 214 | "metadata": { 215 | "collapsed": true 216 | }, 217 | "source": [ 218 | "### Now let's hook our solver to AMPL!" 219 | ] 220 | }, 221 | { 222 | "cell_type": "markdown", 223 | "metadata": {}, 224 | "source": [ 225 | "AMPL is a commercial algebraic modeling language. You can download a limited demo of AMPL [here](http://ampl.com/try-ampl/download-a-demo-version/) and extract it into the ``ampl-demo`` directory under your home directory to follow along." 226 | ] 227 | }, 228 | { 229 | "cell_type": "code", 230 | "execution_count": null, 231 | "metadata": { 232 | "collapsed": false 233 | }, 234 | "outputs": [], 235 | "source": [ 236 | "mod = \"\"\"\n", 237 | "var x;\n", 238 | "var y;\n", 239 | "\n", 240 | "minimize OBJ:\n", 241 | " (x-5)^2 + (y-3)^2;\n", 242 | "\n", 243 | "write gquad;\n", 244 | "\n", 245 | "\"\"\"\n", 246 | "fd = open(\"quad.mod\",\"w\")\n", 247 | "write(fd, mod)\n", 248 | "close(fd)" 249 | ] 250 | }, 251 | { 252 | "cell_type": "markdown", 253 | "metadata": {}, 254 | "source": [ 255 | "The proprietary `ampl` executable translates the `.mod` file into a `.nl` file which we can directly read from Julia." 256 | ] 257 | }, 258 | { 259 | "cell_type": "code", 260 | "execution_count": null, 261 | "metadata": { 262 | "collapsed": false 263 | }, 264 | "outputs": [], 265 | "source": [ 266 | ";~/ampl-demo/ampl quad.mod" 267 | ] 268 | }, 269 | { 270 | "cell_type": "code", 271 | "execution_count": null, 272 | "metadata": { 273 | "collapsed": false 274 | }, 275 | "outputs": [], 276 | "source": [ 277 | ";cat quad.nl | head -n 3" 278 | ] 279 | }, 280 | { 281 | "cell_type": "markdown", 282 | "metadata": {}, 283 | "source": [ 284 | "The [AmplNLReader](https://github.com/dpo/AmplNLReader.jl) and [AMPLMathProgInterface](https://github.com/mlubin/AMPLMathProgInterface.jl) packages are not yet officially registered as Julia packages. They're not yet recommended for general use.\n", 285 | "\n", 286 | "But anyway, here's a demo of what they can do:" 287 | ] 288 | }, 289 | { 290 | "cell_type": "code", 291 | "execution_count": null, 292 | "metadata": { 293 | "collapsed": false 294 | }, 295 | "outputs": [], 296 | "source": [ 297 | "using AmplNLReader, AMPLMathProgInterface" 298 | ] 299 | }, 300 | { 301 | "cell_type": "markdown", 302 | "metadata": {}, 303 | "source": [ 304 | "Here we read in the `.nl` file and call the ``AMPLMathProgInterface.loadamplproblem!`` method to load the `.nl` file into our solver which we just wrote. Then we just call ``optimize!`` and we're done." 305 | ] 306 | }, 307 | { 308 | "cell_type": "code", 309 | "execution_count": null, 310 | "metadata": { 311 | "collapsed": false 312 | }, 313 | "outputs": [], 314 | "source": [ 315 | "nlp = AmplNLReader.AmplModel(\"quad.nl\")\n", 316 | "\n", 317 | "m = MathProgBase.model(NewtonSolver())\n", 318 | "AMPLMathProgInterface.loadamplproblem!(m, nlp)\n", 319 | "MathProgBase.optimize!(m);" 320 | ] 321 | }, 322 | { 323 | "cell_type": "markdown", 324 | "metadata": {}, 325 | "source": [ 326 | "Again we converge to the optimal solution in one step, but this time we used AMPL to compute the derivatives of the model instead of JuMP." 327 | ] 328 | } 329 | ], 330 | "metadata": { 331 | "kernelspec": { 332 | "display_name": "Julia 0.4.3", 333 | "language": "julia", 334 | "name": "julia-0.4" 335 | }, 336 | "language_info": { 337 | "file_extension": ".jl", 338 | "mimetype": "application/julia", 339 | "name": "julia", 340 | "version": "0.4.3" 341 | } 342 | }, 343 | "nbformat": 4, 344 | "nbformat_minor": 0 345 | } 346 | -------------------------------------------------------------------------------- /notebooks/JuMP-Sudoku.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Disclaimer\n", 8 | "This notebook is only working under the versions:\n", 9 | "\n", 10 | "- JuMP 0.19\n", 11 | "\n", 12 | "- MathOptInterface 0.8.4\n", 13 | "\n", 14 | "- GLPK 0.9.1" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "**Description**: Shows how to solve [Sudoku](http://en.wikipedia.org/wiki/Sudoku) puzzles using integer programming and JuMP.\n", 22 | "\n", 23 | "**Author**: Iain Dunning\n", 24 | "\n", 25 | "**License**: \"Creative
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License." 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "metadata": {}, 31 | "source": [ 32 | "# Solving Sudoku with JuMP\n", 33 | "\n", 34 | "\n", 35 | "

A partially solved Sudoku puzzle

\n", 36 | "\n", 37 | "Sudoku is a popular number puzzle. The goal is to place the digits 1,...,9 on a nine-by-nine grid, with some of the digits already filled in. Your solution must satisfy the following rules:\n", 38 | "\n", 39 | "* The numbers 1 to 9 must appear in each 3x3 square\n", 40 | "* The numbers 1 to 9 must appear in each row\n", 41 | "* The numbers 1 to 9 must appear in each column\n", 42 | "\n", 43 | "This isn't an optimization problem, its actually a *feasibility* problem: we wish to find a feasible solution that satsifies these rules. You can think of it as an optimization problem with an objective of 0.\n", 44 | "\n", 45 | "We can model this problem using 0-1 integer programming: a problem where all the decision variables are binary. We'll use JuMP to create the model, and then we can solve it with any integer programming solver." 46 | ] 47 | }, 48 | { 49 | "cell_type": "code", 50 | "execution_count": 1, 51 | "metadata": {}, 52 | "outputs": [ 53 | { 54 | "data": { 55 | "text/plain": [ 56 | "MathOptInterface.Utilities" 57 | ] 58 | }, 59 | "execution_count": 1, 60 | "metadata": {}, 61 | "output_type": "execute_result" 62 | } 63 | ], 64 | "source": [ 65 | "# Load JuMP\n", 66 | "using JuMP\n", 67 | "using MathOptInterface\n", 68 | "# Load solver package\n", 69 | "using GLPK\n", 70 | "# shortcuts\n", 71 | "const MOI = MathOptInterface\n", 72 | "const MOIU = MathOptInterface.Utilities" 73 | ] 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": {}, 78 | "source": [ 79 | "We will define a binary variable (a variable that is either 0 or 1) for each possible number in each possible cell. The meaning of each variable is as follows:\n", 80 | "\n", 81 | " x[i,j,k] = 1 if and only if cell (i,j) has number k\n", 82 | "\n", 83 | "where `i` is the row and `j` is the column." 84 | ] 85 | }, 86 | { 87 | "cell_type": "code", 88 | "execution_count": 2, 89 | "metadata": {}, 90 | "outputs": [ 91 | { 92 | "data": { 93 | "text/plain": [ 94 | "9×9×9 Array{VariableRef,3}:\n", 95 | "[:, :, 1] =\n", 96 | " x[1,1,1] x[1,2,1] x[1,3,1] x[1,4,1] … x[1,7,1] x[1,8,1] x[1,9,1]\n", 97 | " x[2,1,1] x[2,2,1] x[2,3,1] x[2,4,1] x[2,7,1] x[2,8,1] x[2,9,1]\n", 98 | " x[3,1,1] x[3,2,1] x[3,3,1] x[3,4,1] x[3,7,1] x[3,8,1] x[3,9,1]\n", 99 | " x[4,1,1] x[4,2,1] x[4,3,1] x[4,4,1] x[4,7,1] x[4,8,1] x[4,9,1]\n", 100 | " x[5,1,1] x[5,2,1] x[5,3,1] x[5,4,1] x[5,7,1] x[5,8,1] x[5,9,1]\n", 101 | " x[6,1,1] x[6,2,1] x[6,3,1] x[6,4,1] … x[6,7,1] x[6,8,1] x[6,9,1]\n", 102 | " x[7,1,1] x[7,2,1] x[7,3,1] x[7,4,1] x[7,7,1] x[7,8,1] x[7,9,1]\n", 103 | " x[8,1,1] x[8,2,1] x[8,3,1] x[8,4,1] x[8,7,1] x[8,8,1] x[8,9,1]\n", 104 | " x[9,1,1] x[9,2,1] x[9,3,1] x[9,4,1] x[9,7,1] x[9,8,1] x[9,9,1]\n", 105 | "\n", 106 | "[:, :, 2] =\n", 107 | " x[1,1,2] x[1,2,2] x[1,3,2] x[1,4,2] … x[1,7,2] x[1,8,2] x[1,9,2]\n", 108 | " x[2,1,2] x[2,2,2] x[2,3,2] x[2,4,2] x[2,7,2] x[2,8,2] x[2,9,2]\n", 109 | " x[3,1,2] x[3,2,2] x[3,3,2] x[3,4,2] x[3,7,2] x[3,8,2] x[3,9,2]\n", 110 | " x[4,1,2] x[4,2,2] x[4,3,2] x[4,4,2] x[4,7,2] x[4,8,2] x[4,9,2]\n", 111 | " x[5,1,2] x[5,2,2] x[5,3,2] x[5,4,2] x[5,7,2] x[5,8,2] x[5,9,2]\n", 112 | " x[6,1,2] x[6,2,2] x[6,3,2] x[6,4,2] … x[6,7,2] x[6,8,2] x[6,9,2]\n", 113 | " x[7,1,2] x[7,2,2] x[7,3,2] x[7,4,2] x[7,7,2] x[7,8,2] x[7,9,2]\n", 114 | " x[8,1,2] x[8,2,2] x[8,3,2] x[8,4,2] x[8,7,2] x[8,8,2] x[8,9,2]\n", 115 | " x[9,1,2] x[9,2,2] x[9,3,2] x[9,4,2] x[9,7,2] x[9,8,2] x[9,9,2]\n", 116 | "\n", 117 | "[:, :, 3] =\n", 118 | " x[1,1,3] x[1,2,3] x[1,3,3] x[1,4,3] … x[1,7,3] x[1,8,3] x[1,9,3]\n", 119 | " x[2,1,3] x[2,2,3] x[2,3,3] x[2,4,3] x[2,7,3] x[2,8,3] x[2,9,3]\n", 120 | " x[3,1,3] x[3,2,3] x[3,3,3] x[3,4,3] x[3,7,3] x[3,8,3] x[3,9,3]\n", 121 | " x[4,1,3] x[4,2,3] x[4,3,3] x[4,4,3] x[4,7,3] x[4,8,3] x[4,9,3]\n", 122 | " x[5,1,3] x[5,2,3] x[5,3,3] x[5,4,3] x[5,7,3] x[5,8,3] x[5,9,3]\n", 123 | " x[6,1,3] x[6,2,3] x[6,3,3] x[6,4,3] … x[6,7,3] x[6,8,3] x[6,9,3]\n", 124 | " x[7,1,3] x[7,2,3] x[7,3,3] x[7,4,3] x[7,7,3] x[7,8,3] x[7,9,3]\n", 125 | " x[8,1,3] x[8,2,3] x[8,3,3] x[8,4,3] x[8,7,3] x[8,8,3] x[8,9,3]\n", 126 | " x[9,1,3] x[9,2,3] x[9,3,3] x[9,4,3] x[9,7,3] x[9,8,3] x[9,9,3]\n", 127 | "\n", 128 | "[:, :, 4] =\n", 129 | " x[1,1,4] x[1,2,4] x[1,3,4] x[1,4,4] … x[1,7,4] x[1,8,4] x[1,9,4]\n", 130 | " x[2,1,4] x[2,2,4] x[2,3,4] x[2,4,4] x[2,7,4] x[2,8,4] x[2,9,4]\n", 131 | " x[3,1,4] x[3,2,4] x[3,3,4] x[3,4,4] x[3,7,4] x[3,8,4] x[3,9,4]\n", 132 | " x[4,1,4] x[4,2,4] x[4,3,4] x[4,4,4] x[4,7,4] x[4,8,4] x[4,9,4]\n", 133 | " x[5,1,4] x[5,2,4] x[5,3,4] x[5,4,4] x[5,7,4] x[5,8,4] x[5,9,4]\n", 134 | " x[6,1,4] x[6,2,4] x[6,3,4] x[6,4,4] … x[6,7,4] x[6,8,4] x[6,9,4]\n", 135 | " x[7,1,4] x[7,2,4] x[7,3,4] x[7,4,4] x[7,7,4] x[7,8,4] x[7,9,4]\n", 136 | " x[8,1,4] x[8,2,4] x[8,3,4] x[8,4,4] x[8,7,4] x[8,8,4] x[8,9,4]\n", 137 | " x[9,1,4] x[9,2,4] x[9,3,4] x[9,4,4] x[9,7,4] x[9,8,4] x[9,9,4]\n", 138 | "\n", 139 | "[:, :, 5] =\n", 140 | " x[1,1,5] x[1,2,5] x[1,3,5] x[1,4,5] … x[1,7,5] x[1,8,5] x[1,9,5]\n", 141 | " x[2,1,5] x[2,2,5] x[2,3,5] x[2,4,5] x[2,7,5] x[2,8,5] x[2,9,5]\n", 142 | " x[3,1,5] x[3,2,5] x[3,3,5] x[3,4,5] x[3,7,5] x[3,8,5] x[3,9,5]\n", 143 | " x[4,1,5] x[4,2,5] x[4,3,5] x[4,4,5] x[4,7,5] x[4,8,5] x[4,9,5]\n", 144 | " x[5,1,5] x[5,2,5] x[5,3,5] x[5,4,5] x[5,7,5] x[5,8,5] x[5,9,5]\n", 145 | " x[6,1,5] x[6,2,5] x[6,3,5] x[6,4,5] … x[6,7,5] x[6,8,5] x[6,9,5]\n", 146 | " x[7,1,5] x[7,2,5] x[7,3,5] x[7,4,5] x[7,7,5] x[7,8,5] x[7,9,5]\n", 147 | " x[8,1,5] x[8,2,5] x[8,3,5] x[8,4,5] x[8,7,5] x[8,8,5] x[8,9,5]\n", 148 | " x[9,1,5] x[9,2,5] x[9,3,5] x[9,4,5] x[9,7,5] x[9,8,5] x[9,9,5]\n", 149 | "\n", 150 | "[:, :, 6] =\n", 151 | " x[1,1,6] x[1,2,6] x[1,3,6] x[1,4,6] … x[1,7,6] x[1,8,6] x[1,9,6]\n", 152 | " x[2,1,6] x[2,2,6] x[2,3,6] x[2,4,6] x[2,7,6] x[2,8,6] x[2,9,6]\n", 153 | " x[3,1,6] x[3,2,6] x[3,3,6] x[3,4,6] x[3,7,6] x[3,8,6] x[3,9,6]\n", 154 | " x[4,1,6] x[4,2,6] x[4,3,6] x[4,4,6] x[4,7,6] x[4,8,6] x[4,9,6]\n", 155 | " x[5,1,6] x[5,2,6] x[5,3,6] x[5,4,6] x[5,7,6] x[5,8,6] x[5,9,6]\n", 156 | " x[6,1,6] x[6,2,6] x[6,3,6] x[6,4,6] … x[6,7,6] x[6,8,6] x[6,9,6]\n", 157 | " x[7,1,6] x[7,2,6] x[7,3,6] x[7,4,6] x[7,7,6] x[7,8,6] x[7,9,6]\n", 158 | " x[8,1,6] x[8,2,6] x[8,3,6] x[8,4,6] x[8,7,6] x[8,8,6] x[8,9,6]\n", 159 | " x[9,1,6] x[9,2,6] x[9,3,6] x[9,4,6] x[9,7,6] x[9,8,6] x[9,9,6]\n", 160 | "\n", 161 | "[:, :, 7] =\n", 162 | " x[1,1,7] x[1,2,7] x[1,3,7] x[1,4,7] … x[1,7,7] x[1,8,7] x[1,9,7]\n", 163 | " x[2,1,7] x[2,2,7] x[2,3,7] x[2,4,7] x[2,7,7] x[2,8,7] x[2,9,7]\n", 164 | " x[3,1,7] x[3,2,7] x[3,3,7] x[3,4,7] x[3,7,7] x[3,8,7] x[3,9,7]\n", 165 | " x[4,1,7] x[4,2,7] x[4,3,7] x[4,4,7] x[4,7,7] x[4,8,7] x[4,9,7]\n", 166 | " x[5,1,7] x[5,2,7] x[5,3,7] x[5,4,7] x[5,7,7] x[5,8,7] x[5,9,7]\n", 167 | " x[6,1,7] x[6,2,7] x[6,3,7] x[6,4,7] … x[6,7,7] x[6,8,7] x[6,9,7]\n", 168 | " x[7,1,7] x[7,2,7] x[7,3,7] x[7,4,7] x[7,7,7] x[7,8,7] x[7,9,7]\n", 169 | " x[8,1,7] x[8,2,7] x[8,3,7] x[8,4,7] x[8,7,7] x[8,8,7] x[8,9,7]\n", 170 | " x[9,1,7] x[9,2,7] x[9,3,7] x[9,4,7] x[9,7,7] x[9,8,7] x[9,9,7]\n", 171 | "\n", 172 | "[:, :, 8] =\n", 173 | " x[1,1,8] x[1,2,8] x[1,3,8] x[1,4,8] … x[1,7,8] x[1,8,8] x[1,9,8]\n", 174 | " x[2,1,8] x[2,2,8] x[2,3,8] x[2,4,8] x[2,7,8] x[2,8,8] x[2,9,8]\n", 175 | " x[3,1,8] x[3,2,8] x[3,3,8] x[3,4,8] x[3,7,8] x[3,8,8] x[3,9,8]\n", 176 | " x[4,1,8] x[4,2,8] x[4,3,8] x[4,4,8] x[4,7,8] x[4,8,8] x[4,9,8]\n", 177 | " x[5,1,8] x[5,2,8] x[5,3,8] x[5,4,8] x[5,7,8] x[5,8,8] x[5,9,8]\n", 178 | " x[6,1,8] x[6,2,8] x[6,3,8] x[6,4,8] … x[6,7,8] x[6,8,8] x[6,9,8]\n", 179 | " x[7,1,8] x[7,2,8] x[7,3,8] x[7,4,8] x[7,7,8] x[7,8,8] x[7,9,8]\n", 180 | " x[8,1,8] x[8,2,8] x[8,3,8] x[8,4,8] x[8,7,8] x[8,8,8] x[8,9,8]\n", 181 | " x[9,1,8] x[9,2,8] x[9,3,8] x[9,4,8] x[9,7,8] x[9,8,8] x[9,9,8]\n", 182 | "\n", 183 | "[:, :, 9] =\n", 184 | " x[1,1,9] x[1,2,9] x[1,3,9] x[1,4,9] … x[1,7,9] x[1,8,9] x[1,9,9]\n", 185 | " x[2,1,9] x[2,2,9] x[2,3,9] x[2,4,9] x[2,7,9] x[2,8,9] x[2,9,9]\n", 186 | " x[3,1,9] x[3,2,9] x[3,3,9] x[3,4,9] x[3,7,9] x[3,8,9] x[3,9,9]\n", 187 | " x[4,1,9] x[4,2,9] x[4,3,9] x[4,4,9] x[4,7,9] x[4,8,9] x[4,9,9]\n", 188 | " x[5,1,9] x[5,2,9] x[5,3,9] x[5,4,9] x[5,7,9] x[5,8,9] x[5,9,9]\n", 189 | " x[6,1,9] x[6,2,9] x[6,3,9] x[6,4,9] … x[6,7,9] x[6,8,9] x[6,9,9]\n", 190 | " x[7,1,9] x[7,2,9] x[7,3,9] x[7,4,9] x[7,7,9] x[7,8,9] x[7,9,9]\n", 191 | " x[8,1,9] x[8,2,9] x[8,3,9] x[8,4,9] x[8,7,9] x[8,8,9] x[8,9,9]\n", 192 | " x[9,1,9] x[9,2,9] x[9,3,9] x[9,4,9] x[9,7,9] x[9,8,9] x[9,9,9]" 193 | ] 194 | }, 195 | "execution_count": 2, 196 | "metadata": {}, 197 | "output_type": "execute_result" 198 | } 199 | ], 200 | "source": [ 201 | "# Create a model\n", 202 | "sudoku = Model(with_optimizer(GLPK.Optimizer))\n", 203 | "\n", 204 | "# Create our variables\n", 205 | "@variable(sudoku, x[i=1:9, j=1:9, k=1:9], Bin)" 206 | ] 207 | }, 208 | { 209 | "cell_type": "markdown", 210 | "metadata": {}, 211 | "source": [ 212 | "Now we can begin to add our constraints. We'll actually start with something obvious to us as humans, but what we need to enforce: that there can be only one number per cell." 213 | ] 214 | }, 215 | { 216 | "cell_type": "code", 217 | "execution_count": 3, 218 | "metadata": {}, 219 | "outputs": [], 220 | "source": [ 221 | "for i = 1:9, j = 1:9 # Each row and each column\n", 222 | " # Sum across all the possible digits\n", 223 | " # One and only one of the digits can be in this cell, \n", 224 | " # so the sum must be equal to one\n", 225 | " @constraint(sudoku, sum(x[i,j,k] for k in 1:9) == 1)\n", 226 | "end" 227 | ] 228 | }, 229 | { 230 | "cell_type": "markdown", 231 | "metadata": {}, 232 | "source": [ 233 | "Next we'll add the constraints for the rows and the columns. These constraints are all very similar, so much so that we can actually add them at the same time." 234 | ] 235 | }, 236 | { 237 | "cell_type": "code", 238 | "execution_count": 4, 239 | "metadata": {}, 240 | "outputs": [], 241 | "source": [ 242 | "for ind = 1:9 # Each row, OR each column\n", 243 | " for k = 1:9 # Each digit\n", 244 | " # Sum across columns (j) - row constraint\n", 245 | " @constraint(sudoku, sum(x[ind,j,k] for j in 1:9) == 1)\n", 246 | " # Sum across rows (i) - column constraint\n", 247 | " @constraint(sudoku, sum(x[i,ind,k] for i in 1:9) == 1)\n", 248 | " end\n", 249 | "end" 250 | ] 251 | }, 252 | { 253 | "cell_type": "markdown", 254 | "metadata": {}, 255 | "source": [ 256 | "Finally, we have the to enforce the constraint that each digit appears once in each of the nine 3x3 sub-grids. Our strategy will be to index over the top-left corners of each 3x3 square with `for` loops, then sum over the squares." 257 | ] 258 | }, 259 | { 260 | "cell_type": "code", 261 | "execution_count": 5, 262 | "metadata": {}, 263 | "outputs": [], 264 | "source": [ 265 | "for i = 1:3:7, j = 1:3:7, k = 1:9\n", 266 | " # i is the top left row, j is the top left column\n", 267 | " # We'll sum from i to i+2, e.g. i=4, r=4, 5, 6\n", 268 | " @constraint(sudoku, sum(x[r,c,k] for r in i:i+2, c in j:j+2) == 1)\n", 269 | "end" 270 | ] 271 | }, 272 | { 273 | "cell_type": "markdown", 274 | "metadata": {}, 275 | "source": [ 276 | "The final step is to add the initial solution as a set of constraints. We'll solve the problem that is in the picture at the start of the notebook. We'll put a `0` if there is no digit in that location." 277 | ] 278 | }, 279 | { 280 | "cell_type": "code", 281 | "execution_count": 6, 282 | "metadata": {}, 283 | "outputs": [], 284 | "source": [ 285 | "# The given digits\n", 286 | "init_sol = [ 5 3 0 0 7 0 0 0 0;\n", 287 | " 6 0 0 1 9 5 0 0 0;\n", 288 | " 0 9 8 0 0 0 0 6 0;\n", 289 | " 8 0 0 0 6 0 0 0 3;\n", 290 | " 4 0 0 8 0 3 0 0 1;\n", 291 | " 7 0 0 0 2 0 0 0 6;\n", 292 | " 0 6 0 0 0 0 2 8 0;\n", 293 | " 0 0 0 4 1 9 0 0 5;\n", 294 | " 0 0 0 0 8 0 0 7 9]\n", 295 | "for i = 1:9, j = 1:9\n", 296 | " # If the space isn't empty\n", 297 | " if init_sol[i,j] != 0\n", 298 | " # Then the corresponding variable for that digit\n", 299 | " # and location must be 1\n", 300 | " @constraint(sudoku, x[i,j,init_sol[i,j]] == 1)\n", 301 | " end\n", 302 | "end" 303 | ] 304 | }, 305 | { 306 | "cell_type": "code", 307 | "execution_count": 7, 308 | "metadata": {}, 309 | "outputs": [], 310 | "source": [ 311 | "# solve problem\n", 312 | "optimize!(sudoku)" 313 | ] 314 | }, 315 | { 316 | "cell_type": "code", 317 | "execution_count": 8, 318 | "metadata": {}, 319 | "outputs": [ 320 | { 321 | "name": "stdout", 322 | "output_type": "stream", 323 | "text": [ 324 | "has_values(sudoku) = true\n", 325 | "termination_status(sudoku) == MOI.OPTIMAL = true\n", 326 | "primal_status(sudoku) == MOI.FEASIBLE_POINT = true\n" 327 | ] 328 | }, 329 | { 330 | "data": { 331 | "text/plain": [ 332 | "true" 333 | ] 334 | }, 335 | "execution_count": 8, 336 | "metadata": {}, 337 | "output_type": "execute_result" 338 | } 339 | ], 340 | "source": [ 341 | "# test if optimization worked out properly\n", 342 | "@show has_values(sudoku)\n", 343 | "@show termination_status(sudoku) == MOI.OPTIMAL\n", 344 | "@show primal_status(sudoku) == MOI.FEASIBLE_POINT" 345 | ] 346 | }, 347 | { 348 | "cell_type": "markdown", 349 | "metadata": {}, 350 | "source": [ 351 | "To display the solution, we need to look for the values of `x[i,j,k]` that are 1." 352 | ] 353 | }, 354 | { 355 | "cell_type": "code", 356 | "execution_count": 9, 357 | "metadata": {}, 358 | "outputs": [ 359 | { 360 | "data": { 361 | "text/plain": [ 362 | "9×9 Array{Int64,2}:\n", 363 | " 5 3 4 6 7 8 9 1 2\n", 364 | " 6 7 2 1 9 5 3 4 8\n", 365 | " 1 9 8 3 4 2 5 6 7\n", 366 | " 8 5 9 7 6 1 4 2 3\n", 367 | " 4 2 6 8 5 3 7 9 1\n", 368 | " 7 1 3 9 2 4 8 5 6\n", 369 | " 9 6 1 5 3 7 2 8 4\n", 370 | " 2 8 7 4 1 9 6 3 5\n", 371 | " 3 4 5 2 8 6 1 7 9" 372 | ] 373 | }, 374 | "execution_count": 9, 375 | "metadata": {}, 376 | "output_type": "execute_result" 377 | } 378 | ], 379 | "source": [ 380 | "# Extract the values of x\n", 381 | "x_val = value.(x)\n", 382 | "# Create a matrix to store the solution\n", 383 | "sol = zeros(Int,9,9) # 9x9 matrix of integers\n", 384 | "for i in 1:9, j in 1:9, k in 1:9\n", 385 | " # Integer programs are solved as a series of linear programs\n", 386 | " # so the values might not be precisely 0 and 1. We can just\n", 387 | " # round them to the nearest integer to make it easier\n", 388 | " if round(Int,x_val[i,j,k]) == 1\n", 389 | " sol[i,j] = k\n", 390 | " end\n", 391 | "end\n", 392 | "# Display the solution\n", 393 | "sol" 394 | ] 395 | }, 396 | { 397 | "cell_type": "markdown", 398 | "metadata": {}, 399 | "source": [ 400 | "Which is the correct solution:\n", 401 | "\n", 402 | "

A completed Sudoku puzzle

" 403 | ] 404 | }, 405 | { 406 | "cell_type": "code", 407 | "execution_count": null, 408 | "metadata": { 409 | "collapsed": true 410 | }, 411 | "outputs": [], 412 | "source": [] 413 | } 414 | ], 415 | "metadata": { 416 | "kernelspec": { 417 | "display_name": "Julia 1.0.3", 418 | "language": "julia", 419 | "name": "julia-1.0" 420 | }, 421 | "language_info": { 422 | "file_extension": ".jl", 423 | "mimetype": "application/julia", 424 | "name": "julia", 425 | "version": "1.0.3" 426 | } 427 | }, 428 | "nbformat": 4, 429 | "nbformat_minor": 1 430 | } 431 | -------------------------------------------------------------------------------- /notebooks/DualNumbers.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "**Description**: Introduction to automatic differentiation and the ``DualNumbers`` package. Prepared for MIT course 15.S60 \"Software Tools for Operations Research\", January 2015.\n", 8 | "\n", 9 | "**Author**: Miles Lubin\n", 10 | "\n", 11 | "**License**: \"Creative
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License." 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "metadata": {}, 17 | "source": [ 18 | "# Computing derivatives for nonlinear optimization: Forward mode automatic differentiation" 19 | ] 20 | }, 21 | { 22 | "cell_type": "markdown", 23 | "metadata": {}, 24 | "source": [ 25 | "Consider a general constrained nonlinear optimization problem:\n", 26 | "$$\n", 27 | "\\begin{align}\n", 28 | "\\min \\quad&f(x)\\\\\n", 29 | "\\text{s.t.} \\quad& g(x) = 0, \\\\\n", 30 | "& h(x) \\leq 0.\n", 31 | "\\end{align}\n", 32 | "$$\n", 33 | "where $f : \\mathbb{R}^n \\to \\mathbb{R}, g : \\mathbb{R}^n \\to \\mathbb{R}^r$, and $h: \\mathbb{R}^n \\to \\mathbb{R}^s$.\n", 34 | "\n", 35 | "When $f$ and $h$ are convex and $g$ is affine, we can hope for a globally optimal solution, otherwise typically we can only ask for a locally optimal solution.\n", 36 | "\n", 37 | "What approaches can we use to solve this?\n", 38 | " - When $r=0$ and $s = 0$ (unconstrained), and $f$ differentiable, most classical approach is [gradient descent](http://en.wikipedia.org/wiki/Gradient_descent), also fancier methods like [Newton's method](http://en.wikipedia.org/wiki/Newton%27s_method) and quasi-newton methods like [BFGS](http://en.wikipedia.org/wiki/Broyden%E2%80%93Fletcher%E2%80%93Goldfarb%E2%80%93Shanno_algorithm).\n", 39 | " - When $f$ differentiable and $g$ and $h$ linear, [gradient projection](http://neos-guide.org/content/gradient-projection-methods)\n", 40 | " - When $f$, $g$, and $h$ twice differentiable, [interior-point methods](http://en.wikipedia.org/wiki/Interior_point_method), [sequential quadratic programming](http://www.neos-guide.org/content/sequential-quadratic-programming)\n", 41 | " - When derivatives \"not available\", [derivative-free optimization](http://rd.springer.com/article/10.1007/s10898-012-9951-y)\n", 42 | " \n", 43 | "This is not meant to be an exhaustive list, see http://plato.asu.edu/sub/nlores.html#general and http://www.neos-guide.org/content/nonlinear-programming for more details." 44 | ] 45 | }, 46 | { 47 | "cell_type": "markdown", 48 | "metadata": {}, 49 | "source": [ 50 | "## How are derivatives computed?\n", 51 | "\n", 52 | "- Hand-written by applying chain rule\n", 53 | "- Finite difference approximation $\\frac{\\partial f}{\\partial x_i} = \\lim_{h\\to 0} \\frac{f(x+h e_i)-f(x)}{h}$\n", 54 | "- **Automatic differentiation**\n", 55 | " - Idea: Automatically transform code to compute a function into code to compute its derivatives" 56 | ] 57 | }, 58 | { 59 | "cell_type": "markdown", 60 | "metadata": {}, 61 | "source": [ 62 | "## Dual Numbers\n", 63 | "\n", 64 | "Consider numbers of the form $x + y\\epsilon$ with $x,y \\in \\mathbb{R}$. We *define* $\\epsilon^2 = 0$, so\n", 65 | "$$\n", 66 | "(x_1 + y_1\\epsilon)(x_2+y_2\\epsilon) = x_1x_2 + (x_1y_2 + x_2y_1)\\epsilon.\n", 67 | "$$\n", 68 | "These are called the *dual numbers*. Think of $\\epsilon$ as an infinitesimal perturbation (you've probably seen hand-wavy algebra using $(dx)^2 = 0$ when computing integrals - this is the same idea).\n", 69 | "\n", 70 | "If we are given an infinitely differentiable function in Taylor expanded form\n", 71 | "$$\n", 72 | "f(x) = \\sum_{k=0}^{\\infty} \\frac{f^{(k)}(a)}{k!} (x-a)^k\n", 73 | "$$\n", 74 | "it follows that \n", 75 | "$$\n", 76 | "f(x+y\\epsilon) = \\sum_{k=0}^{\\infty} \\frac{f^{(k)}(a)}{k!} (x-a+y\\epsilon)^k = \\sum_{k=0}^{\\infty} \\frac{f^{(k)}(a)}{k!} (x-a)^k + y\\epsilon\\sum_{k=0}^{\\infty} \\frac{f^{(k)}(a)}{k!}\\binom{k}{1} (x-a)^{k-1} = f(x) + yf'(x)\\epsilon\n", 77 | "$$\n", 78 | "\n", 79 | "Let's unpack what's going on here. We started with a function $f : \\mathbb{R} \\to \\mathbb{R}$. Dual numbers are *not* real numbers, so it doesn't even make sense to ask for the value $f(x+y\\epsilon)$ given $x+y\\epsilon \\in \\mathbb{D}$ (the set of dual numbers). But anyway we plugged the dual number into the Taylor expansion, and by using the algebra rule $\\epsilon^2 = 0$ we found that $f(x+y\\epsilon)$ must be equal to $f(x) + yf'(x)\\epsilon$ if we use the Taylor expansion as the definition of $f : \\mathbb{D} \\to \\mathbb{R}$." 80 | ] 81 | }, 82 | { 83 | "cell_type": "markdown", 84 | "metadata": {}, 85 | "source": [ 86 | "Alternatively, for any once differentiable function $f : \\mathbb{R} \\to \\mathbb{R}$, we can *define* its extension to the dual numbers as\n", 87 | "$$\n", 88 | "f(x+y\\epsilon) = f(x) + yf'(x)\\epsilon.\n", 89 | "$$\n", 90 | "This is essentially equivalent to the previous definition.\n", 91 | "\n", 92 | "Let's verify a very basic property, the chain rule, using this definition.\n", 93 | "\n", 94 | "Suppose $h(x) = f(g(x))$. Then,\n", 95 | "$$\n", 96 | "h(x+y\\epsilon) = f(g(x+y\\epsilon)) = f(g(x) + yg'(x)\\epsilon) = f(g(x)) + yg'(x)f'(g(x))\\epsilon = h(x) + yh'(x)\\epsilon.\n", 97 | "$$\n", 98 | "\n", 99 | "Maybe that's not too surprising, but it's actually a quite useful observation." 100 | ] 101 | }, 102 | { 103 | "cell_type": "markdown", 104 | "metadata": {}, 105 | "source": [ 106 | "### Implementation\n", 107 | "\n", 108 | "Dual numbers are implemented in the [DualNumbers](https://github.com/JuliaDiff/DualNumbers.jl) package in Julia." 109 | ] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "execution_count": 1, 114 | "metadata": { 115 | "collapsed": false 116 | }, 117 | "outputs": [], 118 | "source": [ 119 | "using DualNumbers" 120 | ] 121 | }, 122 | { 123 | "cell_type": "markdown", 124 | "metadata": {}, 125 | "source": [ 126 | "You construct $x + y\\epsilon$ with ``Dual(x,y)``. The real and epsilon components are accessed as ``real(d)`` and ``epsilon(d)``:" 127 | ] 128 | }, 129 | { 130 | "cell_type": "code", 131 | "execution_count": 2, 132 | "metadata": { 133 | "collapsed": false 134 | }, 135 | "outputs": [ 136 | { 137 | "data": { 138 | "text/plain": [ 139 | "2.0 + 1.0ɛ" 140 | ] 141 | }, 142 | "execution_count": 2, 143 | "metadata": {}, 144 | "output_type": "execute_result" 145 | } 146 | ], 147 | "source": [ 148 | "d = Dual(2.0,1.0)" 149 | ] 150 | }, 151 | { 152 | "cell_type": "code", 153 | "execution_count": 3, 154 | "metadata": { 155 | "collapsed": false 156 | }, 157 | "outputs": [ 158 | { 159 | "data": { 160 | "text/plain": [ 161 | "DualNumbers.Dual{Float64}" 162 | ] 163 | }, 164 | "execution_count": 3, 165 | "metadata": {}, 166 | "output_type": "execute_result" 167 | } 168 | ], 169 | "source": [ 170 | "typeof(d)" 171 | ] 172 | }, 173 | { 174 | "cell_type": "code", 175 | "execution_count": 4, 176 | "metadata": { 177 | "collapsed": false 178 | }, 179 | "outputs": [ 180 | { 181 | "data": { 182 | "text/plain": [ 183 | "2.0" 184 | ] 185 | }, 186 | "execution_count": 4, 187 | "metadata": {}, 188 | "output_type": "execute_result" 189 | } 190 | ], 191 | "source": [ 192 | "realpart(d)" 193 | ] 194 | }, 195 | { 196 | "cell_type": "code", 197 | "execution_count": 5, 198 | "metadata": { 199 | "collapsed": false 200 | }, 201 | "outputs": [ 202 | { 203 | "data": { 204 | "text/plain": [ 205 | "1.0" 206 | ] 207 | }, 208 | "execution_count": 5, 209 | "metadata": {}, 210 | "output_type": "execute_result" 211 | } 212 | ], 213 | "source": [ 214 | "epsilon(d)" 215 | ] 216 | }, 217 | { 218 | "cell_type": "markdown", 219 | "metadata": {}, 220 | "source": [ 221 | "How is addition of dual numbers defined?" 222 | ] 223 | }, 224 | { 225 | "cell_type": "code", 226 | "execution_count": null, 227 | "metadata": { 228 | "collapsed": false 229 | }, 230 | "outputs": [], 231 | "source": [ 232 | "@which d+Dual(3.0,4.0)" 233 | ] 234 | }, 235 | { 236 | "cell_type": "markdown", 237 | "metadata": {}, 238 | "source": [ 239 | "Clicking on the link, we'll see:\n", 240 | "```julia\n", 241 | "+(z::Dual, w::Dual) = dual(real(z)+real(w), epsilon(z)+epsilon(w))\n", 242 | "```" 243 | ] 244 | }, 245 | { 246 | "cell_type": "markdown", 247 | "metadata": {}, 248 | "source": [ 249 | "Multiplication?" 250 | ] 251 | }, 252 | { 253 | "cell_type": "code", 254 | "execution_count": 6, 255 | "metadata": { 256 | "collapsed": false 257 | }, 258 | "outputs": [ 259 | { 260 | "data": { 261 | "text/plain": [ 262 | "6.0 + 14.0ɛ" 263 | ] 264 | }, 265 | "execution_count": 6, 266 | "metadata": {}, 267 | "output_type": "execute_result" 268 | } 269 | ], 270 | "source": [ 271 | "Dual(2.0,2.0)*Dual(3.0,4.0)" 272 | ] 273 | }, 274 | { 275 | "cell_type": "code", 276 | "execution_count": null, 277 | "metadata": { 278 | "collapsed": false 279 | }, 280 | "outputs": [], 281 | "source": [ 282 | "@which Dual(2.0,2.0)*Dual(3.0,4.0)" 283 | ] 284 | }, 285 | { 286 | "cell_type": "markdown", 287 | "metadata": {}, 288 | "source": [ 289 | "The code is:\n", 290 | "```julia\n", 291 | "*(z::Dual, w::Dual) = dual(real(z)*real(w), epsilon(z)*real(w)+real(z)*epsilon(w))\n", 292 | "```" 293 | ] 294 | }, 295 | { 296 | "cell_type": "markdown", 297 | "metadata": {}, 298 | "source": [ 299 | "Basic univariate functions?" 300 | ] 301 | }, 302 | { 303 | "cell_type": "code", 304 | "execution_count": 7, 305 | "metadata": { 306 | "collapsed": false 307 | }, 308 | "outputs": [ 309 | { 310 | "data": { 311 | "text/plain": [ 312 | "0.6931471805599453 + 0.5ɛ" 313 | ] 314 | }, 315 | "execution_count": 7, 316 | "metadata": {}, 317 | "output_type": "execute_result" 318 | } 319 | ], 320 | "source": [ 321 | "log(Dual(2.0,1.0))" 322 | ] 323 | }, 324 | { 325 | "cell_type": "code", 326 | "execution_count": 8, 327 | "metadata": { 328 | "collapsed": false 329 | }, 330 | "outputs": [ 331 | { 332 | "data": { 333 | "text/plain": [ 334 | "0.5" 335 | ] 336 | }, 337 | "execution_count": 8, 338 | "metadata": {}, 339 | "output_type": "execute_result" 340 | } 341 | ], 342 | "source": [ 343 | "1/2.0" 344 | ] 345 | }, 346 | { 347 | "cell_type": "markdown", 348 | "metadata": {}, 349 | "source": [ 350 | "How is this implemented?" 351 | ] 352 | }, 353 | { 354 | "cell_type": "code", 355 | "execution_count": 9, 356 | "metadata": { 357 | "collapsed": false 358 | }, 359 | "outputs": [ 360 | { 361 | "data": { 362 | "text/plain": [ 363 | "LambdaInfo template for log(z::DualNumbers.Dual) at /home/mlubin/.julia/v0.5/DualNumbers/src/dual.jl:273\n", 364 | ":(begin \n", 365 | " nothing\n", 366 | " x = (DualNumbers.value)(z) # line 274:\n", 367 | " xp = (DualNumbers.epsilon)(z) # line 275:\n", 368 | " return (DualNumbers.Dual)((DualNumbers.log)(x),xp * (1 / x))\n", 369 | " end)" 370 | ] 371 | }, 372 | "execution_count": 9, 373 | "metadata": {}, 374 | "output_type": "execute_result" 375 | } 376 | ], 377 | "source": [ 378 | "@code_lowered log(Dual(2.0,1.0))" 379 | ] 380 | }, 381 | { 382 | "cell_type": "markdown", 383 | "metadata": {}, 384 | "source": [ 385 | "Trig functions?" 386 | ] 387 | }, 388 | { 389 | "cell_type": "code", 390 | "execution_count": 10, 391 | "metadata": { 392 | "collapsed": false 393 | }, 394 | "outputs": [ 395 | { 396 | "data": { 397 | "text/plain": [ 398 | "LambdaInfo template for sin(z::DualNumbers.Dual) at /home/mlubin/.julia/v0.5/DualNumbers/src/dual.jl:273\n", 399 | ":(begin \n", 400 | " nothing\n", 401 | " x = (DualNumbers.value)(z) # line 274:\n", 402 | " xp = (DualNumbers.epsilon)(z) # line 275:\n", 403 | " return (DualNumbers.Dual)((DualNumbers.sin)(x),xp * (DualNumbers.cos)(x))\n", 404 | " end)" 405 | ] 406 | }, 407 | "execution_count": 10, 408 | "metadata": {}, 409 | "output_type": "execute_result" 410 | } 411 | ], 412 | "source": [ 413 | "@code_lowered sin(Dual(2.0,1.0))" 414 | ] 415 | }, 416 | { 417 | "cell_type": "markdown", 418 | "metadata": {}, 419 | "source": [ 420 | "## Computing derivatives of functions" 421 | ] 422 | }, 423 | { 424 | "cell_type": "markdown", 425 | "metadata": {}, 426 | "source": [ 427 | "We can define a function in Julia as:" 428 | ] 429 | }, 430 | { 431 | "cell_type": "code", 432 | "execution_count": 11, 433 | "metadata": { 434 | "collapsed": false 435 | }, 436 | "outputs": [ 437 | { 438 | "name": "stderr", 439 | "output_type": "stream", 440 | "text": [ 441 | "WARNING: Method definition f(Any) in module Main at In[11]:1 overwritten at In[11]:4.\n" 442 | ] 443 | }, 444 | { 445 | "data": { 446 | "text/plain": [ 447 | "f (generic function with 1 method)" 448 | ] 449 | }, 450 | "execution_count": 11, 451 | "metadata": {}, 452 | "output_type": "execute_result" 453 | } 454 | ], 455 | "source": [ 456 | "f(x) = x^2 - log(x)\n", 457 | "# Or equivalently\n", 458 | "function f(x)\n", 459 | " return x^2 - log(x)\n", 460 | "end" 461 | ] 462 | }, 463 | { 464 | "cell_type": "code", 465 | "execution_count": 12, 466 | "metadata": { 467 | "collapsed": false 468 | }, 469 | "outputs": [ 470 | { 471 | "data": { 472 | "text/plain": [ 473 | "0.8465735902799727 - 2.220446049250313e-16ɛ" 474 | ] 475 | }, 476 | "execution_count": 12, 477 | "metadata": {}, 478 | "output_type": "execute_result" 479 | } 480 | ], 481 | "source": [ 482 | "f(Dual(1/sqrt(2),1))" 483 | ] 484 | }, 485 | { 486 | "cell_type": "markdown", 487 | "metadata": {}, 488 | "source": [ 489 | ">**\\[Exercise\\]**: Differentiate it!\n", 490 | "\n", 491 | "> 1. Evaluate $f$ at $1 + \\epsilon$. What are $f(1)$ and $f'(1)$?\n", 492 | "> 2. Evaluate $f$ at $\\frac{1}{\\sqrt{2}} + \\epsilon$. What are $f(\\frac{1}{\\sqrt{2}})$ and $f'(\\frac{1}{\\sqrt{2}})$?\n", 493 | "> 3. Define a new function ``fprime`` which returns the derivative of ``f`` by using ``DualNumbers``.\n", 494 | "> 3. Use the finite difference formula $$\n", 495 | "f'(x) \\approx \\frac{f(x+h)-f(x)}{h}\n", 496 | "$$\n", 497 | "to evaluate $f'(\\frac{1}{\\sqrt{2}})$ approximately using a range of values of $h$. Visualize the approximation error using ``@manipulate``, plots, or both!" 498 | ] 499 | }, 500 | { 501 | "cell_type": "markdown", 502 | "metadata": {}, 503 | "source": [ 504 | "### How general is it?\n", 505 | "\n", 506 | "Recall [Newton's iterative method](http://en.wikipedia.org/wiki/Newton%27s_method) for finding zeros:\n", 507 | "$$\n", 508 | "x \\leftarrow x - \\frac{f(x)}{f'(x)}\n", 509 | "$$\n", 510 | "until $f(x) \\approx 0$." 511 | ] 512 | }, 513 | { 514 | "cell_type": "markdown", 515 | "metadata": {}, 516 | "source": [ 517 | "Let's use this method to compute $\\sqrt{x}$ by solving $f(z) = 0$ where $f(z) = z^2-x$.\n", 518 | "So $f'(z) = 2z$, and we can implement the algorithm as:" 519 | ] 520 | }, 521 | { 522 | "cell_type": "code", 523 | "execution_count": 13, 524 | "metadata": { 525 | "collapsed": false 526 | }, 527 | "outputs": [ 528 | { 529 | "data": { 530 | "text/plain": [ 531 | "squareroot (generic function with 1 method)" 532 | ] 533 | }, 534 | "execution_count": 13, 535 | "metadata": {}, 536 | "output_type": "execute_result" 537 | } 538 | ], 539 | "source": [ 540 | "function squareroot(x)\n", 541 | " z = x # Initial starting point\n", 542 | " while abs(z*z - x) > 1e-13\n", 543 | " z = z - (z*z-x)/(2z)\n", 544 | " end\n", 545 | " return z\n", 546 | "end" 547 | ] 548 | }, 549 | { 550 | "cell_type": "code", 551 | "execution_count": 14, 552 | "metadata": { 553 | "collapsed": false 554 | }, 555 | "outputs": [ 556 | { 557 | "data": { 558 | "text/plain": [ 559 | "10.0" 560 | ] 561 | }, 562 | "execution_count": 14, 563 | "metadata": {}, 564 | "output_type": "execute_result" 565 | } 566 | ], 567 | "source": [ 568 | "squareroot(100)" 569 | ] 570 | }, 571 | { 572 | "cell_type": "markdown", 573 | "metadata": {}, 574 | "source": [ 575 | "Can we differentiate this code? **Yes!**" 576 | ] 577 | }, 578 | { 579 | "cell_type": "code", 580 | "execution_count": 15, 581 | "metadata": { 582 | "collapsed": false 583 | }, 584 | "outputs": [ 585 | { 586 | "data": { 587 | "text/plain": [ 588 | "10.0 + 0.049999999999999996ɛ" 589 | ] 590 | }, 591 | "execution_count": 15, 592 | "metadata": {}, 593 | "output_type": "execute_result" 594 | } 595 | ], 596 | "source": [ 597 | "d = squareroot(Dual(100.0,1.0))" 598 | ] 599 | }, 600 | { 601 | "cell_type": "code", 602 | "execution_count": 16, 603 | "metadata": { 604 | "collapsed": false 605 | }, 606 | "outputs": [ 607 | { 608 | "data": { 609 | "text/plain": [ 610 | "0.049999999999999996" 611 | ] 612 | }, 613 | "execution_count": 16, 614 | "metadata": {}, 615 | "output_type": "execute_result" 616 | } 617 | ], 618 | "source": [ 619 | "epsilon(d) # Computed derivative" 620 | ] 621 | }, 622 | { 623 | "cell_type": "code", 624 | "execution_count": 17, 625 | "metadata": { 626 | "collapsed": false 627 | }, 628 | "outputs": [ 629 | { 630 | "data": { 631 | "text/plain": [ 632 | "0.05" 633 | ] 634 | }, 635 | "execution_count": 17, 636 | "metadata": {}, 637 | "output_type": "execute_result" 638 | } 639 | ], 640 | "source": [ 641 | "1/(2*sqrt(100)) # The exact derivative" 642 | ] 643 | }, 644 | { 645 | "cell_type": "code", 646 | "execution_count": 18, 647 | "metadata": { 648 | "collapsed": false 649 | }, 650 | "outputs": [ 651 | { 652 | "data": { 653 | "text/plain": [ 654 | "6.938893903907228e-18" 655 | ] 656 | }, 657 | "execution_count": 18, 658 | "metadata": {}, 659 | "output_type": "execute_result" 660 | } 661 | ], 662 | "source": [ 663 | "abs(epsilon(d)-1/(2*sqrt(100)))" 664 | ] 665 | }, 666 | { 667 | "cell_type": "markdown", 668 | "metadata": {}, 669 | "source": [ 670 | "### Multivariate functions?\n", 671 | "\n", 672 | "Dual numbers can be used to compute the gradient of a function $f: \\mathbb{R}^n \\to \\mathbb{R}$. This requires $n$ evaluations of $f$ with dual number input, essentially computing the partial derivative in each of the $n$ dimensions. We won't get into the details, but this procedure is [implemented](https://github.com/JuliaOpt/Optim.jl/blob/583907676b5b99cdb2d4cba37f6026a3fe620a49/src/autodiff.jl) in [Optim](https://github.com/JuliaOpt/Optim.jl) with the ``autodiff=true`` keyword." 673 | ] 674 | }, 675 | { 676 | "cell_type": "code", 677 | "execution_count": null, 678 | "metadata": { 679 | "collapsed": false 680 | }, 681 | "outputs": [], 682 | "source": [ 683 | "using Optim\n", 684 | "rosenbrock(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2\n", 685 | "optimize(rosenbrock, [0.0, 0.0], method = :l_bfgs, autodiff = true)" 686 | ] 687 | }, 688 | { 689 | "cell_type": "markdown", 690 | "metadata": {}, 691 | "source": [ 692 | "When $n$ is large, there's an alternative procedure called [reverse-mode automatic differentiation](http://en.wikipedia.org/wiki/Automatic_differentiation#Reverse_accumulation) which requires only $O(1)$ evaluations of $f$ to compute its gradient. This is the method used internally by JuMP (implemented in [ReverseDiffSparse](https://github.com/mlubin/ReverseDiffSparse.jl))." 693 | ] 694 | }, 695 | { 696 | "cell_type": "markdown", 697 | "metadata": {}, 698 | "source": [ 699 | "## Conclusions\n", 700 | "\n", 701 | "- We can compute numerically exact derivatives of any differentiable function which is implemented by using a sequence of basic operations.\n", 702 | "- In Julia it's very easy to use dual numbers for this!\n", 703 | "- Reconsider when derivatives are \"not available.\"\n", 704 | "\n", 705 | "This was just an introduction to one technique from the area of automatic differentiation. For more references, see [autodiff.org](http://www.autodiff.org/?module=Introduction&submenu=Selected%20Books)." 706 | ] 707 | } 708 | ], 709 | "metadata": { 710 | "anaconda-cloud": {}, 711 | "kernelspec": { 712 | "display_name": "Julia 0.5.0", 713 | "language": "julia", 714 | "name": "julia-0.5" 715 | }, 716 | "keywords": "DualNumbers,AD,derivatives", 717 | "language_info": { 718 | "file_extension": ".jl", 719 | "mimetype": "application/julia", 720 | "name": "julia", 721 | "version": "0.5.0" 722 | } 723 | }, 724 | "nbformat": 4, 725 | "nbformat_minor": 0 726 | } 727 | -------------------------------------------------------------------------------- /notebooks/JuMP-NetRevMgmt.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Disclaimer\n", 8 | "This notebook is only working under the versions:\n", 9 | "\n", 10 | "- JuMP 0.19 (unreleased, but currently in master)\n", 11 | "\n", 12 | "- MathOptInterface 0.4.1\n", 13 | "\n", 14 | "- GLPK 0.6.0" 15 | ] 16 | }, 17 | { 18 | "cell_type": "markdown", 19 | "metadata": {}, 20 | "source": [ 21 | "**Description**: Shows how to solve a network revenue management problem using JuMP.\n", 22 | "\n", 23 | "**Author**: Iain Dunning\n", 24 | "\n", 25 | "**License**: \"Creative
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License." 26 | ] 27 | }, 28 | { 29 | "cell_type": "markdown", 30 | "metadata": {}, 31 | "source": [ 32 | "# Airline Network Revenue Management\n", 33 | "\n", 34 | "\n", 35 | "\n", 36 | "In the airline network revenue management problem we are trying to decide how many tickets for each origin-destination (O-D) pair to sell at each price level. The goal is to maximize revenue, and we cannot sell more tickets than there is demand for, or space on the planes for.\n", 37 | "\n", 38 | "## Three Flight Problem\n", 39 | "\n", 40 | "We'll start with a toy problem that has three origin-destination pairs, with two price classes for each pair. The three origin-destination pairs are BOS-MDW, MDW-SFO, or BOS-SFO via MDW. BOS stands for Boston, MDW is Chicago-Midway, and SFO is San Francisco. Each O-D pair has a \"regular\" and \"discount\" fare class. The data we will use is summarized as follows:\n", 41 | "\n", 42 | "```\n", 43 | "PLANE CAPACITY: 166\n", 44 | "\n", 45 | "BOS-MDW\n", 46 | " Regular Discount\n", 47 | "Price: 428 190\n", 48 | "Demand: 80 120\n", 49 | "\n", 50 | "BOS-SFO\n", 51 | " Regular Discount\n", 52 | "Price: 642 224\n", 53 | "Demand: 75 100\n", 54 | "\n", 55 | "MDW-SFO\n", 56 | " Regular Discount\n", 57 | "Price: 512 190\n", 58 | "Demand: 60 110\n", 59 | "```" 60 | ] 61 | }, 62 | { 63 | "cell_type": "code", 64 | "execution_count": 1, 65 | "metadata": { 66 | "collapsed": false 67 | }, 68 | "outputs": [ 69 | { 70 | "data": { 71 | "text/plain": [ 72 | "3-element Array{Any,1}:\n", 73 | " \"C:\\\\Users\\\\joaquimgarcia\\\\AppData\\\\Local\\\\Julia-0.6.0\\\\local\\\\share\\\\julia\\\\site\\\\v0.6\"\n", 74 | " \"C:\\\\Users\\\\joaquimgarcia\\\\AppData\\\\Local\\\\Julia-0.6.0\\\\share\\\\julia\\\\site\\\\v0.6\" \n", 75 | " \"D:\\\\MOI\" " 76 | ] 77 | }, 78 | "execution_count": 1, 79 | "metadata": {}, 80 | "output_type": "execute_result" 81 | } 82 | ], 83 | "source": [ 84 | "push!(Base.LOAD_PATH,\"D:\\\\MOI\")" 85 | ] 86 | }, 87 | { 88 | "cell_type": "code", 89 | "execution_count": 2, 90 | "metadata": { 91 | "collapsed": false 92 | }, 93 | "outputs": [ 94 | { 95 | "data": { 96 | "text/plain": [ 97 | "MathOptInterface.Utilities" 98 | ] 99 | }, 100 | "execution_count": 2, 101 | "metadata": {}, 102 | "output_type": "execute_result" 103 | } 104 | ], 105 | "source": [ 106 | "# Load JuMP\n", 107 | "using JuMP\n", 108 | "using MathOptInterface\n", 109 | "# Load solver package\n", 110 | "using GLPK\n", 111 | "# shortcuts\n", 112 | "const MOI = MathOptInterface\n", 113 | "const MOIU = MathOptInterface.Utilities" 114 | ] 115 | }, 116 | { 117 | "cell_type": "markdown", 118 | "metadata": {}, 119 | "source": [ 120 | "### Model\n", 121 | "Now we need to create a `Model`. A `Model` is an object that has associated variables and constraints. We can also pick and customize different solvers to use with the model. In this case we'll assume you a LP solver installed - JuMP will detect it and use it automatically." 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "execution_count": 3, 127 | "metadata": { 128 | "collapsed": false 129 | }, 130 | "outputs": [ 131 | { 132 | "data": { 133 | "text/plain": [ 134 | "A JuMP Model" 135 | ] 136 | }, 137 | "execution_count": 3, 138 | "metadata": {}, 139 | "output_type": "execute_result" 140 | } 141 | ], 142 | "source": [ 143 | "nrm = Model(optimizer = GLPK.GLPKOptimizerLP())" 144 | ] 145 | }, 146 | { 147 | "cell_type": "markdown", 148 | "metadata": {}, 149 | "source": [ 150 | "We can see that JuMP displays the model in a human-readable form.\n", 151 | "\n", 152 | "### Variables\n", 153 | "Now we can create our variables, in the optimization sense. Variables have a name, bounds, and type. For this problem, we need six continuous variables, each with a lower and upper bound on their value.\n", 154 | "\n", 155 | "Here we will spell out each variable one-by-one - rest assured that later we will do something smarter that will scale to millions of variables!" 156 | ] 157 | }, 158 | { 159 | "cell_type": "code", 160 | "execution_count": 4, 161 | "metadata": { 162 | "collapsed": false 163 | }, 164 | "outputs": [ 165 | { 166 | "data": { 167 | "text/plain": [ 168 | "A JuMP Model" 169 | ] 170 | }, 171 | "execution_count": 4, 172 | "metadata": {}, 173 | "output_type": "execute_result" 174 | } 175 | ], 176 | "source": [ 177 | "@variables(nrm, begin \n", 178 | " 0 <= BOStoMDW_R <= 80\n", 179 | " 0 <= BOStoMDW_D <= 120\n", 180 | " 0 <= BOStoSFO_R <= 75\n", 181 | " 0 <= BOStoSFO_D <= 100\n", 182 | " 0 <= MDWtoSFO_R <= 60\n", 183 | " 0 <= MDWtoSFO_D <= 110\n", 184 | "end)\n", 185 | "nrm" 186 | ] 187 | }, 188 | { 189 | "cell_type": "markdown", 190 | "metadata": {}, 191 | "source": [ 192 | "Great, now we are getting somewhere!\n", 193 | "\n", 194 | "### Objective\n", 195 | "The objective is to maximize the revenue. We set the objective with `@objective`, which takes three arguments: the model, the sense (`Max` or `Min`), and the expression." 196 | ] 197 | }, 198 | { 199 | "cell_type": "code", 200 | "execution_count": 5, 201 | "metadata": { 202 | "collapsed": false 203 | }, 204 | "outputs": [ 205 | { 206 | "data": { 207 | "text/plain": [ 208 | "A JuMP Model" 209 | ] 210 | }, 211 | "execution_count": 5, 212 | "metadata": {}, 213 | "output_type": "execute_result" 214 | } 215 | ], 216 | "source": [ 217 | "@objective(nrm, Max, 428*BOStoMDW_R + 190*BOStoMDW_D +\n", 218 | " 642*BOStoSFO_R + 224*BOStoSFO_D +\n", 219 | " 512*MDWtoSFO_R + 190*MDWtoSFO_D)\n", 220 | "nrm" 221 | ] 222 | }, 223 | { 224 | "cell_type": "markdown", 225 | "metadata": {}, 226 | "source": [ 227 | "### Constraints\n", 228 | "We have only two constraints, one per physical flight: that the number of people on each flight is less than the capacity of the planes. \n", 229 | "\n", 230 | "We add constraints with `@constraint`, which takes two arguments: the model, and an expression with an inequality in it: `<=`, `==`, `>=`.\n", 231 | "\n", 232 | "(note that there are other forms of `@constraint` that can be useful sometimes - see the manual or other examples for details)" 233 | ] 234 | }, 235 | { 236 | "cell_type": "code", 237 | "execution_count": 6, 238 | "metadata": { 239 | "collapsed": false 240 | }, 241 | "outputs": [ 242 | { 243 | "data": { 244 | "text/plain": [ 245 | "A JuMP Model" 246 | ] 247 | }, 248 | "execution_count": 6, 249 | "metadata": {}, 250 | "output_type": "execute_result" 251 | } 252 | ], 253 | "source": [ 254 | "@constraint(nrm, BOStoMDW_R + BOStoMDW_D + \n", 255 | " BOStoSFO_R + BOStoSFO_D <= 166)\n", 256 | "@constraint(nrm, MDWtoSFO_R + MDWtoSFO_D + \n", 257 | " BOStoSFO_R + BOStoSFO_D <= 166)\n", 258 | "nrm" 259 | ] 260 | }, 261 | { 262 | "cell_type": "markdown", 263 | "metadata": {}, 264 | "source": [ 265 | "Our model is complete!\n", 266 | "\n", 267 | "## Solve\n", 268 | "The easy part! Lets check out the finished model before solving it. We didn't specify a solver before, but JuMP knows we have an LP solver installed, so it will use that." 269 | ] 270 | }, 271 | { 272 | "cell_type": "code", 273 | "execution_count": 7, 274 | "metadata": { 275 | "collapsed": false 276 | }, 277 | "outputs": [], 278 | "source": [ 279 | "# solve problem\n", 280 | "JuMP.optimize(nrm)" 281 | ] 282 | }, 283 | { 284 | "cell_type": "code", 285 | "execution_count": 8, 286 | "metadata": { 287 | "collapsed": false 288 | }, 289 | "outputs": [ 290 | { 291 | "name": "stdout", 292 | "output_type": "stream", 293 | "text": [ 294 | "JuMP.hasresultvalues(nrm) = true\n", 295 | "JuMP.terminationstatus(nrm) == MOI.Success = true\n", 296 | "JuMP.primalstatus(nrm) == MOI.FeasiblePoint = true\n" 297 | ] 298 | }, 299 | { 300 | "data": { 301 | "text/plain": [ 302 | "true" 303 | ] 304 | }, 305 | "execution_count": 8, 306 | "metadata": {}, 307 | "output_type": "execute_result" 308 | } 309 | ], 310 | "source": [ 311 | "@show JuMP.hasresultvalues(nrm)\n", 312 | "@show JuMP.terminationstatus(nrm) == MOI.Success\n", 313 | "@show JuMP.primalstatus(nrm) == MOI.FeasiblePoint" 314 | ] 315 | }, 316 | { 317 | "cell_type": "markdown", 318 | "metadata": {}, 319 | "source": [ 320 | "## Inspect the solution\n", 321 | "We can use `getvalue()` to inspect the value of solutions" 322 | ] 323 | }, 324 | { 325 | "cell_type": "code", 326 | "execution_count": 9, 327 | "metadata": { 328 | "collapsed": false 329 | }, 330 | "outputs": [ 331 | { 332 | "data": { 333 | "text/plain": [ 334 | "80.0" 335 | ] 336 | }, 337 | "execution_count": 9, 338 | "metadata": {}, 339 | "output_type": "execute_result" 340 | } 341 | ], 342 | "source": [ 343 | "JuMP.resultvalue(BOStoMDW_R)" 344 | ] 345 | }, 346 | { 347 | "cell_type": "code", 348 | "execution_count": 10, 349 | "metadata": { 350 | "collapsed": false 351 | }, 352 | "outputs": [ 353 | { 354 | "data": { 355 | "text/plain": [ 356 | "11.0" 357 | ] 358 | }, 359 | "execution_count": 10, 360 | "metadata": {}, 361 | "output_type": "execute_result" 362 | } 363 | ], 364 | "source": [ 365 | "JuMP.resultvalue(BOStoMDW_D)" 366 | ] 367 | }, 368 | { 369 | "cell_type": "code", 370 | "execution_count": 11, 371 | "metadata": { 372 | "collapsed": false 373 | }, 374 | "outputs": [ 375 | { 376 | "ename": "LoadError", 377 | "evalue": "\u001b[91mUndefVarError: getobjectivevalue not defined\u001b[39m", 378 | "output_type": "error", 379 | "traceback": [ 380 | "\u001b[91mUndefVarError: getobjectivevalue not defined\u001b[39m", 381 | "" 382 | ] 383 | } 384 | ], 385 | "source": [ 386 | "JuMP.getobjectivevalue(nrm)" 387 | ] 388 | }, 389 | { 390 | "cell_type": "markdown", 391 | "metadata": {}, 392 | "source": [ 393 | "## General Model\n", 394 | "\n", 395 | "We'll now code our model in a more general way, to take any number of cities and flights. Hard-coding every variable would be painful and hard to update - it'd be better to *index* the variables, just like in mathematical notation. \n", 396 | "\n", 397 | "Consider a generalized version of our first problem, where there are flights in both directions and one extra airport YYZ - Toronto!\n", 398 | "\n", 399 | "Rather than hardcode data, we will generate some random data.\n", 400 | "\n", 401 | "*(If you don't understand all of this right away, thats OK - its not critical to understanding JuMP)*" 402 | ] 403 | }, 404 | { 405 | "cell_type": "code", 406 | "execution_count": 12, 407 | "metadata": { 408 | "collapsed": false 409 | }, 410 | "outputs": [ 411 | { 412 | "name": "stdout", 413 | "output_type": "stream", 414 | "text": [ 415 | "demand[(:BOS, :YYZ, :REG)] = 90\n" 416 | ] 417 | }, 418 | { 419 | "data": { 420 | "text/plain": [ 421 | "90" 422 | ] 423 | }, 424 | "execution_count": 12, 425 | "metadata": {}, 426 | "output_type": "execute_result" 427 | } 428 | ], 429 | "source": [ 430 | "# Set the random seed to ensure we always\n", 431 | "# get the same stream of 'random' numbers\n", 432 | "srand(1988) \n", 433 | "\n", 434 | "# Lets create a vector of symbols, one for each airport\n", 435 | "airports = [:BOS, :MDW, :SFO, :YYZ]\n", 436 | "num_airport = length(airports)\n", 437 | "\n", 438 | "# We'll also create a vector of fare classes\n", 439 | "classes = [:REG, :DIS]\n", 440 | "\n", 441 | "# All the demand and price data for each triple of\n", 442 | "# (origin, destination, class) will be stored in\n", 443 | "# 'dictionaries', also known as 'maps'.\n", 444 | "demand = Dict()\n", 445 | "prices = Dict()\n", 446 | "\n", 447 | "# Generate a demand and price for each pair of airports\n", 448 | "# To keep the code simple we will generate info for\n", 449 | "# nonsense flights like BOS-BOS and SFO-SFO, but they\n", 450 | "# won't appear in our final model.\n", 451 | "for origin in airports, dest in airports\n", 452 | " # Generate demand:\n", 453 | " # - Regular demand is Uniform(50,90)\n", 454 | " # - Discount demand is Uniform(100,130)\n", 455 | " demand[(origin,dest,:REG)] = rand(50:90) \n", 456 | " demand[(origin,dest,:DIS)] = rand(100:130)\n", 457 | " # Generate prices:\n", 458 | " # - Regular price is Uniform(400,700)\n", 459 | " # - Discount price is Uniform(150,300)\n", 460 | " prices[(origin,dest,:REG)] = rand(400:700)\n", 461 | " prices[(origin,dest,:DIS)] = rand(150:300)\n", 462 | "end\n", 463 | "\n", 464 | "# Finally set all places to have the same capacity\n", 465 | "plane_cap = rand(150:200)\n", 466 | "\n", 467 | "# Lets look at a sample demand at random\n", 468 | "@show demand[(:BOS,:YYZ,:REG)]" 469 | ] 470 | }, 471 | { 472 | "cell_type": "markdown", 473 | "metadata": {}, 474 | "source": [ 475 | "Now we have the data, we can build the model." 476 | ] 477 | }, 478 | { 479 | "cell_type": "code", 480 | "execution_count": 13, 481 | "metadata": { 482 | "collapsed": false 483 | }, 484 | "outputs": [ 485 | { 486 | "data": { 487 | "text/plain": [ 488 | "3-dimensional JuMPArray{JuMP.VariableRef,3,...} with index sets:\n", 489 | " Dimension 1, Symbol[:BOS, :MDW, :SFO, :YYZ]\n", 490 | " Dimension 2, Symbol[:BOS, :MDW, :SFO, :YYZ]\n", 491 | " Dimension 3, Symbol[:REG, :DIS]\n", 492 | "And data, a 4×4×2 Array{JuMP.VariableRef,3}:\n", 493 | "[:, :, :REG] =\n", 494 | " x[BOS,BOS,REG] x[BOS,MDW,REG] x[BOS,SFO,REG] x[BOS,YYZ,REG]\n", 495 | " x[MDW,BOS,REG] x[MDW,MDW,REG] x[MDW,SFO,REG] x[MDW,YYZ,REG]\n", 496 | " x[SFO,BOS,REG] x[SFO,MDW,REG] x[SFO,SFO,REG] x[SFO,YYZ,REG]\n", 497 | " x[YYZ,BOS,REG] x[YYZ,MDW,REG] x[YYZ,SFO,REG] x[YYZ,YYZ,REG]\n", 498 | "\n", 499 | "[:, :, :DIS] =\n", 500 | " x[BOS,BOS,DIS] x[BOS,MDW,DIS] x[BOS,SFO,DIS] x[BOS,YYZ,DIS]\n", 501 | " x[MDW,BOS,DIS] x[MDW,MDW,DIS] x[MDW,SFO,DIS] x[MDW,YYZ,DIS]\n", 502 | " x[SFO,BOS,DIS] x[SFO,MDW,DIS] x[SFO,SFO,DIS] x[SFO,YYZ,DIS]\n", 503 | " x[YYZ,BOS,DIS] x[YYZ,MDW,DIS] x[YYZ,SFO,DIS] x[YYZ,YYZ,DIS]" 504 | ] 505 | }, 506 | "execution_count": 13, 507 | "metadata": {}, 508 | "output_type": "execute_result" 509 | } 510 | ], 511 | "source": [ 512 | "nrm2 = Model(optimizer = GLPK.GLPKOptimizerLP())\n", 513 | "# Create a variable indexed by 3 things:\n", 514 | "# * Origin\n", 515 | "# * Destination\n", 516 | "# * Class\n", 517 | "# And set the upper bound for each variable differently.\n", 518 | "# The origins and destinations should be indexed across\n", 519 | "# the vector of airports, and the class should be indexed\n", 520 | "# across the vector of classes.\n", 521 | "# We'll draw the upper bounds from the demand dictionary.\n", 522 | "@variable(nrm2, 0 <= x[o=airports,\n", 523 | " d=airports,\n", 524 | " c=classes] <= demand[(o,d,c)])\n", 525 | "# Note that we don't have to split it up across multiple lines,\n", 526 | "# its personal preference!" 527 | ] 528 | }, 529 | { 530 | "cell_type": "markdown", 531 | "metadata": {}, 532 | "source": [ 533 | "JuMP displays the variable in a compact form - note that the upper bound is\n", 534 | "blank because the upper bound is different for each variable." 535 | ] 536 | }, 537 | { 538 | "cell_type": "code", 539 | "execution_count": 14, 540 | "metadata": { 541 | "collapsed": false 542 | }, 543 | "outputs": [], 544 | "source": [ 545 | "# The objective is just like before, except now we can use\n", 546 | "# the sum() functionality provided by JuMP to sum over all\n", 547 | "# combinations of origin/destination/class, and provide a\n", 548 | "# filter to exclude all cases where\n", 549 | "# the origin is equal to the destination\n", 550 | "@objective(nrm2, Max, sum(prices[(o,d,c)]*x[o,d,c] for \n", 551 | " o in airports, d in airports, c in classes if o != d))" 552 | ] 553 | }, 554 | { 555 | "cell_type": "code", 556 | "execution_count": 15, 557 | "metadata": { 558 | "collapsed": false 559 | }, 560 | "outputs": [ 561 | { 562 | "name": "stdout", 563 | "output_type": "stream", 564 | "text": [ 565 | "Adding constraint for hub (MDW) to BOS\n", 566 | "Adding constraint for hub (MDW) to SFO\n", 567 | "Adding constraint for hub (MDW) to YYZ\n" 568 | ] 569 | }, 570 | { 571 | "data": { 572 | "text/plain": [ 573 | "A JuMP Model" 574 | ] 575 | }, 576 | "execution_count": 15, 577 | "metadata": {}, 578 | "output_type": "execute_result" 579 | } 580 | ], 581 | "source": [ 582 | "# Next we'll add constraints on flights from the hub\n", 583 | "# to any of the non-hub airports.\n", 584 | "for d in airports\n", 585 | " if d == :MDW\n", 586 | " continue\n", 587 | " end\n", 588 | " println(\"Adding constraint for hub (MDW) to $d\")\n", 589 | " @constraint(nrm2, \n", 590 | " sum(x[o,d,c] for o in airports, c in classes if o!=d) <= plane_cap)\n", 591 | "end\n", 592 | "nrm2" 593 | ] 594 | }, 595 | { 596 | "cell_type": "code", 597 | "execution_count": 16, 598 | "metadata": { 599 | "collapsed": false 600 | }, 601 | "outputs": [ 602 | { 603 | "name": "stdout", 604 | "output_type": "stream", 605 | "text": [ 606 | "Adding constraint for BOS to hub (MDW)\n", 607 | "Adding constraint for SFO to hub (MDW)\n", 608 | "Adding constraint for YYZ to hub (MDW)\n" 609 | ] 610 | }, 611 | { 612 | "data": { 613 | "text/plain": [ 614 | "A JuMP Model" 615 | ] 616 | }, 617 | "execution_count": 16, 618 | "metadata": {}, 619 | "output_type": "execute_result" 620 | } 621 | ], 622 | "source": [ 623 | "# Now flights into the hub\n", 624 | "for o in airports\n", 625 | " if o == :MDW\n", 626 | " continue\n", 627 | " end\n", 628 | " println(\"Adding constraint for $o to hub (MDW)\")\n", 629 | " @constraint(nrm2, \n", 630 | " sum(x[o,d,c] for d in airports, c in classes if o!=d) <= plane_cap)\n", 631 | "end\n", 632 | "nrm2" 633 | ] 634 | }, 635 | { 636 | "cell_type": "code", 637 | "execution_count": 17, 638 | "metadata": { 639 | "collapsed": false 640 | }, 641 | "outputs": [], 642 | "source": [ 643 | "# solve problem\n", 644 | "JuMP.optimize(nrm2)" 645 | ] 646 | }, 647 | { 648 | "cell_type": "code", 649 | "execution_count": 18, 650 | "metadata": { 651 | "collapsed": false 652 | }, 653 | "outputs": [ 654 | { 655 | "data": { 656 | "text/plain": [ 657 | "3-dimensional JuMPArray{Float64,3,...} with index sets:\n", 658 | " Dimension 1, Symbol[:BOS, :MDW, :SFO, :YYZ]\n", 659 | " Dimension 2, Symbol[:BOS, :MDW, :SFO, :YYZ]\n", 660 | " Dimension 3, Symbol[:REG, :DIS]\n", 661 | "And data, a 4×4×2 Array{Float64,3}:\n", 662 | "[:, :, :REG] =\n", 663 | " 0.0 55.0 46.0 81.0\n", 664 | " 86.0 0.0 75.0 63.0\n", 665 | " 54.0 90.0 0.0 38.0\n", 666 | " 0.0 62.0 61.0 0.0\n", 667 | "\n", 668 | "[:, :, :DIS] =\n", 669 | " 0.0 0.0 0.0 0.0\n", 670 | " 42.0 0.0 0.0 0.0\n", 671 | " 0.0 0.0 0.0 0.0\n", 672 | " 0.0 59.0 0.0 0.0" 673 | ] 674 | }, 675 | "execution_count": 18, 676 | "metadata": {}, 677 | "output_type": "execute_result" 678 | } 679 | ], 680 | "source": [ 681 | "JuMP.resultvalue.(x)" 682 | ] 683 | }, 684 | { 685 | "cell_type": "code", 686 | "execution_count": null, 687 | "metadata": { 688 | "collapsed": true 689 | }, 690 | "outputs": [], 691 | "source": [] 692 | } 693 | ], 694 | "metadata": { 695 | "anaconda-cloud": {}, 696 | "kernelspec": { 697 | "display_name": "Julia 0.6.0", 698 | "language": "julia", 699 | "name": "julia-0.6" 700 | }, 701 | "language_info": { 702 | "file_extension": ".jl", 703 | "mimetype": "application/julia", 704 | "name": "julia", 705 | "version": "0.6.0" 706 | } 707 | }, 708 | "nbformat": 4, 709 | "nbformat_minor": 0 710 | } 711 | -------------------------------------------------------------------------------- /notebooks/JuMP-SensitivityAnalysis.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "**Description**: Shows how to implement sensitivity analysis in JuMP.\n", 8 | "\n", 9 | "**Author**: Jack Dunn\n", 10 | "\n", 11 | "**License**: \"Creative
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License." 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "metadata": {}, 17 | "source": [ 18 | "## Sensitivity Analysis using JuMP\n", 19 | "\n", 20 | "In this notebook, we will see how to use JuMP to conduct sensitivity analysis after solving a linear program and replicate the output offered by other packages (e.g. the Sensitivity Report in Excel Solver).\n", 21 | "\n", 22 | "We will consder a production planning problem by J E Beasley, made available on [OR-Notes](http://people.brunel.ac.uk/~mastjjb/jeb/or/lpsens_solver.html). \n", 23 | "\n", 24 | "### Problem Statement and Model Formulation\n", 25 | "\n", 26 | "(This section is reproduced exactly from the above link)\n", 27 | "\n", 28 | "A company manufactures four variants of the same product and in the final part of the manufacturing process there are assembly, polishing and packing operations. For each variant the time required for these operations is shown below (in minutes) as is the profit per unit sold.\n", 29 | "\n", 30 | "| Variant | Assembly | Polish | Pack | Profit |\n", 31 | "| :-----: | :------: | :----: | :--: | :----: |\n", 32 | "| 1 | 2 | 3 | 2 | 1.50 |\n", 33 | "| 2 | 4 | 2 | 3 | 2.50 |\n", 34 | "| 3 | 3 | 3 | 2 | 3.00 |\n", 35 | "| 4 | 7 | 4 | 5 | 4.50 |\n", 36 | "\n", 37 | "Given the current state of the labour force the company estimate that, each year, they have 100000 minutes of assembly time, 50000 minutes of polishing time and 60000 minutes of packing time available. How many of each variant should the company make per year and what is the associated profit?\n", 38 | "\n", 39 | "#### Variables\n", 40 | "\n", 41 | "Let $x_i \\geq 0$ be the number of units of variant i ($i=1,2,3,4$) made per year\n", 42 | "\n", 43 | "#### Constraints\n", 44 | "\n", 45 | "The operation time limits give the following constraints:\n", 46 | "\n", 47 | "- $2 x_1 + 4 x_2 + 3 x_3 + 7 x_4 \\leq 100000~~~$ (assembly) \n", 48 | "- $3 x_1 + 2 x_2 + 3 x_3 + 4 x_4 \\leq 50000~~~$ (polish) \n", 49 | "- $2 x_1 + 3 x_2 + 2 x_3 + 5 x_4 \\leq 60000~~~$ (pack)\n", 50 | "\n", 51 | "#### Objective\n", 52 | "\n", 53 | "Presumably to maximise profit - hence we have\n", 54 | "\n", 55 | "- $\\max 1.5 x_1 + 2.5 x_2 + 3.0 x_3 + 4.5 x_4$\n", 56 | "\n", 57 | "### Complete Formulation\n", 58 | "\n", 59 | "$$\\begin{align*}\n", 60 | "\\max~~~ &1.5 x_1 + 2.5 x_2 + 3.0 x_3 + 4.5 x_4 \\\\\n", 61 | "\\textrm{s.t.}~~~ &2 x_1 + 4 x_2 + 3 x_3 + 7 x_4 \\leq 100000 \\\\\n", 62 | " &3 x_1 + 2 x_2 + 3 x_3 + 4 x_4 \\leq 50000 \\\\\n", 63 | " &2 x_1 + 3 x_2 + 2 x_3 + 5 x_4 \\leq 60000 \\\\\n", 64 | " &x_i \\geq 0,~~~i = 1, 2, 3, 4\n", 65 | "\\end{align*}$$" 66 | ] 67 | }, 68 | { 69 | "cell_type": "markdown", 70 | "metadata": {}, 71 | "source": [ 72 | "### Solving the model in JuMP\n", 73 | "\n", 74 | "Now we can formulate and solve this model in JuMP:" 75 | ] 76 | }, 77 | { 78 | "cell_type": "code", 79 | "execution_count": 1, 80 | "metadata": { 81 | "collapsed": false 82 | }, 83 | "outputs": [ 84 | { 85 | "name": "stdout", 86 | "output_type": "stream", 87 | "text": [ 88 | "Optimize a model with 3 rows, 4 columns and 12 nonzeros\n", 89 | "Coefficient statistics:\n", 90 | " Matrix range [2e+00, 7e+00]\n", 91 | " Objective range [2e+00, 4e+00]\n", 92 | " Bounds range [0e+00, 0e+00]\n", 93 | " RHS range [5e+04, 1e+05]\n", 94 | "Presolve time: 0.00s\n", 95 | "Presolved: 3 rows, 4 columns, 12 nonzeros\n", 96 | "\n", 97 | "Iteration Objective Primal Inf. Dual Inf. Time\n", 98 | " 0 1.4000000e+31 7.875000e+30 1.400000e+01 0s\n", 99 | " 2 5.8000000e+04 0.000000e+00 0.000000e+00 0s\n", 100 | "\n", 101 | "Solved in 2 iterations and 0.00 seconds\n", 102 | "Optimal objective 5.800000000e+04\n", 103 | "getvalue(x) = [0.0,16000.000000000002,5999.999999999999,0.0]\n", 104 | "getobjectivevalue(model) = 58000.0\n" 105 | ] 106 | } 107 | ], 108 | "source": [ 109 | "# Define the data\n", 110 | "m = 3\n", 111 | "n = 4\n", 112 | "c = [1.5; 2.5; 3.0; 4.5]\n", 113 | "A = [2 4 3 7;\n", 114 | " 3 2 3 4;\n", 115 | " 2 3 2 5]\n", 116 | "b = [100000; 50000; 60000]\n", 117 | "\n", 118 | "# Import necessary packages and define model\n", 119 | "using JuMP\n", 120 | "using Gurobi # We need Gurobi for Sensitivity Analysis later\n", 121 | "model = Model(solver=GurobiSolver())\n", 122 | "\n", 123 | "# Define the variables\n", 124 | "@variable(model, x[i=1:n] >= 0)\n", 125 | "\n", 126 | "# Add the objective\n", 127 | "@objective(model, Max, sum{c[i] * x[i], i=1:n})\n", 128 | "\n", 129 | "# Add the constraints row-by-row, naming them according to each resource\n", 130 | "# The names will allow us to refer to the constraints during sensitivity analysis\n", 131 | "@constraint(model, assembly, dot(vec(A[1,:]), x) <= b[1])\n", 132 | "@constraint(model, polish, dot(vec(A[2,:]), x) <= b[2])\n", 133 | "@constraint(model, pack, dot(vec(A[3,:]), x) <= b[3])\n", 134 | "\n", 135 | "# Solve the model and show the optimal solution and objective value\n", 136 | "solve(model)\n", 137 | "@show getvalue(x)\n", 138 | "@show getobjectivevalue(model);" 139 | ] 140 | }, 141 | { 142 | "cell_type": "markdown", 143 | "metadata": {}, 144 | "source": [ 145 | "We see that the optimal production plan is to make 16000 units of variant 1 and 6000 units of variant 2, with an optimal profit of \\$58000" 146 | ] 147 | }, 148 | { 149 | "cell_type": "markdown", 150 | "metadata": {}, 151 | "source": [ 152 | "### Sensitivity Analysis\n", 153 | "\n", 154 | "Once we have solved a model, it is often useful to analyze the sensitivity of the solution to the model parameters. Other modeling tools like Excel Solver can produce a Sensitivity Report, which summarizes all of the sensitivity information in one table. \n", 155 | "\n", 156 | "The Sensitivity Report produced for the production planning solution above is as follows (image from OR-Tools):\n", 157 | "\n", 158 | "![Sensitivity Report](https://i.imgur.com/6tYVVwH.gif)\n", 159 | "\n", 160 | "The table contains information on the shadow prices and reduced costs in the model, as well as the ranges on the cost coefficients and right-hand side values for which the current basis is optimal.\n", 161 | "\n", 162 | "We will now reproduce this table using our JuMP model\n", 163 | "\n", 164 | "#### Final Values\n", 165 | "\n", 166 | "We can get the final values of the variables with `getValue()` as before:" 167 | ] 168 | }, 169 | { 170 | "cell_type": "code", 171 | "execution_count": 2, 172 | "metadata": { 173 | "collapsed": false 174 | }, 175 | "outputs": [ 176 | { 177 | "data": { 178 | "text/plain": [ 179 | "4-element Array{Float64,1}:\n", 180 | " 0.0\n", 181 | " 16000.0\n", 182 | " 6000.0\n", 183 | " 0.0" 184 | ] 185 | }, 186 | "execution_count": 2, 187 | "metadata": {}, 188 | "output_type": "execute_result" 189 | } 190 | ], 191 | "source": [ 192 | "x_final = getvalue(x)" 193 | ] 194 | }, 195 | { 196 | "cell_type": "markdown", 197 | "metadata": {}, 198 | "source": [ 199 | "We can get the final values of the constraints by calculating $Ax$:" 200 | ] 201 | }, 202 | { 203 | "cell_type": "code", 204 | "execution_count": 3, 205 | "metadata": { 206 | "collapsed": false 207 | }, 208 | "outputs": [ 209 | { 210 | "data": { 211 | "text/plain": [ 212 | "3-element Array{Float64,1}:\n", 213 | " 82000.0\n", 214 | " 50000.0\n", 215 | " 60000.0" 216 | ] 217 | }, 218 | "execution_count": 3, 219 | "metadata": {}, 220 | "output_type": "execute_result" 221 | } 222 | ], 223 | "source": [ 224 | "con_final = A * x_final" 225 | ] 226 | }, 227 | { 228 | "cell_type": "markdown", 229 | "metadata": {}, 230 | "source": [ 231 | "#### Reduced Costs/Shadow Prices\n", 232 | "\n", 233 | "We can extract the reduced costs by calling `getDual()` on the variables:" 234 | ] 235 | }, 236 | { 237 | "cell_type": "code", 238 | "execution_count": 4, 239 | "metadata": { 240 | "collapsed": false 241 | }, 242 | "outputs": [ 243 | { 244 | "data": { 245 | "text/plain": [ 246 | "4-element Array{Float64,1}:\n", 247 | " -1.5\n", 248 | " 0.0\n", 249 | " 0.0\n", 250 | " -0.2" 251 | ] 252 | }, 253 | "execution_count": 4, 254 | "metadata": {}, 255 | "output_type": "execute_result" 256 | } 257 | ], 258 | "source": [ 259 | "red_costs = getdual(x)" 260 | ] 261 | }, 262 | { 263 | "cell_type": "markdown", 264 | "metadata": {}, 265 | "source": [ 266 | "Similarly, we can extract the shadow prices by using `getDual()` on our constraint references:" 267 | ] 268 | }, 269 | { 270 | "cell_type": "code", 271 | "execution_count": 5, 272 | "metadata": { 273 | "collapsed": false 274 | }, 275 | "outputs": [ 276 | { 277 | "data": { 278 | "text/plain": [ 279 | "0.0" 280 | ] 281 | }, 282 | "execution_count": 5, 283 | "metadata": {}, 284 | "output_type": "execute_result" 285 | } 286 | ], 287 | "source": [ 288 | "getdual(assembly)" 289 | ] 290 | }, 291 | { 292 | "cell_type": "code", 293 | "execution_count": 6, 294 | "metadata": { 295 | "collapsed": false 296 | }, 297 | "outputs": [ 298 | { 299 | "data": { 300 | "text/plain": [ 301 | "0.8" 302 | ] 303 | }, 304 | "execution_count": 6, 305 | "metadata": {}, 306 | "output_type": "execute_result" 307 | } 308 | ], 309 | "source": [ 310 | "getdual(polish)" 311 | ] 312 | }, 313 | { 314 | "cell_type": "code", 315 | "execution_count": 7, 316 | "metadata": { 317 | "collapsed": false 318 | }, 319 | "outputs": [ 320 | { 321 | "data": { 322 | "text/plain": [ 323 | "0.3" 324 | ] 325 | }, 326 | "execution_count": 7, 327 | "metadata": {}, 328 | "output_type": "execute_result" 329 | } 330 | ], 331 | "source": [ 332 | "getdual(pack)" 333 | ] 334 | }, 335 | { 336 | "cell_type": "markdown", 337 | "metadata": {}, 338 | "source": [ 339 | "We can put these together into a vector as well:" 340 | ] 341 | }, 342 | { 343 | "cell_type": "code", 344 | "execution_count": 8, 345 | "metadata": { 346 | "collapsed": false 347 | }, 348 | "outputs": [ 349 | { 350 | "data": { 351 | "text/plain": [ 352 | "3-element Array{Float64,1}:\n", 353 | " 0.0\n", 354 | " 0.8\n", 355 | " 0.3" 356 | ] 357 | }, 358 | "execution_count": 8, 359 | "metadata": {}, 360 | "output_type": "execute_result" 361 | } 362 | ], 363 | "source": [ 364 | "shadow_prices = getdual([assembly; polish; pack])" 365 | ] 366 | }, 367 | { 368 | "cell_type": "markdown", 369 | "metadata": {}, 370 | "source": [ 371 | "#### Current Values\n", 372 | "\n", 373 | "The \"Objective Coefficient\" and \"Constraint R. H. Side\" columns simply contain the values of $c$ and $b$, respectively:" 374 | ] 375 | }, 376 | { 377 | "cell_type": "code", 378 | "execution_count": 9, 379 | "metadata": { 380 | "collapsed": false 381 | }, 382 | "outputs": [ 383 | { 384 | "data": { 385 | "text/plain": [ 386 | "4-element Array{Float64,1}:\n", 387 | " 1.5\n", 388 | " 2.5\n", 389 | " 3.0\n", 390 | " 4.5" 391 | ] 392 | }, 393 | "execution_count": 9, 394 | "metadata": {}, 395 | "output_type": "execute_result" 396 | } 397 | ], 398 | "source": [ 399 | "obj_coeff = c" 400 | ] 401 | }, 402 | { 403 | "cell_type": "code", 404 | "execution_count": 10, 405 | "metadata": { 406 | "collapsed": false 407 | }, 408 | "outputs": [ 409 | { 410 | "data": { 411 | "text/plain": [ 412 | "3-element Array{Int64,1}:\n", 413 | " 100000\n", 414 | " 50000\n", 415 | " 60000" 416 | ] 417 | }, 418 | "execution_count": 10, 419 | "metadata": {}, 420 | "output_type": "execute_result" 421 | } 422 | ], 423 | "source": [ 424 | "con_rhs = b" 425 | ] 426 | }, 427 | { 428 | "cell_type": "markdown", 429 | "metadata": {}, 430 | "source": [ 431 | "#### Allowable Increase/Decrease\n", 432 | "\n", 433 | "We now want to retrieve the range of parameter values for which the optimal basis does not change. There are multiple ways to do this.\n", 434 | "\n", 435 | "One approach is to get the optimal basis using the `MathProgBase.getbasis()` function, and this can then be used to calculate the allowable ranges of increase and decrease using standard linear programming theory of sensitivity analysis.\n", 436 | "\n", 437 | "Alternatively, some solvers (like Gurobi) provide this information directly without the need for us to compute it manually. In this case, we can access the data through the Gurobi API directly, which is what we will do in the rest of this example. " 438 | ] 439 | }, 440 | { 441 | "cell_type": "code", 442 | "execution_count": 11, 443 | "metadata": { 444 | "collapsed": false 445 | }, 446 | "outputs": [ 447 | { 448 | "data": { 449 | "text/plain": [ 450 | "Gurobi Model: \n", 451 | " type : LP\n", 452 | " sense : maximize\n", 453 | " number of variables = 4\n", 454 | " number of linear constraints = 3\n", 455 | " number of quadratic constraints = 0\n", 456 | " number of sos constraints = 0\n", 457 | " number of non-zero coeffs = 12\n", 458 | " number of non-zero qp objective terms = 0\n", 459 | " number of non-zero qp constraint terms = 0\n" 460 | ] 461 | }, 462 | "execution_count": 11, 463 | "metadata": {}, 464 | "output_type": "execute_result" 465 | } 466 | ], 467 | "source": [ 468 | "# Import Gurobi so we can access the API\n", 469 | "import Gurobi\n", 470 | "\n", 471 | "# Get a reference to the internal Gurobi model so we can use the API\n", 472 | "g = getrawsolver(model)" 473 | ] 474 | }, 475 | { 476 | "cell_type": "markdown", 477 | "metadata": { 478 | "collapsed": true 479 | }, 480 | "source": [ 481 | "We can now access the sensitivity information directly. To do this, we make use of the `get_dblattrarray()` function from [Gurobi.jl](https://github.com/JuliaOpt/Gurobi.jl/blob/master/src/grb_attrs.jl#L58), which allows us to retrieve [attributes from the Gurobi API](http://www.gurobi.com/documentation/6.5/refman/attributes.html) that are arrays of floating point values. \n", 482 | "\n", 483 | "When calling `get_dblattrarray()`, we have to specify the internal Gurobi model, the name of the attribute we want to retrieve, the index at which to starting reading from the array, and the number of values to read.\n", 484 | "\n", 485 | "In our case, we use the `SARHSLow` and `SARHSUp` attributes to get the lower and upper bounds on the RHS values, and in each case we start at the first value and read in a total of $m$ values. Similarly, we use `SAObjLow` and `SAObjUp` to get the lower and upper bounds for the objective coefficients, and in this case we read in all $n$ values." 486 | ] 487 | }, 488 | { 489 | "cell_type": "code", 490 | "execution_count": 12, 491 | "metadata": { 492 | "collapsed": false 493 | }, 494 | "outputs": [ 495 | { 496 | "name": "stdout", 497 | "output_type": "stream", 498 | "text": [ 499 | "rhs_lb = [82000.0,40000.0,33333.33333333333]\n", 500 | "rhs_ub = [1.0e100,90000.0,75000.0]\n", 501 | "obj_lb = [-1.0e100,2.357142857142857,2.499999999999999,-1.0e100]\n", 502 | "obj_ub = [3.0000000000000004,4.5,3.75,4.7]\n" 503 | ] 504 | } 505 | ], 506 | "source": [ 507 | "# RHS value lower and upper bounds\n", 508 | "rhs_lb = Gurobi.get_dblattrarray(g, \"SARHSLow\", 1, m)\n", 509 | "rhs_ub = Gurobi.get_dblattrarray(g, \"SARHSUp\", 1, m)\n", 510 | "@show rhs_lb\n", 511 | "@show rhs_ub\n", 512 | "\n", 513 | "# Objective coefficient lower and upper bounds\n", 514 | "obj_lb = Gurobi.get_dblattrarray(g, \"SAObjLow\", 1, n)\n", 515 | "obj_ub = Gurobi.get_dblattrarray(g, \"SAObjUp\", 1, n)\n", 516 | "@show obj_lb\n", 517 | "@show obj_ub;" 518 | ] 519 | }, 520 | { 521 | "cell_type": "markdown", 522 | "metadata": {}, 523 | "source": [ 524 | "The order of values in these arrays is not necessarily obvious in larger problems, and generally we do not know the order in which the information is returned by Gurobi. We can use the `getLinearIndex()` function on our variables and constraints to find their position in these arrays:" 525 | ] 526 | }, 527 | { 528 | "cell_type": "code", 529 | "execution_count": 13, 530 | "metadata": { 531 | "collapsed": false 532 | }, 533 | "outputs": [ 534 | { 535 | "data": { 536 | "text/plain": [ 537 | "4-element Array{Int64,1}:\n", 538 | " 1\n", 539 | " 2\n", 540 | " 3\n", 541 | " 4" 542 | ] 543 | }, 544 | "execution_count": 13, 545 | "metadata": {}, 546 | "output_type": "execute_result" 547 | } 548 | ], 549 | "source": [ 550 | "x_order = map(linearindex, x)" 551 | ] 552 | }, 553 | { 554 | "cell_type": "code", 555 | "execution_count": 14, 556 | "metadata": { 557 | "collapsed": false 558 | }, 559 | "outputs": [ 560 | { 561 | "data": { 562 | "text/plain": [ 563 | "3-element Array{Int64,1}:\n", 564 | " 1\n", 565 | " 2\n", 566 | " 3" 567 | ] 568 | }, 569 | "execution_count": 14, 570 | "metadata": {}, 571 | "output_type": "execute_result" 572 | } 573 | ], 574 | "source": [ 575 | "con_order = map(linearindex, [assembly, polish, pack])" 576 | ] 577 | }, 578 | { 579 | "cell_type": "markdown", 580 | "metadata": {}, 581 | "source": [ 582 | "We see that the variables and constraints are already ordered for us, but this isn't true all the time, so it pays to always rearrange the arrays according to this ordering in case the order is different" 583 | ] 584 | }, 585 | { 586 | "cell_type": "code", 587 | "execution_count": 15, 588 | "metadata": { 589 | "collapsed": false 590 | }, 591 | "outputs": [], 592 | "source": [ 593 | "rhs_lb_sorted = rhs_lb[con_order];\n", 594 | "rhs_ub_sorted = rhs_ub[con_order];\n", 595 | "obj_lb_sorted = obj_lb[x_order];\n", 596 | "obj_ub_sorted = obj_ub[x_order];" 597 | ] 598 | }, 599 | { 600 | "cell_type": "markdown", 601 | "metadata": {}, 602 | "source": [ 603 | "Now, we can use these lower and upper bounds to obtain the allowable increase and decrease on each objective coefficient and RHS value:" 604 | ] 605 | }, 606 | { 607 | "cell_type": "code", 608 | "execution_count": 16, 609 | "metadata": { 610 | "collapsed": false 611 | }, 612 | "outputs": [ 613 | { 614 | "name": "stdout", 615 | "output_type": "stream", 616 | "text": [ 617 | "rhs_dec = con_rhs - rhs_lb_sorted = [18000.0,10000.0,26666.66666666667]\n", 618 | "rhs_inc = rhs_ub_sorted - con_rhs = [1.0e100,40000.0,15000.0]\n", 619 | "obj_dec = obj_coeff - obj_lb_sorted = [1.0e100,0.1428571428571428,0.5000000000000009,1.0e100]\n", 620 | "obj_inc = obj_ub_sorted - obj_coeff = [1.5000000000000004,2.0,0.75,0.20000000000000018]\n" 621 | ] 622 | } 623 | ], 624 | "source": [ 625 | "@show rhs_dec = con_rhs - rhs_lb_sorted;\n", 626 | "@show rhs_inc = rhs_ub_sorted - con_rhs;\n", 627 | "\n", 628 | "@show obj_dec = obj_coeff - obj_lb_sorted;\n", 629 | "@show obj_inc = obj_ub_sorted - obj_coeff;" 630 | ] 631 | }, 632 | { 633 | "cell_type": "markdown", 634 | "metadata": {}, 635 | "source": [ 636 | "### Final Sensitivity Table\n", 637 | "\n", 638 | "We can put all of this together to form the final Sensitivity Report tables.\n", 639 | "\n", 640 | "First, the variables:" 641 | ] 642 | }, 643 | { 644 | "cell_type": "code", 645 | "execution_count": 17, 646 | "metadata": { 647 | "collapsed": false 648 | }, 649 | "outputs": [ 650 | { 651 | "data": { 652 | "text/plain": [ 653 | "4x5 Array{Float64,2}:\n", 654 | " 0.0 -1.5 1.5 1.5 1.0e100 \n", 655 | " 16000.0 0.0 2.5 2.0 0.142857\n", 656 | " 6000.0 0.0 3.0 0.75 0.5 \n", 657 | " 0.0 -0.2 4.5 0.2 1.0e100 " 658 | ] 659 | }, 660 | "execution_count": 17, 661 | "metadata": {}, 662 | "output_type": "execute_result" 663 | } 664 | ], 665 | "source": [ 666 | "var_sensitivity = [x_final red_costs obj_coeff obj_inc obj_dec]" 667 | ] 668 | }, 669 | { 670 | "cell_type": "markdown", 671 | "metadata": {}, 672 | "source": [ 673 | "And similarly for the constraints:" 674 | ] 675 | }, 676 | { 677 | "cell_type": "code", 678 | "execution_count": 18, 679 | "metadata": { 680 | "collapsed": false 681 | }, 682 | "outputs": [ 683 | { 684 | "data": { 685 | "text/plain": [ 686 | "3x5 Array{Float64,2}:\n", 687 | " 82000.0 0.0 100000.0 1.0e100 18000.0\n", 688 | " 50000.0 0.8 50000.0 40000.0 10000.0\n", 689 | " 60000.0 0.3 60000.0 15000.0 26666.7" 690 | ] 691 | }, 692 | "execution_count": 18, 693 | "metadata": {}, 694 | "output_type": "execute_result" 695 | } 696 | ], 697 | "source": [ 698 | "con_sensitivity = [con_final shadow_prices con_rhs rhs_inc rhs_dec]" 699 | ] 700 | }, 701 | { 702 | "cell_type": "markdown", 703 | "metadata": {}, 704 | "source": [ 705 | "This leads to the following tables, which we see are identical to those produced by Excel Solver:\n", 706 | "\n", 707 | "**Variables:**\n", 708 | "\n", 709 | "| Name | Final Value | Red. Cost | Obj. Coeff | Allow. Inc. | Allow. Dec. |\n", 710 | "| :---: | ----------: | --------: | ---------: | ----------: | ----------: |\n", 711 | "| $x_1$ | 0 | -1.5 | 1.5 | 1.50 | 1E+100 |\n", 712 | "| $x_2$ | 16000 | 0.0 | 2.5 | 2.00 | 0.1429 |\n", 713 | "| $x_3$ | 6000 | 0.0 | 3.0 | 0.75 | 0.5 |\n", 714 | "| $x_4$ | 0 | -0.2 | 4.5 | 0.20 | 1E+100 |\n", 715 | "\n", 716 | "**Constraints:**\n", 717 | "\n", 718 | "| Name | Final Value | Shadow Price | RHS | Allow. Inc. | Allow. Dec. |\n", 719 | "| :------: | ----------: | -----------: | -----: | ----------: | ----------: |\n", 720 | "| Assembly | 82000 | 0.0 | 100000 | 1E+100 | 18000 |\n", 721 | "| Polish | 50000 | 0.8 | 50000 | 40000 | 10000 |\n", 722 | "| Pack | 60000 | 0.3 | 60000 | 15000 | 26667 |\n" 723 | ] 724 | } 725 | ], 726 | "metadata": { 727 | "kernelspec": { 728 | "display_name": "Julia 0.4.3", 729 | "language": "julia", 730 | "name": "julia-0.4" 731 | }, 732 | "language_info": { 733 | "file_extension": ".jl", 734 | "mimetype": "application/julia", 735 | "name": "julia", 736 | "version": "0.4.3" 737 | } 738 | }, 739 | "nbformat": 4, 740 | "nbformat_minor": 0 741 | } 742 | -------------------------------------------------------------------------------- /notebooks/Shuvomoy - Benders decomposition.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Disclaimer\n", 8 | "This notebook is only working under the versions:\n", 9 | "\n", 10 | "- JuMP 0.19 (unreleased, but currently in master)\n", 11 | "\n", 12 | "- MathOptInterface 0.4.1\n", 13 | "\n", 14 | "- GLPK 0.6.0\n", 15 | "\n", 16 | "## Disclaimer 2\n", 17 | "\n", 18 | "The second part od this notebook (using Lazy Constraints) is not working" 19 | ] 20 | }, 21 | { 22 | "cell_type": "markdown", 23 | "metadata": {}, 24 | "source": [ 25 | "**Description:** This notebook describes how to implement Benders decomposition, which is a large scale optimization scheme, in JuMP. Both the classical approach (using loop) and the modern approach (using lazy constraints) are described.\n", 26 | "\n", 27 | "**Author:** [Shuvomoy Das Gupta](http://scg.utoronto.ca/~shuvomoy.dasgupta/)\n", 28 | "\n", 29 | "**License:** \"Creative
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.\n", 30 | "\n", 31 | "# Using Julia+JuMP for optimization - Benders decomposition\n", 32 | "\n", 33 | "\n", 34 | "--------------------------\n", 35 | "
\n", 36 | "\n", 37 | "To illustrate implementation of solver callback in JuMP, we consider applying Benders decomposition to the following general mixed integer problem:\n", 38 | "\n", 39 | "\\begin{align}\n", 40 | "& \\text{maximize} \\quad &&c_1^T x+c_2^T v \\\\\n", 41 | "& \\text{subject to} \\quad &&A_1 x+ A_2 v \\preceq b \\\\\n", 42 | "& &&x \\succeq 0, x \\in \\mathbb{Z}^n \\\\\n", 43 | "& &&v \\succeq 0, v \\in \\mathbb{R}^p \\\\\n", 44 | "\\end{align}\n", 45 | "\n", 46 | "where $b \\in \\mathbb{R}^m$, $A_1 \\in \\mathbb{R}^{m \\times n}$ and $A_2 \\in \\mathbb{R}^{m \\times p}$. Here the symbol $\\succeq$ ($\\preceq$) stands for element-wise greater (less) than or equal to. Any mixed integer programming problem can be written in the form above.\n", 47 | "\n", 48 | "We want to write the Benders decomposition algorithm for the problem above. Consider the polyhedron $\\{u \\in \\mathbb{R}^m| A_2^T u \\succeq 0, u \\succeq 0\\}$. Assume the set of vertices and extreme rays of the polyhedron is denoted by $P$ and $Q$ respectively. Assume on the $k$th iteration the subset of vertices of the polyhedron mentioned is denoted by $T(k)$ and the subset of extreme rays are denoted by $Q(k)$, which will be generated by the Benders decomposition problem below." 49 | ] 50 | }, 51 | { 52 | "cell_type": "markdown", 53 | "metadata": {}, 54 | "source": [ 55 | "### Benders decomposition algorithm\n", 56 | "---------------------------------------\n", 57 | "\n", 58 | "**Step 1 (Initialization)**
\n", 59 | "We start with $T(1)=Q(1)=\\emptyset$. Let $f_m^{(1)}$ be arbitrarily large and $x^{(1)}$ be any non-negative integer vector and go to Step 3." 60 | ] 61 | }, 62 | { 63 | "cell_type": "markdown", 64 | "metadata": {}, 65 | "source": [ 66 | "** Step 2 (Solving the master problem)**
\n", 67 | "Solve the master problem:\n", 68 | "\n", 69 | "\n", 70 | "\\begin{align}\n", 71 | "& f_\\text{m}^{(k)}= \\\\\n", 72 | "&\\text{maximize} &&\\quad t \\\\\n", 73 | "&\\text{subject to} \\quad &&\\forall \\bar{u} \\in T(k) \\qquad t + (A_1^T \\bar{u} - c_1)^T x \\leq b^T \\bar{u} \\\\\n", 74 | "& && \\forall \\bar{y} \\in Q(k) \\qquad (A_1 ^T \\bar{y})^T x \\leq b^T \\bar{y} \\\\\n", 75 | "& && \\qquad \\qquad \\qquad \\; x \\succeq 0, x \\in \\mathbb{Z}^n\n", 76 | "\\end{align}\n", 77 | "\n", 78 | "Let the maximizer corresponding to the objective value $f_\\text{m}^{(k)}$ be denoted by $x^{(k)}$. Now there are three possibilities:\n", 79 | "\n", 80 | "- If $f_\\text{m}^{(k)}=-\\infty$, i.e., the master problem is infeasible, then the original proble is infeasible and sadly, we are done.\n", 81 | "\n", 82 | "- If $f_\\text{m}^{(k)}=\\infty$, i.e. the master problem is unbounded above, then we take $f_\\text{m}^{(k)}$ to be arbitrarily large and $x^{(k)}$ to be a corresponding feasible solution. Go to Step 3\n", 83 | "\n", 84 | "- If $f_\\text{m}^{(k)}$ is finite, then we collect $t^{(k)}$ and $x^{(k)}$ and go to Step 3.\n" 85 | ] 86 | }, 87 | { 88 | "cell_type": "markdown", 89 | "metadata": {}, 90 | "source": [ 91 | "** Step 3 (Solving the subproblem and add Benders cut when needed) **
\n", 92 | "Solve the subproblem\n", 93 | "\n", 94 | "\\begin{align}\n", 95 | " f_s(x^{(k)})= \\\\\n", 96 | " c_1^T x^{(k)} + & \\text{minimize} && (b-A_1 x^{(k)})^T u \\\\\n", 97 | " & \\text{subject to} && A_2^T u \\succeq c_2 \\\\\n", 98 | " & && u \\succeq 0, u \\in \\mathbb{R}^m\n", 99 | "\\end{align}\n", 100 | "\n", 101 | "Let the minimizer corresponding to the objective value $f_s(x^{(k)})$ be denoted by $u^{(k)}$. There are three possibilities:\n", 102 | "\n", 103 | "- If $f_s(x^{(k)}) = \\infty$, the original problem is either infeasible or unbounded. We quit from Benders algorithm and use special purpose algorithm to find a feasible solution if there exists one.\n", 104 | "\n", 105 | "- If $f_s(x^{(k)}) = - \\infty$, we arrive at an extreme ray $y^{(k)}$. We add the Benders cut corresponding to this extreme ray $(A_1 ^T y^{(k)})^T x \\leq b^T y^{(k)}$ to the master problem i.e., $Q(k+1):= Q(k) \\cup \\{y^{(k)}\\}$. Take $k:=k+1$ and go to Step 3.\n", 106 | "\n", 107 | "- If $f_s(x^{(k)})$ is finite, then\n", 108 | "\n", 109 | " * If $f_s(x^{(k)})=f_m^{(k)}$ we arrive at the optimal solution. The optimum objective value of the original problem is $f_s(x^{(k)})=f_m^{(k)}$, an optimal $x$ is $x^{(k)}$ and an optimal $v$ is the dual values for the second constraints of the subproblem. We are happily done!\n", 110 | " \n", 111 | " * If $f_s(x^{(k)}) < f_m^{(k)}$ we get an suboptimal vertex $u^{(k)}$. We add the corresponding Benders cut $u_0 + (A_1^T u^{(k)} - c_1)^T x \\leq b^T u^{(k)}$ to the master problem, i.e., $T(k+1) := T(k) \\cup \\{u^{(k)}\\}$. Take $k:=k+1$ and go to Step 3.\n", 112 | "\n" 113 | ] 114 | }, 115 | { 116 | "cell_type": "markdown", 117 | "metadata": {}, 118 | "source": [ 119 | "## Data for the problem\n", 120 | "\n", 121 | "The input data is from page 139, Integer programming by Garfinkel and Nemhauser, 1972." 122 | ] 123 | }, 124 | { 125 | "cell_type": "code", 126 | "execution_count": 15, 127 | "metadata": { 128 | "collapsed": false 129 | }, 130 | "outputs": [ 131 | { 132 | "data": { 133 | "text/plain": [ 134 | "1000" 135 | ] 136 | }, 137 | "execution_count": 15, 138 | "metadata": {}, 139 | "output_type": "execute_result" 140 | } 141 | ], 142 | "source": [ 143 | "# Data for the problem\n", 144 | "#---------------------\n", 145 | "c1=[-1;-4]\n", 146 | "c2=[-2; -3]\n", 147 | "dimX=length(c1)\n", 148 | "dimU=length(c2)\n", 149 | "b=[-2;-3]\n", 150 | "A1=[\n", 151 | " 1 -3;\n", 152 | " -1 -3\n", 153 | " ]\n", 154 | "A2=[\n", 155 | " 1 -2;\n", 156 | " -1 -1\n", 157 | " ]\n", 158 | "M=1000" 159 | ] 160 | }, 161 | { 162 | "cell_type": "markdown", 163 | "metadata": {}, 164 | "source": [ 165 | "## How to implement the Benders decomposition algorithm in JuMP\n", 166 | "\n", 167 | "There are two ways we can implement Benders decomposition in JuMP:\n", 168 | "\n", 169 | "- *Classical approach:* Adding the Benders cuts in a loop, and\n", 170 | "- *Modern approach:* Adding the Benders cuts as lazy constraints.\n", 171 | "\n", 172 | "The classical approach might be inferior to the modern one, as the solver\n", 173 | "- might revisit previously eliminated solution, and\n", 174 | "- might discard the optimal solution to the original problem in favor of a better but ultimately infeasible solution to the relaxed one.\n", 175 | "\n", 176 | "For more details on the comparison between the two approaches, see [Paul Rubin's blog on Benders Decomposition](http://orinanobworld.blogspot.ca/2011/10/benders-decomposition-then-and-now.html)." 177 | ] 178 | }, 179 | { 180 | "cell_type": "markdown", 181 | "metadata": {}, 182 | "source": [ 183 | "## Classical approach: adding the Benders cuts in a loop" 184 | ] 185 | }, 186 | { 187 | "cell_type": "markdown", 188 | "metadata": {}, 189 | "source": [ 190 | "Let's describe the master problem first. Note that there are no constraints, which we will added later using Benders decomposition." 191 | ] 192 | }, 193 | { 194 | "cell_type": "code", 195 | "execution_count": 16, 196 | "metadata": { 197 | "collapsed": false 198 | }, 199 | "outputs": [ 200 | { 201 | "name": "stdout", 202 | "output_type": "stream", 203 | "text": [ 204 | "A JuMP Model" 205 | ] 206 | } 207 | ], 208 | "source": [ 209 | "# Loading the necessary packages\n", 210 | "#-------------------------------\n", 211 | "using JuMP \n", 212 | "using GLPK\n", 213 | "using MathOptInterface\n", 214 | "const MOI = MathOptInterface\n", 215 | "\n", 216 | "# Master Problem Description\n", 217 | "# --------------------------\n", 218 | "\n", 219 | "masterProblemModel = Model(optimizer=GLPK.GLPKOptimizerMIP()) \n", 220 | "\n", 221 | "# Variable Definition \n", 222 | "# ----------------------------------------------------------------\n", 223 | "@variable(masterProblemModel, 0<= x[1:dimX]<= 1e6 , Int) \n", 224 | "@variable(masterProblemModel, t<=1e6)\n", 225 | "\n", 226 | "# Objective Setting\n", 227 | "# -----------------\n", 228 | "@objective(masterProblemModel, Max, t)\n", 229 | "iC=1 # iC is the iteration counter \n", 230 | "\n", 231 | "print(masterProblemModel)" 232 | ] 233 | }, 234 | { 235 | "cell_type": "markdown", 236 | "metadata": {}, 237 | "source": [ 238 | "Here is the loop that checks the status of the master problem and the subproblem and then adds necessary Benders cuts accordingly." 239 | ] 240 | }, 241 | { 242 | "cell_type": "code", 243 | "execution_count": 17, 244 | "metadata": { 245 | "collapsed": false 246 | }, 247 | "outputs": [ 248 | { 249 | "name": "stdout", 250 | "output_type": "stream", 251 | "text": [ 252 | "\n", 253 | "-----------------------\n", 254 | "Iteration number = 1\n", 255 | "-----------------------\n", 256 | "\n", 257 | "The current master problem is\n", 258 | "A JuMP ModelStatus of the master problem isSuccess\n", 259 | "with fmCurrent = 1.0e6\n", 260 | "xCurrent = [0.0, 0.0]\n", 261 | "\n", 262 | "The current subproblem model is \n", 263 | "A JuMP ModelStatus of the subproblem is Success\n", 264 | "with fsxCurrent= -7.666666666666667\n", 265 | "and fmCurrent= 1.0e6\n", 266 | "\n", 267 | "There is a suboptimal vertex, add the corresponding constraint\n", 268 | "t + [-1.0, -4.0]ᵀ x <= -7.666666666666667\n", 269 | "\n", 270 | "-----------------------\n", 271 | "Iteration number = 2\n", 272 | "-----------------------\n", 273 | "\n", 274 | "The current master problem is\n", 275 | "A JuMP ModelStatus of the master problem isSuccess\n", 276 | "with fmCurrent = 1.0e6\n", 277 | "xCurrent = [0.0, 250002.0]\n", 278 | "\n", 279 | "The current subproblem model is \n", 280 | "A JuMP ModelStatus of the subproblem is Success\n", 281 | "with fsxCurrent= -1.000008e6\n", 282 | "and fmCurrent= 1.0e6\n", 283 | "\n", 284 | "There is a suboptimal vertex, add the corresponding constraint\n", 285 | "t + [1.0, 4.0]ᵀ x <= 0.0\n", 286 | "\n", 287 | "-----------------------\n", 288 | "Iteration number = 3\n", 289 | "-----------------------\n", 290 | "\n", 291 | "The current master problem is\n", 292 | "A JuMP ModelStatus of the master problem isSuccess\n", 293 | "with fmCurrent = -4.0\n", 294 | "xCurrent = [4.0, 0.0]\n", 295 | "\n", 296 | "The current subproblem model is \n", 297 | "A JuMP ModelStatus of the subproblem is Success\n", 298 | "with fsxCurrent= -13.0\n", 299 | "and fmCurrent= -4.0\n", 300 | "\n", 301 | "There is a suboptimal vertex, add the corresponding constraint\n", 302 | "t + [2.5, -0.5]ᵀ x <= -3.0\n", 303 | "\n", 304 | "-----------------------\n", 305 | "Iteration number = 4\n", 306 | "-----------------------\n", 307 | "\n", 308 | "The current master problem is\n", 309 | "A JuMP ModelStatus of the master problem isSuccess\n", 310 | "with fmCurrent = -4.0\n", 311 | "xCurrent = [0.0, 1.0]\n", 312 | "\n", 313 | "The current subproblem model is \n", 314 | "A JuMP ModelStatus of the subproblem is Success\n", 315 | "with fsxCurrent= -4.0\n", 316 | "and fmCurrent= -4.0\n", 317 | "\n", 318 | "################################################\n", 319 | "Optimal solution of the original problem found\n", 320 | "The optimal objective value t is -4.0\n", 321 | "The optimal x is [0.0, 1.0]\n", 322 | "The optimal v is [0.0, 0.0]\n", 323 | "################################################\n", 324 | "\n" 325 | ] 326 | } 327 | ], 328 | "source": [ 329 | "# Trying the entire benders decomposition in one step\n", 330 | "while(true)\n", 331 | " println(\"\\n-----------------------\")\n", 332 | " println(\"Iteration number = \", iC)\n", 333 | " println(\"-----------------------\\n\")\n", 334 | " println(\"The current master problem is\")\n", 335 | " print(masterProblemModel)\n", 336 | " \n", 337 | " JuMP.optimize(masterProblemModel)\n", 338 | " t_status = JuMP.terminationstatus(masterProblemModel)# == MOI.Success\n", 339 | " p_status = JuMP.primalstatus(masterProblemModel)# == MOI.FeasiblePoint\n", 340 | " \n", 341 | " if p_status == MOI.InfeasiblePoint\n", 342 | " println(\"The problem is infeasible :-(\")\n", 343 | " break\n", 344 | " end\n", 345 | "\n", 346 | " if t_status == MOI.InfeasibleOrUnbounded\n", 347 | " fmCurrent = M\n", 348 | " xCurrent=M*ones(dimX)\n", 349 | " end\n", 350 | "\n", 351 | "\n", 352 | " if p_status == MOI.FeasiblePoint\n", 353 | " fmCurrent = JuMP.resultvalue(t)\n", 354 | " xCurrent=Float64[]\n", 355 | " for i in 1:dimX\n", 356 | " push!(xCurrent, JuMP.resultvalue(x[i]))\n", 357 | " end\n", 358 | " end\n", 359 | "\n", 360 | " println(\"Status of the master problem is\", t_status, \n", 361 | " \"\\nwith fmCurrent = \", fmCurrent, \n", 362 | " \"\\nxCurrent = \", xCurrent)\n", 363 | "\n", 364 | "\n", 365 | " subProblemModel = Model(optimizer=GLPK.GLPKOptimizerLP())\n", 366 | "\n", 367 | " cSub=b-A1*xCurrent\n", 368 | "\n", 369 | " @variable(subProblemModel, u[1:dimU]>=0)\n", 370 | "\n", 371 | "\n", 372 | " @constraint(subProblemModel, constrRefSubProblem[j=1:size(A2,2)],sum(A2[i,j]*u[i] for i in 1:size(A2,1))>=c2[j])\n", 373 | " # The second argument of @constraint macro, constrRefSubProblem[j=1:size(A2,2)] means that the j-th constraint is\n", 374 | " # referenced by constrRefSubProblem[j]. \n", 375 | "\n", 376 | " \n", 377 | " @objective(subProblemModel, Min, dot(c1, xCurrent) + sum(cSub[i]*u[i] for i in 1:dimU))\n", 378 | "\n", 379 | " print(\"\\nThe current subproblem model is \\n\", subProblemModel)\n", 380 | "\n", 381 | " JuMP.optimize(subProblemModel) \n", 382 | " t_status_sub = JuMP.terminationstatus(subProblemModel)# == MOI.Success\n", 383 | " p_status_sub = JuMP.primalstatus(subProblemModel)# == MOI.FeasiblePoint\n", 384 | "\n", 385 | " fsxCurrent = JuMP.objectivevalue(subProblemModel) \n", 386 | "\n", 387 | " uCurrent = Float64[]\n", 388 | "\n", 389 | " for i in 1:dimU\n", 390 | " push!(uCurrent, JuMP.resultvalue(u[i]))\n", 391 | " end\n", 392 | "\n", 393 | " γ=dot(b,uCurrent)\n", 394 | "\n", 395 | " println(\"Status of the subproblem is \", t_status_sub, \n", 396 | " \"\\nwith fsxCurrent= \", fsxCurrent, \n", 397 | " \"\\nand fmCurrent= \", fmCurrent) \n", 398 | " \n", 399 | " if p_status_sub == MOI.FeasiblePoint && fsxCurrent == fmCurrent # we are done\n", 400 | " println(\"\\n################################################\")\n", 401 | " println(\"Optimal solution of the original problem found\")\n", 402 | " println(\"The optimal objective value t is \", fmCurrent)\n", 403 | " println(\"The optimal x is \", xCurrent)\n", 404 | " println(\"The optimal v is \", JuMP.resultdual.(constrRefSubProblem))\n", 405 | " println(\"################################################\\n\")\n", 406 | " break\n", 407 | " end \n", 408 | " \n", 409 | " if p_status_sub == MOI.FeasiblePoint && fsxCurrent < fmCurrent \n", 410 | " println(\"\\nThere is a suboptimal vertex, add the corresponding constraint\")\n", 411 | " cv= A1'*uCurrent - c1\n", 412 | " @constraint(masterProblemModel, t+sum(cv[i]*x[i] for i in 1:dimX) <= γ )\n", 413 | " println(\"t + \", cv, \"ᵀ x <= \", γ)\n", 414 | " end \n", 415 | " \n", 416 | " if t_status_sub == MOI.InfeasibleOrUnbounded\n", 417 | " println(\"\\nThere is an extreme ray, adding the corresponding constraint\")\n", 418 | " ce = A1'*uCurrent\n", 419 | " @constraint(masterProblemModel, sum(ce[i]*x[i] for i in 1:dimX) <= γ)\n", 420 | " println(ce, \"ᵀ x <= \", γ)\n", 421 | " end\n", 422 | " \n", 423 | " iC=iC+1\n", 424 | " sleep(0.5)\n", 425 | " \n", 426 | "end" 427 | ] 428 | }, 429 | { 430 | "cell_type": "code", 431 | "execution_count": null, 432 | "metadata": { 433 | "collapsed": true 434 | }, 435 | "outputs": [], 436 | "source": [] 437 | }, 438 | { 439 | "cell_type": "code", 440 | "execution_count": null, 441 | "metadata": { 442 | "collapsed": true 443 | }, 444 | "outputs": [], 445 | "source": [] 446 | }, 447 | { 448 | "cell_type": "code", 449 | "execution_count": null, 450 | "metadata": { 451 | "collapsed": true 452 | }, 453 | "outputs": [], 454 | "source": [] 455 | }, 456 | { 457 | "cell_type": "code", 458 | "execution_count": null, 459 | "metadata": { 460 | "collapsed": true 461 | }, 462 | "outputs": [], 463 | "source": [] 464 | }, 465 | { 466 | "cell_type": "code", 467 | "execution_count": null, 468 | "metadata": { 469 | "collapsed": true 470 | }, 471 | "outputs": [], 472 | "source": [] 473 | }, 474 | { 475 | "cell_type": "code", 476 | "execution_count": null, 477 | "metadata": { 478 | "collapsed": true 479 | }, 480 | "outputs": [], 481 | "source": [] 482 | }, 483 | { 484 | "cell_type": "code", 485 | "execution_count": null, 486 | "metadata": { 487 | "collapsed": true 488 | }, 489 | "outputs": [], 490 | "source": [] 491 | }, 492 | { 493 | "cell_type": "code", 494 | "execution_count": null, 495 | "metadata": { 496 | "collapsed": true 497 | }, 498 | "outputs": [], 499 | "source": [] 500 | }, 501 | { 502 | "cell_type": "code", 503 | "execution_count": null, 504 | "metadata": { 505 | "collapsed": true 506 | }, 507 | "outputs": [], 508 | "source": [] 509 | }, 510 | { 511 | "cell_type": "code", 512 | "execution_count": null, 513 | "metadata": { 514 | "collapsed": true 515 | }, 516 | "outputs": [], 517 | "source": [] 518 | }, 519 | { 520 | "cell_type": "code", 521 | "execution_count": null, 522 | "metadata": { 523 | "collapsed": true 524 | }, 525 | "outputs": [], 526 | "source": [] 527 | }, 528 | { 529 | "cell_type": "code", 530 | "execution_count": null, 531 | "metadata": { 532 | "collapsed": true 533 | }, 534 | "outputs": [], 535 | "source": [] 536 | }, 537 | { 538 | "cell_type": "code", 539 | "execution_count": null, 540 | "metadata": { 541 | "collapsed": true 542 | }, 543 | "outputs": [], 544 | "source": [] 545 | }, 546 | { 547 | "cell_type": "code", 548 | "execution_count": null, 549 | "metadata": { 550 | "collapsed": true 551 | }, 552 | "outputs": [], 553 | "source": [] 554 | }, 555 | { 556 | "cell_type": "code", 557 | "execution_count": null, 558 | "metadata": { 559 | "collapsed": true 560 | }, 561 | "outputs": [], 562 | "source": [] 563 | }, 564 | { 565 | "cell_type": "code", 566 | "execution_count": null, 567 | "metadata": { 568 | "collapsed": true 569 | }, 570 | "outputs": [], 571 | "source": [] 572 | }, 573 | { 574 | "cell_type": "code", 575 | "execution_count": null, 576 | "metadata": { 577 | "collapsed": true 578 | }, 579 | "outputs": [], 580 | "source": [] 581 | }, 582 | { 583 | "cell_type": "markdown", 584 | "metadata": {}, 585 | "source": [ 586 | "## Modern approach: adding the Benders cuts as lazy constraints" 587 | ] 588 | }, 589 | { 590 | "cell_type": "markdown", 591 | "metadata": {}, 592 | "source": [ 593 | "### What are lazy constraints?\n", 594 | "Mixed integer programming solver works on branch-and-bound strategy. Often in a mixed integer programming problem, including all possible constraints might be too space consuming or computationally too expensive. Instead we can start with a manageable set of constraints in a comparatively smaller, hence relaxed version of the original MIP. Once a feasible integer solution is found, we can check the status of the problem by solving some subproblem. The subproblem can be derived from duality, as in our current problem and/or from logic. In the case of suboptimality, we can add lazy constraints to cut off the current feasible solution at that node of the branch-and-bound tree. \n", 595 | "\n", 596 | "Lazy constraints have the following advantages:\n", 597 | "- does not revisit previously eliminated solution, and\n", 598 | "- does not discard the optimal solution to the original problem in favor of a better but ultimately infeasible solution to the relaxed one." 599 | ] 600 | }, 601 | { 602 | "cell_type": "markdown", 603 | "metadata": {}, 604 | "source": [ 605 | "### How to add lazy constraints in Julia?\n", 606 | "\n", 607 | "Note that, in Step 3 we are solving the subproblem, checking the state of our problem and adding Benders cut if necessary. We write a function `addBendersCut(cb)` which will perform the steps. The argument of the function `cb` refers to the callback handle, which is a reference to the callback management code inside JuMP.\n", 608 | "\n", 609 | "When we add the lazy constraints we have to use the `@addLazyConstraint` macro as follows:\n", 610 | "\n", 611 | ">``@addLazyConstraint(cb, description of the constraint)\n", 612 | "``.\n", 613 | "\n", 614 | "Description of the constraint will be of the same manner as in adding a normal constraint in JuMP using `@constraint` macro. \n", 615 | "\n", 616 | "Note that we have not mentioned which model is associated with the added lazy constraints. So outside the `addBendersCut(cb)` function we invoke the command:\n", 617 | "\n", 618 | ">``addLazyCallback(name of the master problem model, addBendersCut)\n", 619 | "``\n", 620 | "\n", 621 | ", which tells JuMP to add the lazy constraints generated by the function `addBendersCut` to the master problem." 622 | ] 623 | }, 624 | { 625 | "cell_type": "code", 626 | "execution_count": null, 627 | "metadata": { 628 | "collapsed": false 629 | }, 630 | "outputs": [], 631 | "source": [ 632 | " # Loading the necessary packages\n", 633 | " #-------------------------------\n", 634 | "using JuMP \n", 635 | "#using CPLEX\n", 636 | "using Gurobi\n", 637 | "\n", 638 | " # Master Problem Description\n", 639 | " # --------------------------\n", 640 | "\n", 641 | "# Model name\n", 642 | " \n", 643 | "#masterProblemModel = Model(solver = CplexSolver())\n", 644 | "masterProblemModel = Model(optimizer = GurobiOptimizer(Heuristics=0, Cuts = 0, OutputFlag = 0)) # If we want to add Benders lazy constraints \n", 645 | "# in Gurobi, then we have to turn off Gurobi's own Cuts and Heuristics in the master problem\n", 646 | "\n", 647 | "# Variable Definition (Only CplexSolver() works properly for these)\n", 648 | "# ----------------------------------------------------------------\n", 649 | "#@variable(masterProblemModel, x[1:dimX] >=0 , Int) \n", 650 | "#@variable(masterProblemModel, t)\n", 651 | "\n", 652 | "# ***************ALTERNATIVE VARIABLE DEFINITION FOR GUROBI************\n", 653 | "#If we replace the two lines above with the follwoing:\n", 654 | "@variable(masterProblemModel, 0<= x[1:dimX] <= 1e6 , Int)\n", 655 | "@variable(masterProblemModel, t <= 1e6)\n", 656 | "# then all the solvers give the expected solution\n", 657 | "#**********************************************************************\n", 658 | "\n", 659 | "# Objective Setting\n", 660 | "# -----------------\n", 661 | "@objective(masterProblemModel, Max, t)\n", 662 | "\n", 663 | "print(masterProblemModel)\n", 664 | "\n", 665 | "stringOfBenderCuts=AbstractString[] # this is an array of strings which will contain all the\n", 666 | "# Benders cuts added to be displayed later\n", 667 | "\n", 668 | "# There are no constraints when we start, so we will add all the constraints in the\n", 669 | "# form of Benders cuts lazily" 670 | ] 671 | }, 672 | { 673 | "cell_type": "code", 674 | "execution_count": null, 675 | "metadata": { 676 | "collapsed": false 677 | }, 678 | "outputs": [], 679 | "source": [ 680 | "function addBendersCut(cb)\n", 681 | " #***************************************************************************\n", 682 | " # First we store the master problem solution in conventional data structures\n", 683 | " println(\"----------------------------\")\n", 684 | " println(\"ITERATION NUMBER = \", length(stringOfBenderCuts)+1)\n", 685 | " println(\"---------------------------\\n\")\n", 686 | " \n", 687 | " fmCurrent = getvalue(t)\n", 688 | " xCurrent=Float64[]\n", 689 | " for i in 1:dimX\n", 690 | " push!(xCurrent, JuMP.resultvalue(x[i]))\n", 691 | " end\n", 692 | " \n", 693 | " # Display the current solution of the master problem\n", 694 | " println(\"MASTERPROBLEM INFORMATION\")\n", 695 | " println(\"-------------------------\")\n", 696 | " println(\"The master problem that was solved was:\")\n", 697 | " print(masterProblemModel)\n", 698 | " println(\"with \", length(stringOfBenderCuts), \" added lazy constraints\")\n", 699 | " println(stringOfBenderCuts)\n", 700 | " println(\"Current Value of x is: \", xCurrent)\n", 701 | " println(\"Current objective value of master problem, fmCurrent is: \", fmCurrent)\n", 702 | " println(\"\\n\")\n", 703 | " \n", 704 | " #************************************************************************\n", 705 | " \n", 706 | " # ========================================================================\n", 707 | " # Now we solve the subproblem\n", 708 | "\n", 709 | " # subProblemModel=Model(solver=CplexSolver())\n", 710 | "\n", 711 | " subProblemModel = Model(optimizer=GurobiOptimizer(Presolve=0, OutputFlag = 0))\n", 712 | " \n", 713 | " cSub=b-A1*xCurrent\n", 714 | " \n", 715 | " @variable(subProblemModel, u[1:dimU]>=0)\n", 716 | " \n", 717 | "\n", 718 | " @constraint(subProblemModel, constrRefSubProblem[j=1:size(A2,2)], sum(A2[i,j]*u[i] for i in 1:size(A2,1))>=c2[j])\n", 719 | "\n", 720 | " \n", 721 | " @objective(subProblemModel, Min, dot(c1, xCurrent) + sum(cSub[i]*u[i] for i in 1:dimU))\n", 722 | " \n", 723 | " println(\"The subproblem is being solved\")\n", 724 | " \n", 725 | " statusSubProblem = JuMP.optimize(subProblemModel) \n", 726 | "\n", 727 | " # We store the results achieved from the subproblem in conventional data structures \n", 728 | " \n", 729 | " fsxCurrent = JuMP.objectivevalue(subProblemModel) \n", 730 | "\n", 731 | " uCurrent = Float64[]\n", 732 | " for i in 1:dimU\n", 733 | " push!(uCurrent, JuMP.resultvalue(u[i]))\n", 734 | " end\n", 735 | " \n", 736 | " # Display the solution corresponding to the subproblem\n", 737 | " \n", 738 | " println(\"SUBPROBLEM INFORMATION\")\n", 739 | " println(\"----------------------\")\n", 740 | " println(\"The subproblem that was solved was: \")\n", 741 | " print(subProblemModel)\n", 742 | " println(\"Current status of the subproblem is \", statusSubProblem)\n", 743 | " println(\"Current Value of u is: \", uCurrent) # JuMP will return an extreme ray\n", 744 | " # automatically (if the solver supports it), so we do not need to change the syntax\n", 745 | " println(\"Current Value of fs(xCurrent) is: \", fsxCurrent)\n", 746 | " println(\"\\n\")\n", 747 | " \n", 748 | " # ==========================================================================\n", 749 | " \n", 750 | " \n", 751 | " \n", 752 | " # ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n", 753 | " # Now we check the status of the algorithm and add Benders cut when necessary\n", 754 | " γ=dot(b,uCurrent)\n", 755 | " \n", 756 | "\n", 757 | " \n", 758 | " if statusSubProblem == :Optimal && fsxCurrent==fmCurrent # we are done\n", 759 | " println(\"OPTIMAL SOLUTION OF THE ORIGINAL PROBLEM FOUND :-)\")\n", 760 | " println(\"The optimal objective value t is \", fmCurrent)\n", 761 | " println(\"The optimal x is \", xCurrent)\n", 762 | " println(\"The optimal v is \", JuMP.resultdual(constrRefSubProblem))\n", 763 | " println(\"\\n\")\n", 764 | " return\n", 765 | " end \n", 766 | "\n", 767 | " println(\"-------------------ADDING LAZY CONSTRAINT----------------\") \n", 768 | " if statusSubProblem == :Optimal && fsxCurrent < fmCurrent \n", 769 | " println(\"\\nThere is a suboptimal vertex, add the corresponding constraint\")\n", 770 | " cv= A1'*uCurrent - c1\n", 771 | " @lazyconstraint(cb, t+sum(cv[i]*x[i] for i in 1:dimX) <= γ)\n", 772 | " println(\"t + \", cv, \"ᵀ x <= \", γ)\n", 773 | " push!(stringOfBenderCuts, string(\"t+\", cv, \"'x <=\", γ))\n", 774 | " end\n", 775 | " \n", 776 | " if statusSubProblem == :Unbounded \n", 777 | " println(\"\\nThere is an extreme ray, adding the corresponding constraint\")\n", 778 | " ce = A1'*uCurrent\n", 779 | " @lazyconstraint(cb, sum(ce[i]*x[i] for i in 1:dimX) <= γ)\n", 780 | " println(ce, \"x <= \", γ)\n", 781 | " push!(stringOfBenderCuts, string(ce, \"ᵀ x <= \", γ))\n", 782 | " end\n", 783 | " println(\"\\n\") \n", 784 | " #~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n", 785 | " \n", 786 | "end" 787 | ] 788 | }, 789 | { 790 | "cell_type": "markdown", 791 | "metadata": {}, 792 | "source": [ 793 | "Now we tell the solver to use the callback function and solve the problem." 794 | ] 795 | }, 796 | { 797 | "cell_type": "code", 798 | "execution_count": null, 799 | "metadata": { 800 | "collapsed": false 801 | }, 802 | "outputs": [], 803 | "source": [ 804 | "addlazycallback(masterProblemModel, addBendersCut) # Telling the solver to use the \n", 805 | "# callback function" 806 | ] 807 | }, 808 | { 809 | "cell_type": "code", 810 | "execution_count": null, 811 | "metadata": { 812 | "collapsed": false, 813 | "scrolled": false 814 | }, 815 | "outputs": [], 816 | "source": [ 817 | "JuMP.optimize(masterProblemModel)" 818 | ] 819 | } 820 | ], 821 | "metadata": { 822 | "kernelspec": { 823 | "display_name": "Julia 0.6.0", 824 | "language": "julia", 825 | "name": "julia-0.6" 826 | }, 827 | "language_info": { 828 | "file_extension": ".jl", 829 | "mimetype": "application/julia", 830 | "name": "julia", 831 | "version": "0.6.0" 832 | } 833 | }, 834 | "nbformat": 4, 835 | "nbformat_minor": 0 836 | } 837 | -------------------------------------------------------------------------------- /notebooks/Chiwei Yan - Cutting Stock.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "**Description**: Shows how to implement column generation in JuMP for solving the cutting stock problem.\n", 8 | "\n", 9 | "**Author**: Chiwei Yan\n", 10 | "\n", 11 | "**License**: \"Creative
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License." 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "metadata": { 17 | "slideshow": { 18 | "slide_type": "slide" 19 | } 20 | }, 21 | "source": [ 22 | "## Cutting Stock Problem Using Julia/JuMP" 23 | ] 24 | }, 25 | { 26 | "cell_type": "markdown", 27 | "metadata": { 28 | "slideshow": { 29 | "slide_type": "slide" 30 | } 31 | }, 32 | "source": [ 33 | "In this notebook, we deploy column generation to solve the cutting stock problem using Julia/JuMP. The origin of the cutting stock problem is in the paper industry. Given paper rolls of fixed width and a set of orders for rolls of smaller widths, the objective of the cutting stock problem is to determine how to cut the rolls into smaller widths to fulfill the orders in such a way as to minimize the amount of scrap. The cutting stock problem example we use in this notebook is from *Linear Programming* by Vasek Chvatal, 1983.\n", 34 | "\n", 35 | "Suppose that rolls are produced in a uniform width of 100 inches and that orders can be placed for rolls of widths 14 inches, 31 inches, 36 inches, and 45 inches. The company has received the following orders:\n", 36 | "\n", 37 | "\\begin{array}{|c|c|}\n", 38 | "\\hline \\textrm{Order Width} & \\textrm{Quantity} \\\\\\hline\n", 39 | " 14 & 211 \\\\\\hline\n", 40 | " 31 & 395 \\\\\\hline\n", 41 | " 36 & 610 \\\\\\hline\n", 42 | " 45 & 97 \\\\\\hline\n", 43 | "\\end{array}\n", 44 | "\n", 45 | "A single 100 inch roll can be cut into one or more of the order widths. For example, one roll could be cut into two rolls of 45 inches with a 10 inch roll of scrap.\n", 46 | "\n", 47 | "\n", 48 | "\n", 49 | "Or a roll could be cut into a roll of 45 inches, a roll of 31 inches, and a roll of 14 inches with no scrap. Each such possible combination is called a pattern. For this example, there are 37 different patterns. Determining how many of each pattern to cut to satisfy the customer orders while minimizing the scrap is too difficult to do by hand. Instead the problem can be formulated as an optimization problem, specifically an integer linear program.\n", 50 | "\n", 51 | "### Mathematical Formulation ###\n", 52 | "\n", 53 | "**Sets**\n", 54 | "- $I$ = set of order widths\n", 55 | "- $J$ = set of patterns\n", 56 | "\n", 57 | "**Parameters**\n", 58 | "- $a_{ij}$ = number of rolls of width $i$ cut in pattern $j$\n", 59 | "- $b_i$ = demand for order width $i$\n", 60 | "\n", 61 | "**Decision Variables**\n", 62 | "- $x_j$ = number of rolls cut using pattern $j$\n", 63 | "\n", 64 | "The objective of the cutting stock problem is to minimize the number of rolls cut subject to cutting enough rolls to satisfy the customer orders. Using the notation above, the problem can be formulated as follows:\n", 65 | "\n", 66 | "$$\\begin{align}\n", 67 | "\\nonumber\n", 68 | "\\min\\qquad &\\sum_{j\\in J}x_j \\\\ \\nonumber\n", 69 | "\\textrm{s.t.}\\qquad &\\sum_{j\\in J}a_{ij}x_j\\ge b_i,~\\forall i\\in I \\\\ \\nonumber\n", 70 | "&x_j\\in\\textrm{integer},~\\forall j\\in J\n", 71 | "\\end{align}$$" 72 | ] 73 | }, 74 | { 75 | "cell_type": "markdown", 76 | "metadata": {}, 77 | "source": [ 78 | "###Let's see how to formulate this problem in JuMP" 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": 1, 84 | "metadata": { 85 | "collapsed": false 86 | }, 87 | "outputs": [ 88 | { 89 | "data": { 90 | "text/latex": [ 91 | "$$ \\begin{alignat*}{1}\\min\\quad & 0\\\\\n", 92 | "\\text{Subject to} \\quad\\end{alignat*}\n", 93 | " $$" 94 | ], 95 | "text/plain": [ 96 | "Feasibility problem with:\n", 97 | " * 0 linear constraints\n", 98 | " * 0 variables\n", 99 | "Solver set to Default" 100 | ] 101 | }, 102 | "execution_count": 1, 103 | "metadata": {}, 104 | "output_type": "execute_result" 105 | } 106 | ], 107 | "source": [ 108 | "#import necessary packages and define model\n", 109 | "# Load JuMP\n", 110 | "using JuMP\n", 111 | "using MathOptInterface\n", 112 | "# Load solver package\n", 113 | "using MathOptInterfaceXpress\n", 114 | "# shortcuts\n", 115 | "const MOI = MathOptInterface\n", 116 | "const MOIU = MathOptInterface.Utilities\n", 117 | "\n", 118 | "master = Model(optimizer = XpressOptimizer())\n", 119 | "\n", 120 | "#If Gurobi is installed (requires license), you may uncomment the code below to switch solvers\n", 121 | "#using Gurobi\n", 122 | "#master = Model(solver=GurobiSolver(Method=0)) # Switch LP algorithm to Primal Simplex, in order to enjoy warm start" 123 | ] 124 | }, 125 | { 126 | "cell_type": "markdown", 127 | "metadata": {}, 128 | "source": [ 129 | "We are now going to initialize a ***\"restricted master problem\"*** with only two variables, corresponding two cutting patterns: \n", 130 | "- width (14,31,36,45), quantity (1,1,0,1), denoted as $x_1$\n", 131 | "- width (14,31,36,45), quantity (0,0,2,0), denoted as $x_2$\n", 132 | "\n", 133 | "**\\[Recall 1\\]** what's the meaning of each variable? _Number of paper rolls cut using this pattern._\n", 134 | "\n", 135 | "**\\[Recall 2\\]** How should the formulation of the restricted master problem look like?\n", 136 | "\n", 137 | "$$\n", 138 | "\\begin{align}\n", 139 | "\\nonumber\\min\\qquad\\qquad\\quad &x_1+x_2 \\\\\n", 140 | "s.t.\\qquad\\left( \\begin{array}{c}\n", 141 | "1 \\\\\n", 142 | "1 \\\\\n", 143 | "0 \\\\\n", 144 | "1 \\end{array} \\right)&x_1+\\left( \\begin{array}{c}\n", 145 | "0 \\\\\n", 146 | "0 \\\\\n", 147 | "2 \\\\\n", 148 | "0 \\end{array} \\right)x_2 \\ge \\left( \\begin{array}{c}\n", 149 | "211 \\\\\n", 150 | "395 \\\\\n", 151 | "610 \\\\\n", 152 | "97 \\end{array} \\right) \\\\ \n", 153 | "&x_1,x_2\\ge0\n", 154 | "\\end{align}$$" 155 | ] 156 | }, 157 | { 158 | "cell_type": "code", 159 | "execution_count": 2, 160 | "metadata": { 161 | "collapsed": false 162 | }, 163 | "outputs": [ 164 | { 165 | "data": { 166 | "text/latex": [ 167 | "$$ \\begin{alignat*}{1}\\min\\quad & x_{1} + x_{2}\\\\\n", 168 | "\\text{Subject to} \\quad & x_{1} \\geq 211\\\\\n", 169 | " & x_{1} \\geq 395\\\\\n", 170 | " & 2 x_{2} \\geq 610\\\\\n", 171 | " & x_{1} \\geq 97\\\\\n", 172 | " & x_{i} \\geq 0 \\quad\\forall i \\in \\{1,2\\}\\\\\n", 173 | "\\end{alignat*}\n", 174 | " $$" 175 | ], 176 | "text/plain": [ 177 | "Minimization problem with:\n", 178 | " * 4 linear constraints\n", 179 | " * 2 variables\n", 180 | "Solver set to Default" 181 | ] 182 | }, 183 | "execution_count": 2, 184 | "metadata": {}, 185 | "output_type": "execute_result" 186 | } 187 | ], 188 | "source": [ 189 | "#define initial variables\n", 190 | "@variable(master, x[1:2] >= 0)\n", 191 | "\n", 192 | "#width\n", 193 | "w=[14 31 36 45]\n", 194 | "\n", 195 | "#constraint coefficient for initial variables\n", 196 | "A=[1 0; 1 0; 0 2; 1 0]\n", 197 | "b=[211; 395; 610; 97]\n", 198 | "\n", 199 | "\n", 200 | "#define constraint references\n", 201 | "myCons = Any[]\n", 202 | "\n", 203 | "#define constraints\n", 204 | "for i=1:4\n", 205 | " myCon = @constraint(master, dot(x, vec(A[i,:]))>=b[i])\n", 206 | " push!(myCons, myCon)\n", 207 | "end\n", 208 | "\n", 209 | "#define objective\n", 210 | "@objective(master, Min, sum(x))\n", 211 | "\n", 212 | "master" 213 | ] 214 | }, 215 | { 216 | "cell_type": "code", 217 | "execution_count": 3, 218 | "metadata": { 219 | "collapsed": false 220 | }, 221 | "outputs": [ 222 | { 223 | "name": "stdout", 224 | "output_type": "stream", 225 | "text": [ 226 | "\n", 227 | "Optimal Solution is:\n", 228 | "\n", 229 | "width: [14 31 36 45]\n", 230 | "Cutting Pattern: [1,1,0,1], Number of Paper Rolls Cut Using this Pattern: 395.0\n", 231 | "Cutting Pattern: [0,0,2,0], Number of Paper Rolls Cut Using this Pattern: 305.0\n" 232 | ] 233 | } 234 | ], 235 | "source": [ 236 | "JuMP.optimize(master)\n", 237 | "status = JuMP.terminationstatus(master)\n", 238 | "JuMP.resultvalue.(x)\n", 239 | "\n", 240 | "#get the optimal solution\n", 241 | "println(\"\\nOptimal Solution is:\\n\")\n", 242 | "\n", 243 | "println(\"width: \", w)\n", 244 | "\n", 245 | "epsilon=1e-6\n", 246 | "\n", 247 | "for i=1:size(A,2)\n", 248 | " \n", 249 | " if JuMP.resultvalue(x[i])>epsilon \n", 250 | " println(\"Cutting Pattern: \", A[:,i], \", Number of Paper Rolls Cut Using this Pattern: \", JuMP.resultvalue(x[i]))\n", 251 | " end\n", 252 | "end" 253 | ] 254 | }, 255 | { 256 | "cell_type": "markdown", 257 | "metadata": {}, 258 | "source": [ 259 | "**\\[Result Analysis\\]**\n", 260 | "\n", 261 | "The minimal number of paper rolls is 700. \n", 262 | "\n", 263 | "Clearly this is not the best we can do, because we are not considering all possible feasible patterns.\n", 264 | "\n", 265 | "Let's now generate some new patterns based on the value of reduced costs. Denote $r=(r_1,r_2,r_3,r_4)$ as the optimal dual price of constraints 1, 2, 3, 4. The reduced cost of a potential variable $x_k$, with cutting pattern $A_k$ can be calculated as\n", 266 | "$$rc(x_k)=1-A_k^Tr$$\n", 267 | "\n", 268 | "We want to add a potential variable $x_k$ such that $rc(x_k)<0$, this can be done by solving the following sub-problem:\n", 269 | "\n", 270 | "$$\\begin{align}\n", 271 | "z^*=\\max\\qquad &r_1a_{k,1}+r_2a_{k,2}+r_3a_{k,3}+r_4a_{k,4} \\\\\n", 272 | "s.t.\\qquad &14a_{k,1}+31a_{k,2}+36a_{k,3}+45a_{k,4}\\le 100 \\\\\n", 273 | "&a_{k,1},a_{k,2},a_{k,3},a_{k,4}\\ge0,~\\textrm{and are integers}\n", 274 | "\\end{align}$$\n", 275 | "\n", 276 | "If $z^*>1$, then $x_k$ with cutting pattern $(a_{k,1},a_{k,2},a_{k,3},a_{k,4})$ should be added to the master problem. And resolve the master problem." 277 | ] 278 | }, 279 | { 280 | "cell_type": "code", 281 | "execution_count": 4, 282 | "metadata": { 283 | "collapsed": false 284 | }, 285 | "outputs": [ 286 | { 287 | "name": "stdout", 288 | "output_type": "stream", 289 | "text": [ 290 | "width: [14,31,36,45]\n", 291 | "\n", 292 | "New Cutting Pattern: [0,3,0,0]\n" 293 | ] 294 | } 295 | ], 296 | "source": [ 297 | "r=[getDual(myCons)[1:4]]\n", 298 | "\n", 299 | "sub = Model(optimizer = XpressOptimizer()) \n", 300 | "\n", 301 | "#width\n", 302 | "w=[14,31,36,45]\n", 303 | "\n", 304 | "#define cutting pattern variables\n", 305 | "@variable(sub, a[1:4]>=0, Int)\n", 306 | "\n", 307 | "#define feasible cutting constraint\n", 308 | "@constraint(sub, dot(w,a)<=100)\n", 309 | "\n", 310 | "#define objective\n", 311 | "@objective(sub, Max, dot(r,a))\n", 312 | "\n", 313 | "sub\n", 314 | "\n", 315 | "JuMP.optimize(master)\n", 316 | "status = JuMP.terminationstatus(master)\n", 317 | "\n", 318 | "#print new cutting pattern\n", 319 | "pattern=[getValue(a)[1:4]]\n", 320 | "\n", 321 | "println(\"width: \", w)\n", 322 | "\n", 323 | "println(\"\\nNew Cutting Pattern: \", int(pattern))" 324 | ] 325 | }, 326 | { 327 | "cell_type": "markdown", 328 | "metadata": {}, 329 | "source": [ 330 | "**\\[Result Analysis\\]**\n", 331 | "\n", 332 | "The reduced cost of this variable is $(1-3)=-2<0$. Add this new variable to the ***\"restricted master problem\"***." 333 | ] 334 | }, 335 | { 336 | "cell_type": "code", 337 | "execution_count": 5, 338 | "metadata": { 339 | "collapsed": false 340 | }, 341 | "outputs": [ 342 | { 343 | "data": { 344 | "text/latex": [ 345 | "$$ \\begin{alignat*}{1}\\min\\quad & x_{1} + x_{2}\\\\\n", 346 | "\\text{Subject to} \\quad & x_{1} \\geq 211\\\\\n", 347 | " & x_{1} \\geq 395\\\\\n", 348 | " & 2 x_{2} \\geq 610\\\\\n", 349 | " & x_{1} \\geq 97\\\\\n", 350 | " & x_{i} \\geq 0 \\quad\\forall i \\in \\{1,2\\}\\\\\n", 351 | "\\end{alignat*}\n", 352 | " $$" 353 | ], 354 | "text/plain": [ 355 | "Minimization problem with:\n", 356 | " * 4 linear constraints\n", 357 | " * 2 variables\n", 358 | "Solver set to Default" 359 | ] 360 | }, 361 | "execution_count": 5, 362 | "metadata": {}, 363 | "output_type": "execute_result" 364 | } 365 | ], 366 | "source": [ 367 | "#model before adding new column\n", 368 | "master" 369 | ] 370 | }, 371 | { 372 | "cell_type": "markdown", 373 | "metadata": {}, 374 | "source": [ 375 | "JuMP supports column-wise modeling in defining variables. Think about it, when we add a new variable to the existing model, we need to know:\n", 376 | "\n", 377 | "- What's the coefficient for this new variable in the objective function?\n", 378 | "- Which constraint does this new variable appear? With what coefficient?" 379 | ] 380 | }, 381 | { 382 | "cell_type": "code", 383 | "execution_count": 6, 384 | "metadata": { 385 | "collapsed": false 386 | }, 387 | "outputs": [ 388 | { 389 | "data": { 390 | "text/latex": [ 391 | "$$ \\begin{alignat*}{1}\\min\\quad & x_{1} + x_{2} + z\\\\\n", 392 | "\\text{Subject to} \\quad & x_{1} \\geq 211\\\\\n", 393 | " & x_{1} + 2.9999999999999996 z \\geq 395\\\\\n", 394 | " & 2 x_{2} \\geq 610\\\\\n", 395 | " & x_{1} \\geq 97\\\\\n", 396 | " & x_{i} \\geq 0 \\quad\\forall i \\in \\{1,2\\}\\\\\n", 397 | " & z \\geq 0\\\\\n", 398 | "\\end{alignat*}\n", 399 | " $$" 400 | ], 401 | "text/plain": [ 402 | "Minimization problem with:\n", 403 | " * 4 linear constraints\n", 404 | " * 3 variables\n", 405 | "Solver set to Default" 406 | ] 407 | }, 408 | "execution_count": 6, 409 | "metadata": {}, 410 | "output_type": "execute_result" 411 | } 412 | ], 413 | "source": [ 414 | "#column-wise adding new variable z\n", 415 | "# TODO column addtion\n", 416 | "@variable(master, z>=0, objective=1, inconstraints=myCons, coefficients=pattern)\n", 417 | "\n", 418 | "#look at the master problem again\n", 419 | "master" 420 | ] 421 | }, 422 | { 423 | "cell_type": "code", 424 | "execution_count": 7, 425 | "metadata": { 426 | "collapsed": false 427 | }, 428 | "outputs": [ 429 | { 430 | "name": "stdout", 431 | "output_type": "stream", 432 | "text": [ 433 | "\n", 434 | "Optimal Solution is:\n", 435 | "\n", 436 | "width: [14,31,36,45]\n", 437 | "Cutting Pattern: [1,1,0,1], Number of Paper Rolls Cut Using this Pattern: 211.0\n", 438 | "Cutting Pattern: [0,0,2,0], Number of Paper Rolls Cut Using this Pattern: 305.0\n", 439 | "Cutting Pattern: [0,3,0,0], Number of Paper Rolls Cut Using this Pattern: 61.33333333333334\n" 440 | ] 441 | } 442 | ], 443 | "source": [ 444 | "#solve the master problem again\n", 445 | "JuMP.optimize(master)\n", 446 | "status = JuMP.terminationstatus(master)\n", 447 | "\n", 448 | "#get the optimal solution\n", 449 | "println(\"\\nOptimal Solution is:\\n\")\n", 450 | "\n", 451 | "println(\"width: \", w)\n", 452 | "\n", 453 | "for i=1:length(x)\n", 454 | " \n", 455 | " if JuMP.resultvalue(x[i])>epsilon\n", 456 | " println(\"Cutting Pattern: \", A[:,i], \", Number of Paper Rolls Cut Using this Pattern: \", JuMP.resultvalue(x[i]))\n", 457 | " end\n", 458 | "end\n", 459 | "\n", 460 | "\n", 461 | "if JuMP.resultvalue(z)>epsilon\n", 462 | " println(\"Cutting Pattern: \", int(pattern), \", Number of Paper Rolls Cut Using this Pattern: \", JuMP.resultvalue(z))\n", 463 | "end\n" 464 | ] 465 | }, 466 | { 467 | "cell_type": "markdown", 468 | "metadata": {}, 469 | "source": [ 470 | "**\\[Result Analysis\\]**\n", 471 | "\n", 472 | "We see that after adding a new variable, the objective value is reduced to 577.3 \n", 473 | "\n", 474 | "###Now it's time to put all pieces together###" 475 | ] 476 | }, 477 | { 478 | "cell_type": "code", 479 | "execution_count": 8, 480 | "metadata": { 481 | "collapsed": false 482 | }, 483 | "outputs": [ 484 | { 485 | "name": "stdout", 486 | "output_type": "stream", 487 | "text": [ 488 | "Iteration 1, Master Problem Objective Value:700.0\n", 489 | "\tAdd a new variable with cutting pattern: [0.0,2.9999999999999996,0.0,0.0], reduced cost: -2.0\n", 490 | "\n", 491 | "Iteration 2, Master Problem Objective Value:577.3333333333334\n", 492 | "\tAdd a new variable with cutting pattern: [7.000000000000001,0.0,0.0,0.0], reduced cost: -3.666666666666666\n", 493 | "\n", 494 | "Iteration 3, Master Problem Objective Value:517.6190476190477\n", 495 | "Presolve 0 (-1) rows, 0 (-4) columns and 0 (-4) elements\n", 496 | "Optimal - objective value 1\n", 497 | "After Postsolve, objective 1, infeasibilities - dual 0 (0), primal 0 (0)\n", 498 | "Optimal objective 1 - 0 iterations time 0.002, Presolve 0.00\n", 499 | "Cbc0045I Solution with objective value -1 saved\n", 500 | "\tAdd a new variable with cutting pattern: [2.0,0.0,2.0,0.0], reduced cost: -0.2857142857142856\n", 501 | "\n", 502 | "Iteration 4, Master Problem Objective Value:501.33333333333337\n", 503 | "Presolve 0 (-1) rows, 0 (-4) columns and 0 (-4) elements\n", 504 | "Optimal - objective value 1\n", 505 | "After Postsolve, objective 1, infeasibilities - dual 0 (0), primal 0 (0)\n", 506 | "Optimal objective 1 - 0 iterations time 0.002, Presolve 0.00\n", 507 | "Cbc0045I Solution with objective value -1 saved\n", 508 | "\tAdd a new variable with cutting pattern: [0.0,0.0,0.0,2.0], reduced cost: -0.33333333333333326\n", 509 | "\n", 510 | "Iteration 5, Master Problem Objective Value:485.1666666666667\n", 511 | "Presolve 0 (-1) rows, 0 (-4) columns and 0 (-4) elements\n", 512 | "Optimal - objective value 1\n", 513 | "After Postsolve, objective 1, infeasibilities - dual 0 (0), primal 0 (0)\n", 514 | "Optimal objective 1 - 0 iterations time 0.002, Presolve 0.00\n", 515 | "Cbc0045I Solution with objective value -1 saved\n", 516 | "Presolve 0 (-1) rows, 0 (-4) columns and 0 (-4) elements\n", 517 | "Optimal - objective value 1\n", 518 | "After Postsolve, objective 1, infeasibilities - dual 0 (0), primal 0 (0)\n", 519 | "Optimal objective 1 - 0 iterations time 0.002, Presolve 0.00\n", 520 | "Cbc0045I Solution with objective value -1 saved\n", 521 | "\tAdd a new variable with cutting pattern: [0.0,2.0,1.0,0.0], reduced cost: -0.16666666666666674\n", 522 | "\n", 523 | "Iteration 6, Master Problem Objective Value:452.25\n", 524 | "\n", 525 | "Optimal Solution is:\n", 526 | "\n", 527 | "width: [14,31,36,45]\n", 528 | "Cutting Pattern: [2,0,2,0], Number of Paper Rolls Cut Using this Pattern: 206.25\n", 529 | "Cutting Pattern: [0,0,0,2], Number of Paper Rolls Cut Using this Pattern: 48.5\n", 530 | "Cutting Pattern: [0,2,1,0], Number of Paper Rolls Cut Using this Pattern: 197.5\n", 531 | "Presolve 0 (-1) rows, 0 (-4) columns and 0 (-4) elements\n", 532 | "Optimal - objective value 1\n", 533 | "After Postsolve, objective 1, infeasibilities - dual 0 (0), primal 0 (0)\n", 534 | "Optimal objective 1 - 0 iterations time 0.002, Presolve 0.00\n", 535 | "Cbc0045I Solution with objective value -1 saved\n", 536 | "Coin0505I Presolved problem not optimal, resolve after postsolve\n", 537 | "Coin0505I Presolved problem not optimal, resolve after postsolve\n" 538 | ] 539 | } 540 | ], 541 | "source": [ 542 | "#import necessary packages and define master problem\n", 543 | "using JuMP, Cbc\n", 544 | "master = Model() \n", 545 | "\n", 546 | "#If Gurobi is installed (requires license), you may uncomment the code below to switch solvers\n", 547 | "#using Gurobi\n", 548 | "#master = Model(solver=GurobiSolver(Method=0)) # Switch LP algorithm to Primal Simplex, in order to enjoy warm start\n", 549 | "\n", 550 | "#define initial variables\n", 551 | "@variable(master, x[1:2] >= 0)\n", 552 | "\n", 553 | "#constraint coefficient for initial variables\n", 554 | "A=[1 0; 1 0; 0 2; 1 0]\n", 555 | "b=[211; 395; 610; 97]\n", 556 | "\n", 557 | "#define constraint references (why?)\n", 558 | "@constraint myCons[1:4]\n", 559 | "\n", 560 | "#define constraints\n", 561 | "for i=1:4\n", 562 | " myCons[i] = @constraint(master, dot(x, vec(A[i,:]))>=b[i])\n", 563 | "end\n", 564 | "\n", 565 | "#define objective\n", 566 | "@objective(master, Min, sum(x))\n", 567 | "\n", 568 | "#solve master problem\n", 569 | "JuMP.optimize(master)\n", 570 | "\n", 571 | "println(\"Iteration 1, Master Problem Objective Value:\", getObjectiveValue(master))\n", 572 | "\n", 573 | "#subproblem to iteratively generate new columns\n", 574 | "\n", 575 | "#get optimal dual prices from the master problem\n", 576 | "r=[JuMP.resultdual(myCons)[1:4]]\n", 577 | "\n", 578 | "sub=Model(optimizer = XpressOptimizer()) \n", 579 | "\n", 580 | "#width\n", 581 | "w=[14,31,36,45]\n", 582 | "\n", 583 | "#define cutting pattern variables\n", 584 | "@variable(sub, a[1:4]>=0, Int)\n", 585 | "\n", 586 | "#define feasible cutting constraint\n", 587 | "@constraint(sub, dot(w,a)<=100)\n", 588 | "\n", 589 | "#define objective\n", 590 | "@objective(sub, Max, dot(r,a))\n", 591 | "\n", 592 | "#solve the subproblem\n", 593 | "JuMP.optimize(sub)\n", 594 | "\n", 595 | "sub_obj=JuMP.objectivevalue(sub);\n", 596 | "\n", 597 | "epsilon=1e-6; \n", 598 | "\n", 599 | "#list of new variables\n", 600 | "newColumns=Variable[]\n", 601 | "#pattern list\n", 602 | "A_new=Float64[];\n", 603 | "\n", 604 | "iter=2\n", 605 | "\n", 606 | "while sub_obj>1+epsilon #why?\n", 607 | "\n", 608 | " #cutting pattern (constraint coefficients) for the new variable\n", 609 | " pattern=JuMP.resultvalue(a)[1:4]\n", 610 | " \n", 611 | " #column-wise adding new variable z\n", 612 | " @variable(master, z>=0, objective=1, inconstraints=myCons, coefficients=pattern)\n", 613 | " \n", 614 | " println(\"\\tAdd a new variable with cutting pattern: \", pattern, \", reduced cost: \", (1-sub_obj))\n", 615 | " \n", 616 | " #add new variable to the new variable list\n", 617 | " push!(newColumns, z)\n", 618 | " #add new cutting pattern to pattern list\n", 619 | " append!(A_new, pattern)\n", 620 | " \n", 621 | " JuMP.optimize(master)\n", 622 | " \n", 623 | " println(\"\\nIteration \",iter, \", Master Problem Objective Value:\", JuMP.objectivevalue(master))\n", 624 | " \n", 625 | " #get new optimal dual prices\n", 626 | " r=[JuMP.resultdual(myCons)[1:4]]\n", 627 | " \n", 628 | " #modify the objective of the subproblem based on new dual prices\n", 629 | " @objective(sub, Max, dot(r,a))\n", 630 | " \n", 631 | " JuMP.optimize(sub)\n", 632 | " \n", 633 | " sub_obj=JuMP.objectivevalue(sub)\n", 634 | " \n", 635 | " iter=iter+1\n", 636 | " \n", 637 | "end\n", 638 | "\n", 639 | "#print optimal solution\n", 640 | "A_new=reshape(A_new,4, convert(Int64,length(A_new)/4))\n", 641 | "\n", 642 | "println(\"\\nOptimal Solution is:\\n\")\n", 643 | "\n", 644 | "println(\"width: \", w)\n", 645 | "\n", 646 | "for i=1:length(x)\n", 647 | " \n", 648 | " if JuMP.resultvalue(x[i])>epsilon\n", 649 | " println(\"Cutting Pattern: \", A[:,i], \", Number of Paper Rolls Cut Using this Pattern: \", JuMP.resultvalue(x[i]))\n", 650 | " end\n", 651 | "end\n", 652 | "\n", 653 | "for i=1:length(newColumns)\n", 654 | " \n", 655 | " if getValue(newColumns[i])>epsilon\n", 656 | " println(\"Cutting Pattern: \", int(A_new[:,i]), \", Number of Paper Rolls Cut Using this Pattern: \", JuMP.resultvalue(newColumns[i]))\n", 657 | " end\n", 658 | "end" 659 | ] 660 | }, 661 | { 662 | "cell_type": "markdown", 663 | "metadata": {}, 664 | "source": [ 665 | ">**\\[Exercise and Discussion\\]**: \n", 666 | "\n", 667 | "> - Change the initial variables we use to construct the first restricted master problem (but still maintain the starting restricted master problem feasible). How does it effect the convergence of the algorithm? (number of total columns generated?)\n", 668 | "> - Could you find a way to generate multiple columns whose reduced cost are less than 0 at one iteration?" 669 | ] 670 | }, 671 | { 672 | "cell_type": "markdown", 673 | "metadata": {}, 674 | "source": [ 675 | "### How to obtain INTEGER solution? ###\n", 676 | "\n", 677 | "We solve the LP relaxation successfully using column generation, however the original cutting stock problem is an integer program. Can we apply column generation to obtain optimal integer solution? \n", 678 | "\n", 679 | "The answer is _Yes_. However, it involves an advanced solution methodology called [branch-and-price](http://en.wikipedia.org/wiki/Branch_and_price) where column generation is applied on each node of the branch-and-bound tree. Unfortunately, commercial solvers (Gurobi, CPLEX) don't support this feature. Till now, the only academic solver supports branch-and-price is [SCIP](http://scip.zib.de/).\n", 680 | "\n", 681 | "Instead of solving the integer program to optimality, we here introduce two approximation methods that are widely used in solving real-world problems. " 682 | ] 683 | }, 684 | { 685 | "cell_type": "markdown", 686 | "metadata": {}, 687 | "source": [ 688 | "#### Method 1: Rounding####\n", 689 | "\n", 690 | "Rounding a fractional solution to its _nearest_ and _feasible_ is a common heuristic for solving integer program. It's pretty problem specific. In cutting stock problem, we observe that if we _round up_ all the fractional solutions, feasibility will maintain. Thus we get our first integer solution:" 691 | ] 692 | }, 693 | { 694 | "cell_type": "code", 695 | "execution_count": 9, 696 | "metadata": { 697 | "collapsed": false 698 | }, 699 | "outputs": [ 700 | { 701 | "name": "stdout", 702 | "output_type": "stream", 703 | "text": [ 704 | "\n", 705 | "Integer Solution Based on Rounding is:\n", 706 | "\n", 707 | "width: [14,31,36,45]\n", 708 | "Cutting Pattern: [2,0,2,0], Number of Paper Rolls Cut Using this Pattern: 207.0\n", 709 | "Cutting Pattern: [0,0,0,2], Number of Paper Rolls Cut Using this Pattern: 49.0\n", 710 | "Cutting Pattern: [0,2,1,0], Number of Paper Rolls Cut Using this Pattern: 198.0\n", 711 | "Total Number of Paper Rolls Used: 454.0\n" 712 | ] 713 | } 714 | ], 715 | "source": [ 716 | "println(\"\\nInteger Solution Based on Rounding is:\\n\")\n", 717 | "\n", 718 | "println(\"width: \", w)\n", 719 | "\n", 720 | "summation=0\n", 721 | "\n", 722 | "for i=1:length(x)\n", 723 | " \n", 724 | " if JuMP.resultvalue(x[i])>epsilon\n", 725 | " println(\"Cutting Pattern: \", A[:,i], \", Number of Paper Rolls Cut Using this Pattern: \", ceil(JuMP.resultvalue(x[i])))\n", 726 | " summation=summation+ceil(JuMP.resultvalue(x[i]))\n", 727 | " end\n", 728 | "end\n", 729 | "\n", 730 | "for i=1:length(newColumns)\n", 731 | " \n", 732 | " if getValue(newColumns[i])>epsilon\n", 733 | " println(\"Cutting Pattern: \", int(A_new[:,i]), \", Number of Paper Rolls Cut Using this Pattern: \", ceil(JuMP.resultvalue(newColumns[i])))\n", 734 | " summation=summation+ceil(JuMP.resultvalue(newColumns[i]))\n", 735 | " end\n", 736 | "end\n", 737 | "\n", 738 | "println(\"Total Number of Paper Rolls Used: \", summation)" 739 | ] 740 | }, 741 | { 742 | "cell_type": "markdown", 743 | "metadata": {}, 744 | "source": [ 745 | "We now have an integer solution using 454.0 paper rolls in total. Can we do better?" 746 | ] 747 | }, 748 | { 749 | "cell_type": "markdown", 750 | "metadata": {}, 751 | "source": [ 752 | "#### Method 2: Branch-and-Bound on Root Node ####\n", 753 | "\n", 754 | "It is troublesome to implement column generation on every node of the branch and bound tree. A common industry / research practice is to directly branch-and-bound the model only with columns generated from solving the LP relaxation. This is a heuristic because optimal set of cutting patterns for the IP might not be the same as the LP relaxation, i.e. we might lose some \"good columns\" to reach optimal integer solution. The upside is, it is very easy to implement with commercial solvers." 755 | ] 756 | }, 757 | { 758 | "cell_type": "code", 759 | "execution_count": 10, 760 | "metadata": { 761 | "collapsed": false 762 | }, 763 | "outputs": [ 764 | { 765 | "name": "stdout", 766 | "output_type": "stream", 767 | "text": [ 768 | "\n", 769 | "Integer Solution Based on Branch-and-Bound is:\n", 770 | "\n", 771 | "width: [14,31,36,45]\n", 772 | "Cutting Pattern: [0,0,2,0], Number of Paper Rolls Cut Using this Pattern: 100.99999999999999\n", 773 | "Cutting Pattern: [2,0,2,0], Number of Paper Rolls Cut Using this Pattern: 106.0\n", 774 | "Cutting Pattern: [0,0,0,2], Number of Paper Rolls Cut Using this Pattern: 49.0\n", 775 | "Cutting Pattern: [0,2,1,0], Number of Paper Rolls Cut Using this Pattern: 198.0\n", 776 | "Total Number of Paper Rolls Used: 454.0\n", 777 | "Presolve 0 (-4) rows, 0 (-7) columns and 0 (-11) elements\n", 778 | "Optimal - objective value 453\n", 779 | "After Postsolve, objective 453, infeasibilities - dual 0 (0), primal 0 (0)\n", 780 | "Optimal objective 453 - 0 iterations time 0.002, Presolve 0.00\n", 781 | "Cbc0045I Warning 3 integer variables were more than 1.0e-4 away from integer\n", 782 | "Cbc0045I Given objective value 452.25, computed 453\n", 783 | "Cbc0045I Solution with objective value 453 saved\n" 784 | ] 785 | } 786 | ], 787 | "source": [ 788 | "#change the solver from Clp to Cbc in order to get support for integer variables\n", 789 | "#if you use Gurobi as your solver choice, you don't need to switch solver.\n", 790 | "\n", 791 | "setSolver(master, CbcSolver())\n", 792 | "\n", 793 | "#change variable type from continuous to integer\n", 794 | "\n", 795 | "for i=1:length(x)\n", 796 | " setCategory(x[i], :Int)\n", 797 | "end\n", 798 | "\n", 799 | "for i=1:length(newColumns)\n", 800 | " setCategory(newColumns[i],:Int)\n", 801 | "end\n", 802 | "\n", 803 | "JuMP.optimize(master)\n", 804 | "\n", 805 | "\n", 806 | "#print optimal solution\n", 807 | "\n", 808 | "\n", 809 | "\n", 810 | "println(\"\\nInteger Solution Based on Branch-and-Bound is:\\n\")\n", 811 | "\n", 812 | "println(\"width: \", w)\n", 813 | "\n", 814 | "summation=0\n", 815 | "\n", 816 | "for i=1:length(x)\n", 817 | " \n", 818 | " if JuMP.resultvalue(x[i])>epsilon\n", 819 | " println(\"Cutting Pattern: \", A[:,i], \", Number of Paper Rolls Cut Using this Pattern: \", JuMP.resultvalue(x[i]))\n", 820 | " summation=summation+JuMP.resultvalue(x[i])\n", 821 | " end\n", 822 | "end\n", 823 | "\n", 824 | "for i=1:length(newColumns)\n", 825 | " \n", 826 | " if JuMP.resultvalue(newColumns[i])>epsilon\n", 827 | " println(\"Cutting Pattern: \", int(A_new[:,i]), \", Number of Paper Rolls Cut Using this Pattern: \", JuMP.resultvalue(newColumns[i]))\n", 828 | " summation=summation+JuMP.resultvalue(newColumns[i])\n", 829 | " end\n", 830 | "end\n", 831 | "\n", 832 | "println(\"Total Number of Paper Rolls Used: \", summation)" 833 | ] 834 | }, 835 | { 836 | "cell_type": "markdown", 837 | "metadata": {}, 838 | "source": [ 839 | "We save one paper roll by using method 2" 840 | ] 841 | }, 842 | { 843 | "cell_type": "markdown", 844 | "metadata": {}, 845 | "source": [ 846 | "> **\\[Question\\]:**\n", 847 | "\n", 848 | "> Is method 2 always able to produce feasible integer solution?" 849 | ] 850 | } 851 | ], 852 | "metadata": { 853 | "kernelspec": { 854 | "display_name": "Julia 0.6.0", 855 | "language": "julia", 856 | "name": "julia-0.6" 857 | }, 858 | "language_info": { 859 | "file_extension": ".jl", 860 | "mimetype": "application/julia", 861 | "name": "julia", 862 | "version": "0.6.0" 863 | } 864 | }, 865 | "nbformat": 4, 866 | "nbformat_minor": 0 867 | } 868 | -------------------------------------------------------------------------------- /notebooks/Shuvomoy - Column generation.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "**Description:** This notebook describes how to implement column generation, which is a large scale optimization scheme, in JuMP. The cutting stock problem has been used as an illustrative example.\n", 8 | "\n", 9 | "**Author:** [Shuvomoy Das Gupta](http://scg.utoronto.ca/~shuvomoy.dasgupta/)\n", 10 | "\n", 11 | "**License:** \"Creative
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.\n", 12 | "\n", 13 | "# Using Julia+JuMP for optimization - column generation \n", 14 | "--------------------------\n", 15 | "\n", 16 | "Implementing large scale optimization techniques such as column generation is really easy using JuMP. To explain how to implement column generation in JuMP, we consider the famous cutting stock problem. For more details about the problem, see pages 234-236 of Introduction to Linear Optimization by Bertsimas and Tsitsiklis." 17 | ] 18 | }, 19 | { 20 | "cell_type": "markdown", 21 | "metadata": {}, 22 | "source": [ 23 | "## Notation and notions:\n", 24 | "\n", 25 | "- Width of a large roll is $W$, and it needs to be cut into smaller width papers according to customer demand\n", 26 | "- The set of indices of all feasible patterns is, $\\mathcal{J}=\\{1,2,\\ldots,n\\}$, where $n$ is a very large number\n", 27 | "- A strict subset of $\\mathcal{J}$ that is considered in the master problem is $\\mathcal{J}'$\n", 28 | "- The dummy index for a pattern is $j$ \n", 29 | "- The index set of all possible paper-widths is, $\\mathcal{M}=\\{1,2,\\ldots,m\\}$\n", 30 | "- The width of the paper with index $i$ is $w_i$\n", 31 | "- The demand for the paper of width $w_i$ is $b_i$\n", 32 | "- Number of smaller rolls of width $w_i$ produced by pattern $j$ is denoted by $a_{ij}$\n", 33 | "- Number of large rolls cut according to pattern $j$ is denoted by $x_j$" 34 | ] 35 | }, 36 | { 37 | "cell_type": "markdown", 38 | "metadata": {}, 39 | "source": [ 40 | "\n", 41 | "----------------------\n", 42 | "## Original unabridged problem: \n", 43 | "$$\n", 44 | "\\begin{align}\n", 45 | "&\\text{minimize} && \\sum_{j \\in \\mathcal{J}}{x_j} \\\\\n", 46 | "&\\text{subject to} &&\\\\\n", 47 | "& &&\\forall i \\in \\mathcal{M} \\quad \\sum_{j \\in \\mathcal{J}}{a_{ij} x_j}=b_i \\\\\n", 48 | "& && \\forall j \\in \\mathcal{J} \\quad x_j \\geq 0 \\\\\n", 49 | "\\end{align}\n", 50 | "$$\n", 51 | "\n", 52 | "Because the set $\\mathcal{J}$ can be astronomically large, even storing the problem is a challenge. So, we start with a smaller version of the problem, called the master problem, by replacing $\\mathcal{J}$ with a strict subset $\\mathcal{J}'$, which is much smaller than the original one. " 53 | ] 54 | }, 55 | { 56 | "cell_type": "markdown", 57 | "metadata": {}, 58 | "source": [ 59 | "----------------------\n", 60 | "\n", 61 | "## Structure of the decomposition\n", 62 | "\n", 63 | "**Master Problem:**\n", 64 | "$$\n", 65 | "\\begin{align}\n", 66 | "&\\text{minimize} && \\sum_{j \\in \\mathcal{J}'}{x_j} \\\\\n", 67 | "&\\text{subject to} &&\\\\\n", 68 | "& &&\\forall i \\in \\mathcal{M} \\quad \\sum_{j \\in \\mathcal{J}'}{a_{ij} x_j}=b_i \\\\\n", 69 | "& && \\forall j \\in \\mathcal{J}' \\quad x_j \\geq 0 \\\\\n", 70 | "\\end{align}\n", 71 | "$$" 72 | ] 73 | }, 74 | { 75 | "cell_type": "markdown", 76 | "metadata": {}, 77 | "source": [ 78 | "After solving the master problem, we want to check the optimality status. Structure of the cutting stock problem allows us to construct a subproblem which can do this very easily. \n", 79 | "\n", 80 | "**Subproblem:**\n", 81 | "$$\n", 82 | "\\begin{align}\n", 83 | "&\\text{minimize} && 1 - \\sum_{i \\in \\mathcal{M}} \\quad {p_i a_{i {j^*}}} \\\\\n", 84 | "&\\text{subject to} &&\\\\\n", 85 | "& && \\forall i \\in \\mathcal{M} \\quad a_{i {j^*}} \\geq 0, \\quad a_{ij^*} \\; \\text{integer} \\\\\n", 86 | "& && \\sum_{i \\in \\mathcal{M}}{w_i a_{i{j^*}}} \\leq W\\\\\n", 87 | "\\end{align}\n", 88 | "$$\n", 89 | "\n" 90 | ] 91 | }, 92 | { 93 | "cell_type": "markdown", 94 | "metadata": {}, 95 | "source": [ 96 | "The objective of the subproblem is the minimum of the reduced cost vector of the original problem. If the objective value of the subproblem is greater than or equal to $0$, then the current solution of the master problem is optimal for the original unabridged problem. Otherwise, add the resultant cost reducing column $(a_{i {j^*}})_{i \\in \\mathcal{M}}=A_{j*}$ and a corresponding new variable $x_{j*}$ is added to the master problem. The modified master problem is as follows:\n", 97 | "\n", 98 | "**Modified Master Problem**\n", 99 | "$$\n", 100 | "\\begin{align}\n", 101 | "&\\text{minimize} && \\sum_{j \\in \\mathcal{J}'}{x_j} + x_{j^*} \\\\\n", 102 | "&\\text{subject to} &&\\\\\n", 103 | "& &&\\forall i \\in \\mathcal{M} \\quad \\sum_{j \\in \\mathcal{J}'}{a_{ij} x_j}+a_{i j^*} x_{j^*}=b_i \\\\\n", 104 | "& && \\forall j \\in \\mathcal{J}' \\quad x_j \\geq 0, x_j^* \\geq 0 \\\\\n", 105 | "\\end{align}\n", 106 | "$$\n", 107 | "\n", 108 | "The pseudocode for the cutting stock problem is given below." 109 | ] 110 | }, 111 | { 112 | "cell_type": "markdown", 113 | "metadata": {}, 114 | "source": [ 115 | "## Pseduocode\n", 116 | "\n", 117 | "- Input preliminary data for starting the problem\n", 118 | "- Solve the master problem with the initial data\n", 119 | "\n", 120 | "$$\n", 121 | "\\begin{align}\n", 122 | "&\\text{minimize} && \\sum_{j \\in \\mathcal{J}'}{x_j} \\\\\n", 123 | "&\\text{subject to} &&\\\\\n", 124 | "& &&\\forall i \\in \\mathcal{M} \\quad \\sum_{j \\in \\mathcal{J}'}{a_{ij} x_j}=b_i \\\\\n", 125 | "& && \\forall j \\in \\mathcal{J}' \\quad x_j \\geq 0 \\\\\n", 126 | "\\end{align}\n", 127 | "$$\n", 128 | "\n" 129 | ] 130 | }, 131 | { 132 | "cell_type": "markdown", 133 | "metadata": {}, 134 | "source": [ 135 | "- Collect the dual variables for the equality constraints and store them in an array $(p_i)_{i \\in \\mathcal{M}}$\n", 136 | "\n", 137 | "- Solve the sub problem \n", 138 | "\n", 139 | "$$\n", 140 | "\\begin{align}\n", 141 | "&\\text{minimize} && 1 - \\sum_{i \\in \\mathcal{M}} \\quad {p_i a_{i {j^*}}} \\\\\n", 142 | "&\\text{subject to} &&\\\\\n", 143 | "& && \\forall i \\in \\mathcal{M} \\quad a_{i {j^*}} \\geq 0, \\quad a_{ij^*} \\; \\text{integer} \\\\\n", 144 | "& && \\sum_{i \\in \\mathcal{M}}{w_i a_{i{j^*}}} \\leq W\\\\\n", 145 | "\\end{align}\n", 146 | "$$\n" 147 | ] 148 | }, 149 | { 150 | "cell_type": "markdown", 151 | "metadata": {}, 152 | "source": [ 153 | " \n", 154 | "- Flow control:
\n", 155 | "\n", 156 | "\n", 157 | "while ( $\\text{optimal value of the subproblem} < 0$)
\n", 158 | "> * Add the column $(a_{i {j^*}})_{i \\in \\mathcal{M}}=A_{j*}$ to $A$
\n", 159 | "> * Add a corresponding new variable $x_{j*}$ to the list of variables
\n", 160 | "> * Solve the modified master problem
\n", 161 | "\n", 162 | " $$\n", 163 | " \\begin{align}\n", 164 | "&\\text{minimize} && \\sum_{j \\in \\mathcal{J}'}{x_j} + x_{j^*} \\\\\n", 165 | "&\\text{subject to} &&\\\\\n", 166 | "& &&\\forall i \\in \\mathcal{M} \\quad \\sum_{j \\in \\mathcal{J}'}{a_{ij} x_j}+a_{i j^*} x_{j^*}=b_i \\\\\n", 167 | "& && \\forall j \\in \\mathcal{J}' \\quad x_j \\geq 0 \\\\\n", 168 | "& && \\qquad \\qquad \\; \\; x_{j^*} \\geq 0\n", 169 | "\\end{align}\n", 170 | "$$\n", 171 | "\n", 172 | "> * Collect the dual variables for the equality constraints and store them in an array $(p_i)_{i \\in \\mathcal{M}}$\n", 173 | "> * Solve the sub problem as before
\n", 174 | "> * Set $\\mathcal{J}':=\\mathcal{J}'\\cup \\{j^*\\}$
\n", 175 | "\n", 176 | " end while
\n", 177 | "\n", 178 | "- Display the results " 179 | ] 180 | }, 181 | { 182 | "cell_type": "markdown", 183 | "metadata": {}, 184 | "source": [ 185 | "## Master Problem Modification in JuMP\n", 186 | "The problem modification can be done by using the already mentioned `@variable` macro:\n", 187 | "\n", 188 | "$$\n", 189 | "\\texttt{@variable}(m, l \\leq x_\\text{new} \\leq u, \\texttt{Int}, \\texttt{objective} = c_\\text{new}, \\texttt{inconstraints} = \\text{arrayConstrrefs}, \\texttt{coefficients} = \\text{arrayCoefficients}) \n", 190 | "$$\n" 191 | ] 192 | }, 193 | { 194 | "cell_type": "markdown", 195 | "metadata": {}, 196 | "source": [ 197 | "Here: \n", 198 | "\n", 199 | "- The name of the original model is $m$.\n", 200 | "- The new variable to be added is $x_\\text{new}$ with lower bound $l$ and upper bound $u$.\n", 201 | "- The type of the variable can be `Int`, `Bin`. For real variable the third argument is left vacant.\n", 202 | "- The original objective, say $f_o(x)$ will become $f_o(x) + c_\\text{new} x_\\text{new}$ after modification\n", 203 | "- The array $\\texttt{arrayConstrrefs}$ contain references to those constraints that need to be modified by inclusion of $x_\\text{new}$\n", 204 | "- The array $\\texttt{arrayCoefficients}$ contain the coefficients that have to multiplied with $x_\\text{new}$ and then added to the constraints referenced by $\\texttt{arrayConstrrefs}$ in an orderly manner. For example, if the $i$th element of $\\texttt{arrayConstrrefs}$ refers to a constraint $a_i^T x \\lesseqgtr b_i$, then after invoking the command, the constraint is modified as:\n", 205 | "$a_i^T x +\\texttt{arrayCoefficients}[i] x_\\text{new} \\lesseqgtr b_i$" 206 | ] 207 | }, 208 | { 209 | "cell_type": "markdown", 210 | "metadata": {}, 211 | "source": [ 212 | "## Implementing one iteration of the column generation algorithm\n", 213 | "\n", 214 | "To understand how the column generation is working in Julia, we implement one iteration of the column generation algorithm manually. The entire code is presented in the next section.\n" 215 | ] 216 | }, 217 | { 218 | "cell_type": "code", 219 | "execution_count": 1, 220 | "metadata": { 221 | "collapsed": false 222 | }, 223 | "outputs": [], 224 | "source": [ 225 | "# Uploading the packages:\n", 226 | "# -----------------------\n", 227 | "\n", 228 | "using JuMP \n", 229 | "\n", 230 | "# We will use default solvers" 231 | ] 232 | }, 233 | { 234 | "cell_type": "code", 235 | "execution_count": 2, 236 | "metadata": { 237 | "collapsed": false 238 | }, 239 | "outputs": [ 240 | { 241 | "data": { 242 | "text/plain": [ 243 | "5-element Array{Int64,1}:\n", 244 | " 22\n", 245 | " 42\n", 246 | " 52\n", 247 | " 53\n", 248 | " 78" 249 | ] 250 | }, 251 | "execution_count": 2, 252 | "metadata": {}, 253 | "output_type": "execute_result" 254 | } 255 | ], 256 | "source": [ 257 | "# Input preliminary data for starting the problem\n", 258 | "# -----------------------------------------------\n", 259 | "\n", 260 | "W=100\n", 261 | "cardinalityM=5\n", 262 | "M=collect(1:cardinalityM)\n", 263 | "A=eye(cardinalityM)\n", 264 | "p=zeros(5)\n", 265 | "b=[45; 38; 25; 11; 12]\n", 266 | "w=[22; 42; 52; 53; 78]" 267 | ] 268 | }, 269 | { 270 | "cell_type": "code", 271 | "execution_count": 3, 272 | "metadata": { 273 | "collapsed": false 274 | }, 275 | "outputs": [ 276 | { 277 | "name": "stdout", 278 | "output_type": "stream", 279 | "text": [ 280 | "Min x[1] + x[2] + x[3] + x[4] + x[5]\n", 281 | "Subject to\n", 282 | " x[1] == 45\n", 283 | " x[2] == 38\n", 284 | " x[3] == 25\n", 285 | " x[4] == 11\n", 286 | " x[5] == 12\n", 287 | " 0 <= x[i] <= 1.0e6 for all i in {1,2,..,4,5}\n" 288 | ] 289 | } 290 | ], 291 | "source": [ 292 | "# Description of the master problem with the initial data\n", 293 | "#----------------------\n", 294 | "\n", 295 | "cutstockMain = Model() # Model for the master problem\n", 296 | "Jprime=collect(1:size(A,2)) # Initial number of variables\n", 297 | "@variable(cutstockMain, 0 <= x[Jprime] <= 1000000) # Defining the variables\n", 298 | "@objective(cutstockMain, Min, sum(1*x[j] for j in Jprime)) # Setting the objective\n", 299 | "@constraint(cutstockMain, consRef[i=1:cardinalityM], sum(A[i,j]*x[j] for j in Jprime)==b[i]) \n", 300 | "# Here the second argument consRef[i=1:cardinalityM] means that the i-th constraint aᵢᵀx = bᵢ has the corresponding constraint reference\n", 301 | "# consRef[i]\n", 302 | "print(cutstockMain)" 303 | ] 304 | }, 305 | { 306 | "cell_type": "code", 307 | "execution_count": 4, 308 | "metadata": { 309 | "collapsed": false 310 | }, 311 | "outputs": [ 312 | { 313 | "name": "stdout", 314 | "output_type": "stream", 315 | "text": [ 316 | "Current solution of the master problem is x: 1 dimensions:\n", 317 | "[1] = 45.0\n", 318 | "[2] = 38.0\n", 319 | "[3] = 25.0\n", 320 | "[4] = 11.0\n", 321 | "[5] = 12.0\n", 322 | "\n", 323 | "Current objective value of the master problem is 131.0\n" 324 | ] 325 | } 326 | ], 327 | "source": [ 328 | "# Solving the master problem with the initial data\n", 329 | "# ------------------------------------------------\n", 330 | "solve(cutstockMain)\n", 331 | "println(\"Current solution of the master problem is \", getvalue(x))\n", 332 | "println(\"Current objective value of the master problem is \", getobjectivevalue(cutstockMain))" 333 | ] 334 | }, 335 | { 336 | "cell_type": "code", 337 | "execution_count": 5, 338 | "metadata": { 339 | "collapsed": false 340 | }, 341 | "outputs": [ 342 | { 343 | "name": "stdout", 344 | "output_type": "stream", 345 | "text": [ 346 | "The array storing the dual variables is [1.0,1.0,1.0,1.0,1.0]\n" 347 | ] 348 | } 349 | ], 350 | "source": [ 351 | "#Collect the dual variables for the equality constraints and store them in an array p\n", 352 | "for i in M\n", 353 | " p[i] = getdual(consRef[i]) # These p[i] are the input data for the subproblem\n", 354 | "end \n", 355 | "println(\"The array storing the dual variables is \", p)" 356 | ] 357 | }, 358 | { 359 | "cell_type": "code", 360 | "execution_count": 6, 361 | "metadata": { 362 | "collapsed": false 363 | }, 364 | "outputs": [ 365 | { 366 | "name": "stdout", 367 | "output_type": "stream", 368 | "text": [ 369 | "Min -Ajstar[1] - Ajstar[2] - Ajstar[3] - Ajstar[4] - Ajstar[5] + 1\n", 370 | "Subject to\n", 371 | " 22 Ajstar[1] + 42 Ajstar[2] + 52 Ajstar[3] + 53 Ajstar[4] + 78 Ajstar[5] <= 100\n", 372 | " 0 <= Ajstar[i] <= 1.0e6, integer, for all i in {1,2,..,4,5}\n" 373 | ] 374 | } 375 | ], 376 | "source": [ 377 | "# Describe the sub problem\n", 378 | "# ------------------------\n", 379 | "cutstockSub=Model() # Model for the subproblem\n", 380 | "@variable(cutstockSub, 0 <= Ajstar[M] <= 1000000, Int )\n", 381 | "@objective(cutstockSub, Min, 1-sum(p[i]*Ajstar[i] for i in M))\n", 382 | "@constraint(cutstockSub, sum(w[i]*Ajstar[i] for i in M) <= W)\n", 383 | "print(cutstockSub)" 384 | ] 385 | }, 386 | { 387 | "cell_type": "code", 388 | "execution_count": 7, 389 | "metadata": { 390 | "collapsed": false 391 | }, 392 | "outputs": [ 393 | { 394 | "name": "stdout", 395 | "output_type": "stream", 396 | "text": [ 397 | "The minimum component of the reduced cost vector is -3.0\n" 398 | ] 399 | } 400 | ], 401 | "source": [ 402 | "# Solve the sub problem\n", 403 | "# ---------------------\n", 404 | "solve(cutstockSub)\n", 405 | "minreducedCost=getobjectivevalue(cutstockSub)\n", 406 | "println(\"The minimum component of the reduced cost vector is \", minreducedCost)" 407 | ] 408 | }, 409 | { 410 | "cell_type": "markdown", 411 | "metadata": {}, 412 | "source": [ 413 | "The minimum component of the reduced cost vector is negative, so we have a suboptimal solution." 414 | ] 415 | }, 416 | { 417 | "cell_type": "code", 418 | "execution_count": 8, 419 | "metadata": { 420 | "collapsed": false 421 | }, 422 | "outputs": [ 423 | { 424 | "name": "stdout", 425 | "output_type": "stream", 426 | "text": [ 427 | "We have a cost reducing column Ajstar: 1 dimensions:\n", 428 | "[1] = 4.0\n", 429 | "[2] = 0.0\n", 430 | "[3] = 0.0\n", 431 | "[4] = 0.0\n", 432 | "[5] = 0.0\n", 433 | "\n" 434 | ] 435 | } 436 | ], 437 | "source": [ 438 | "if minreducedCost >= 0\n", 439 | " println(\"We are done, current solution of the master problem is optimal\")\n", 440 | "else\n", 441 | " println(\"We have a cost reducing column \", getvalue(Ajstar))\n", 442 | "end\n" 443 | ] 444 | }, 445 | { 446 | "cell_type": "code", 447 | "execution_count": 9, 448 | "metadata": { 449 | "collapsed": false 450 | }, 451 | "outputs": [ 452 | { 453 | "data": { 454 | "text/plain": [ 455 | "JuMP.JuMPArray{JuMP.Variable,1,Tuple{Array{Int64,1}}}" 456 | ] 457 | }, 458 | "execution_count": 9, 459 | "metadata": {}, 460 | "output_type": "execute_result" 461 | } 462 | ], 463 | "source": [ 464 | "typeof(Ajstar)" 465 | ] 466 | }, 467 | { 468 | "cell_type": "markdown", 469 | "metadata": {}, 470 | "source": [ 471 | "Now `Ajstar` is of type JuMPDict. To use it in the modified master problem, we have to store values from `Ajstar` in a column vector." 472 | ] 473 | }, 474 | { 475 | "cell_type": "code", 476 | "execution_count": 10, 477 | "metadata": { 478 | "collapsed": false 479 | }, 480 | "outputs": [], 481 | "source": [ 482 | "Anew=Float64[] # This Anew correspond to the newly added column to the A matrix\n", 483 | "for i in 1:cardinalityM\n", 484 | " push!(Anew, getvalue(Ajstar)[i])\n", 485 | "end" 486 | ] 487 | }, 488 | { 489 | "cell_type": "markdown", 490 | "metadata": {}, 491 | "source": [ 492 | "When we add the cost reducing column `Anew` to the original matrix `A`, it also gives rise to a new variable `xNew` corresponding to `Anew`. Now we want to keep track of the new variables that are added by the subproblem. We do this by declaring an array of `Variable`s named `xNewArray`, which will contain all such newly added variables in the process of column generation. " 493 | ] 494 | }, 495 | { 496 | "cell_type": "code", 497 | "execution_count": 11, 498 | "metadata": { 499 | "collapsed": false 500 | }, 501 | "outputs": [ 502 | { 503 | "data": { 504 | "text/latex": [ 505 | "$$ Empty Array{Variable} (no indices) $$" 506 | ], 507 | "text/plain": [ 508 | "0-element Array{JuMP.Variable,1}" 509 | ] 510 | }, 511 | "execution_count": 11, 512 | "metadata": {}, 513 | "output_type": "execute_result" 514 | } 515 | ], 516 | "source": [ 517 | "xNewArray=Variable[] # The newly added variables by flow control will be\n", 518 | "# pushed to the new array of variables xNewArray" 519 | ] 520 | }, 521 | { 522 | "cell_type": "markdown", 523 | "metadata": {}, 524 | "source": [ 525 | "Here we just illustrate one iteration of the while loop manually, because, for now, we are interested to understand how JuMP is managing the flow control and modifying the master problem and the sub problem. \n", 526 | "\n", 527 | "Let's modify the master problem by adding the new column `Anew` to the old `A` matrix. Note that we do not have to rewrite the entire model." 528 | ] 529 | }, 530 | { 531 | "cell_type": "code", 532 | "execution_count": 12, 533 | "metadata": { 534 | "collapsed": false 535 | }, 536 | "outputs": [ 537 | { 538 | "name": "stdout", 539 | "output_type": "stream", 540 | "text": [ 541 | "Min x[1] + x[2] + x[3] + x[4] + x[5] + xNew\n", 542 | "Subject to\n", 543 | " x[1] + 4 xNew == 45\n", 544 | " x[2] == 38\n", 545 | " x[3] == 25\n", 546 | " x[4] == 11\n", 547 | " x[5] == 12\n", 548 | " 0 <= x[i] <= 1.0e6 for all i in {1,2,..,4,5}\n", 549 | " 0 <= xNew <= 1.0e6\n" 550 | ] 551 | } 552 | ], 553 | "source": [ 554 | "# Modify the master problem by adding the new column Anew to the old A matrix\n", 555 | "@variable(\n", 556 | "cutstockMain, # Model to be modified\n", 557 | "0 <= xNew <= 1000000, # New variable to be added\n", 558 | "objective=1, # cost coefficient of new variable in the objective\n", 559 | "inconstraints=consRef, # constraints to be modified\n", 560 | "coefficients=Anew # the coefficients of the variable in those constraints\n", 561 | ") \n", 562 | "\n", 563 | "# The line above adds the column (aᵢⱼ*)ᵢ=Aⱼ* to A
\n", 564 | "# and add a corresponding new variable xⱼ* to the list of variable\n", 565 | "\n", 566 | "push!(xNewArray, xNew) # Pushing the new variable in the array of new variables\n", 567 | "print(cutstockMain)" 568 | ] 569 | }, 570 | { 571 | "cell_type": "markdown", 572 | "metadata": {}, 573 | "source": [ 574 | "Though we are showing only one iteration of the flow control, in the final code for sure we want to have a \n", 575 | ">```\n", 576 | "while ( some condition )\n", 577 | "(\n", 578 | "...\n", 579 | ")\n", 580 | "end\n", 581 | "```\n", 582 | "\n", 583 | "block. \n", 584 | "\n", 585 | "Now if we do not do anything else in the final code, all the names of the newly added variables by the `while` loop will be the same: `xNew`! JuMP is intelligent enough to treat them as separate variables, but it is not very human-friendly. It is more convenient if the newly added variables were given different names, which we can achieve by `setName(oldName, newName)` function.\n" 586 | ] 587 | }, 588 | { 589 | "cell_type": "code", 590 | "execution_count": 13, 591 | "metadata": { 592 | "collapsed": false 593 | }, 594 | "outputs": [ 595 | { 596 | "data": { 597 | "text/plain": [ 598 | "\"x[6]\"" 599 | ] 600 | }, 601 | "execution_count": 13, 602 | "metadata": {}, 603 | "output_type": "execute_result" 604 | } 605 | ], 606 | "source": [ 607 | "setname(xNew, string(\"x[\",size(A,2)+1,\"]\")) # Changing the name of the variable \n", 608 | "# otherwise all the newly added variables will have name xNew!\n", 609 | "# size(A,2) gives the column number of A" 610 | ] 611 | }, 612 | { 613 | "cell_type": "markdown", 614 | "metadata": {}, 615 | "source": [ 616 | "Let us see if the name of the variable has changed as desired." 617 | ] 618 | }, 619 | { 620 | "cell_type": "code", 621 | "execution_count": 14, 622 | "metadata": { 623 | "collapsed": false 624 | }, 625 | "outputs": [ 626 | { 627 | "name": "stdout", 628 | "output_type": "stream", 629 | "text": [ 630 | "Min x[1] + x[2] + x[3] + x[4] + x[5] + x[6]\n", 631 | "Subject to\n", 632 | " x[1] + 4 x[6] == 45\n", 633 | " x[2] == 38\n", 634 | " x[3] == 25\n", 635 | " x[4] == 11\n", 636 | " x[5] == 12\n", 637 | " 0 <= x[i] <= 1.0e6 for all i in {1,2,..,4,5}\n", 638 | " 0 <= x[6] <= 1.0e6\n" 639 | ] 640 | } 641 | ], 642 | "source": [ 643 | "print(cutstockMain) # Let us see if the name of the variables have changed as desired" 644 | ] 645 | }, 646 | { 647 | "cell_type": "markdown", 648 | "metadata": {}, 649 | "source": [ 650 | "Indeed it has! Now let's solve the modified master problem, and then collect the associated dual variables for the equality constraints and store them in the array `p`." 651 | ] 652 | }, 653 | { 654 | "cell_type": "code", 655 | "execution_count": 15, 656 | "metadata": { 657 | "collapsed": false 658 | }, 659 | "outputs": [ 660 | { 661 | "name": "stdout", 662 | "output_type": "stream", 663 | "text": [ 664 | "[0.25,1.0,1.0,1.0,1.0]\n" 665 | ] 666 | } 667 | ], 668 | "source": [ 669 | "statusControlFlow=solve(cutstockMain) # Solve the modified master problem\n", 670 | "\n", 671 | "getdual(consRef)\n", 672 | "for i in M\n", 673 | " p[i] = getdual(consRef)[i] \n", 674 | "end \n", 675 | "\n", 676 | "println(p)" 677 | ] 678 | }, 679 | { 680 | "cell_type": "markdown", 681 | "metadata": {}, 682 | "source": [ 683 | "Now we solve the subproblem for the current solution of the master problem:" 684 | ] 685 | }, 686 | { 687 | "cell_type": "code", 688 | "execution_count": 16, 689 | "metadata": { 690 | "collapsed": false 691 | }, 692 | "outputs": [ 693 | { 694 | "name": "stdout", 695 | "output_type": "stream", 696 | "text": [ 697 | "Min -0.25 Ajstar[1] - Ajstar[2] - Ajstar[3] - Ajstar[4] - Ajstar[5] + 1\n", 698 | "Subject to\n", 699 | " 22 Ajstar[1] + 42 Ajstar[2] + 52 Ajstar[3] + 53 Ajstar[4] + 78 Ajstar[5] <= 100\n", 700 | " 22 Ajstar[1] + 42 Ajstar[2] + 52 Ajstar[3] + 53 Ajstar[4] + 78 Ajstar[5] <= 100\n", 701 | " 0 <= Ajstar[i] <= 1.0e6, integer, for all i in {1,2,..,4,5}\n", 702 | " 0 <= Ajstar[i] <= 1.0e6, integer, for all i in {1,2,..,4,5}\n", 703 | "Current value of the minimum of the reduced cost vector is -1.0\n" 704 | ] 705 | }, 706 | { 707 | "name": "stderr", 708 | "output_type": "stream", 709 | "text": [ 710 | "WARNING: Solver does not appear to support providing initial feasible solutions.\n" 711 | ] 712 | } 713 | ], 714 | "source": [ 715 | "# Solving the modified sub problem \n", 716 | "@variable(cutstockSub, 0 <= Ajstar[M] <= 1000000, Int )\n", 717 | "@objective(cutstockSub, Min, 1-sum(p[i]*Ajstar[i] for i in M))\n", 718 | "@constraint(cutstockSub, sum(w[i]*Ajstar[i] for i in M) <= W)\n", 719 | "print(cutstockSub) # Let's see what is the current subproblem looks like\n", 720 | "solve(cutstockSub)\n", 721 | "minReducedCost=getobjectivevalue(cutstockSub)\n", 722 | "println(\"Current value of the minimum of the reduced cost vector is \", minReducedCost)" 723 | ] 724 | }, 725 | { 726 | "cell_type": "markdown", 727 | "metadata": {}, 728 | "source": [ 729 | "The optimal value of the current subproblem is negative (which will be tested by the conditional statement of the while loop in the final code), giving us a cost reducing column to be added in the master problem. As before we have to store the column `Ajstar` in a column vector `Anew`." 730 | ] 731 | }, 732 | { 733 | "cell_type": "code", 734 | "execution_count": 17, 735 | "metadata": { 736 | "collapsed": false 737 | }, 738 | "outputs": [ 739 | { 740 | "name": "stdout", 741 | "output_type": "stream", 742 | "text": [ 743 | "New column to be added to A is: [0.0,2.0,0.0,0.0,0.0]\n" 744 | ] 745 | } 746 | ], 747 | "source": [ 748 | "#Store the components of the solution of current subproblem into the column Anew \n", 749 | "Anew=Float64[]\n", 750 | "for i in 1:cardinalityM\n", 751 | " push!(Anew, getvalue(Ajstar)[i])\n", 752 | "end\n", 753 | "\n", 754 | "println(\"New column to be added to A is: \", Anew)" 755 | ] 756 | }, 757 | { 758 | "cell_type": "markdown", 759 | "metadata": {}, 760 | "source": [ 761 | "Okay, we have understood how JuMP is working in the column generation process. The entire code of the cutting stock problem is given below:" 762 | ] 763 | }, 764 | { 765 | "cell_type": "markdown", 766 | "metadata": {}, 767 | "source": [ 768 | "## Cutting stock problem code:" 769 | ] 770 | }, 771 | { 772 | "cell_type": "code", 773 | "execution_count": 18, 774 | "metadata": { 775 | "collapsed": false 776 | }, 777 | "outputs": [ 778 | { 779 | "name": "stdout", 780 | "output_type": "stream", 781 | "text": [ 782 | " 0.003651 seconds (4.00 k allocations: 265.625 KB)\n", 783 | "Objective value: 57.25\n", 784 | "Current Solution is: x: 1 dimensions:\n", 785 | "[1] = 0.0\n", 786 | "[2] = 0.0\n", 787 | "[3] = 0.0\n", 788 | "[4] = 0.0\n", 789 | "[5] = 0.0\n", 790 | "\n", 791 | "With 5 variables added by flow control:\n", 792 | "[6] = 8.25\n", 793 | "[7] = 1.0\n", 794 | "[8] = 11.0\n", 795 | "[9] = 25.0\n", 796 | "[10] = 12.0\n", 797 | "Reduced cost of the current solution is 0.0\n" 798 | ] 799 | } 800 | ], 801 | "source": [ 802 | "# Verfied to be working:\n", 803 | "\n", 804 | "# Uploading the packages:\n", 805 | "# -----------------------\n", 806 | "\n", 807 | "using JuMP\n", 808 | "using GLPKMathProgInterface\n", 809 | "\n", 810 | "# Input preliminary data for starting the problem\n", 811 | "# -----------------------------------------------\n", 812 | "\n", 813 | "W=100\n", 814 | "cardinalityM=5\n", 815 | "M=collect(1:cardinalityM)\n", 816 | "A=eye(cardinalityM)\n", 817 | "p=zeros(5)\n", 818 | "b=[45; 38; 25; 11; 12]\n", 819 | "w=[22; 42; 52; 53; 78]\n", 820 | "\n", 821 | "@time begin # time measurement begins\n", 822 | "\n", 823 | "# Solve the master problem with the initial data\n", 824 | "#-----------------------------------------------\n", 825 | "\n", 826 | "cutstockMain = Model() # Model for the master problem\n", 827 | "Jprime=collect(1:size(A,2)) # Intial number of variables\n", 828 | "@variable(cutstockMain, 0 <= x[Jprime] <= 1000000) # Defining the variables\n", 829 | " \n", 830 | "@objective(cutstockMain, Min, sum(1*x[j] for j in Jprime)) # Setting the objective\n", 831 | " \n", 832 | "@constraint(cutstockMain, consRef[i=1:cardinalityM], sum(A[i,j]*x[j] for j in Jprime)==b[i]) # Adding the constraints\n", 833 | "# Here the second argument consRef[i=1:cardinalityM] means that the i-th constraint aᵢᵀx = bᵢ has \n", 834 | "# the corresponding constraint reference consRef[i]\n", 835 | "\n", 836 | "solve(cutstockMain)\n", 837 | " \n", 838 | "#Collect the dual variables for the equality constraints and store them in an array p\n", 839 | "getdual(consRef)\n", 840 | "for i in M\n", 841 | " p[i] = getdual(consRef)[i] # These p[i] are the input data for the subproblem\n", 842 | "end \n", 843 | "\n", 844 | "# Solve the sub problem\n", 845 | "# -------------------\n", 846 | "\n", 847 | "cutstockSub=Model() # Model for the subproblem\n", 848 | "@variable(cutstockSub, 0 <= Ajstar[M] <= 1000000, Int )\n", 849 | "@objective(cutstockSub, Min, 1-sum(p[i]*Ajstar[i] for i in M))\n", 850 | "@constraint(cutstockSub, sum(w[i]*Ajstar[i] for i in M) <= W)\n", 851 | "solve(cutstockSub)\n", 852 | "minReducedCost=getobjectivevalue(cutstockSub)\n", 853 | "\n", 854 | "Anew=Float64[] # This Anew correspond to the newly added column to the A matrix\n", 855 | "for i in 1:cardinalityM\n", 856 | " push!(Anew, getvalue(Ajstar)[i])\n", 857 | "end\n", 858 | "\n", 859 | "xNewArray=Variable[] # The newly added variables by flow control will be pushed to the new array of variables xNewArray\n", 860 | "\n", 861 | "k=1 # Counter for the while loop\n", 862 | "\n", 863 | " # Flow control\n", 864 | " # ------------\n", 865 | "\n", 866 | "while minReducedCost < 0 #while (current solution of the master problem is suboptimal, i.e., subproblem objective value < 0)\n", 867 | " # Solve the master problem by adding the new column Anew to the old A matrix\n", 868 | " @variable(\n", 869 | " cutstockMain, # Model to be modified\n", 870 | " 0 <= xNew <= 1000000, # New variable to be added\n", 871 | " objective=1, # cost coefficient of new varaible in the objective\n", 872 | " inconstraints=consRef, # constraints to be modified\n", 873 | " coefficients=Anew # the coefficients of the variable in those constraints\n", 874 | " ) \n", 875 | " # The line above adds the column (aᵢⱼ*)ᵢ=Aⱼ* to A
\n", 876 | " # and add a corresponding new variable xⱼ* to the list of variable\n", 877 | " push!(xNewArray, xNew) # Pushing the new variable in the array of new variables\n", 878 | " setname(xNew, string(\"x[\",size(A,2)+k,\"]\")) # Changing the name of the variable \n", 879 | " # otherwise all the newly added variables will have name xNew!\n", 880 | " k=k+1 # Increasing k by 1\n", 881 | " statusControlFlow=solve(cutstockMain)\n", 882 | "\n", 883 | " #Collect the dual variables for the equality constraints and store them in an array p\n", 884 | " getdual(consRef)\n", 885 | " for i in M\n", 886 | " p[i] = getdual(consRef)[i] \n", 887 | " end \n", 888 | "\n", 889 | " # Solving the modified sub problem \n", 890 | " @variable(cutstockSub, 0 <= Ajstar[M] <= 1000000, Int )\n", 891 | " @objective(cutstockSub, Min, 1-sum{p[i]*Ajstar[i],i in M})\n", 892 | " @constraint(cutstockSub, sum{w[i]*Ajstar[i], i in M} <= W)\n", 893 | " solve(cutstockSub)\n", 894 | " minReducedCost=getobjectivevalue(cutstockSub)\n", 895 | "\n", 896 | " #Store the components of the solution of current subproblem into the column Anew \n", 897 | " Anew=Float64[]\n", 898 | " for i in 1:cardinalityM\n", 899 | " push!(Anew, getvalue(Ajstar)[i])\n", 900 | " end\n", 901 | " end # While loop ends\n", 902 | " \n", 903 | "end # time measurement ends\n", 904 | "\n", 905 | "# Print the results\n", 906 | "# -----------------\n", 907 | "\n", 908 | "println(\"Objective value: \", getobjectivevalue(cutstockMain))\n", 909 | "println(\"Current Solution is: \", getvalue(x))\n", 910 | "println(\"With \", length(xNewArray), \" variables added by flow control:\")\n", 911 | "for i in 1:length(xNewArray)\n", 912 | " println(\"[\",size(A,2)+i,\"] = \",getvalue(xNewArray[i]))\n", 913 | "end\n", 914 | "println(\"Reduced cost of the current solution is \", getobjectivevalue(cutstockSub))" 915 | ] 916 | } 917 | ], 918 | "metadata": { 919 | "kernelspec": { 920 | "display_name": "Julia 0.6.0", 921 | "language": "julia", 922 | "name": "julia-0.6" 923 | }, 924 | "language_info": { 925 | "file_extension": ".jl", 926 | "mimetype": "application/julia", 927 | "name": "julia", 928 | "version": "0.6.0" 929 | } 930 | }, 931 | "nbformat": 4, 932 | "nbformat_minor": 0 933 | } 934 | --------------------------------------------------------------------------------