├── LICENSE ├── README.md ├── cpp └── SQP │ └── SQP.cpp ├── matlab └── SQP │ ├── EQP_Solution.m │ ├── SQP_main.m │ ├── compare_blocking.m │ ├── compare_negative_lambda.m │ ├── constraints.m │ └── scenario_function.m ├── python ├── .idea │ ├── dictionaries │ │ └── monst.xml │ ├── inspectionProfiles │ │ └── profiles_settings.xml │ ├── misc.xml │ ├── modules.xml │ ├── python.iml │ ├── vcs.xml │ └── workspace.xml ├── EQP_Solution.py ├── SQP_main.py ├── __pycache__ │ ├── EQP_Solution.cpython-38.pyc │ └── compare_blocking.cpython-38.pyc └── compare_blocking.py └── video_scanner ├── __pycache__ └── pyimagesearch.cpython-38.pyc ├── demo.py ├── file.jpg ├── file_scan.jpg ├── pyimagesearch.py ├── scanner.py └── test_video.py /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021 Muqian Chen 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # optimization_study 2 | 3 | Here will I record the path that I learn the optimal theory by coding the algorithms with different language. 4 | 5 | # Sequential Quadratic Programming with Python 3.8 and Numpy 6 | 7 | After hard working of two days I have almost implemented SQP-Algorithm with Python. Python-Package Numpy is being used. 8 | The rest work of completing the code is aimed to adapt the program with equation constraints. Because most parts about 9 | this algorithm are caused by inequation constraints, so to determine the total framework of the program I try to write 10 | the code by aiming at inequation constraints. 11 | 12 | ----28.01.2021 13 | 14 | 15 | All works are done and the program is now able to solve a linear optimization problem with whatever equation or 16 | inequation constraints. Some days I may improve the program with taylor expansion for dealing with nonlinear problem. 17 | 18 | ----29.01.2021 19 | 20 | # Sequential Quadratic Programming with C++ and Eigen Library 21 | 22 | After another hard working of 3 days I have implemented SQP algorithm with C++ and Eigen library. The aim of this work 23 | is to learn about how to operate matrix with C++ language and to consolidate the my knowledge about numerical optimizaiton. 24 | And then I would like to use this library on ROS to implement some Tracking Algorithms or Sensor Fusion algorithms. But 25 | there are still some tiny works to be finished such that I have to cover the case of eqution constraints and add some 26 | error raises. 27 | 28 | -----31.01.2021 29 | 30 | Finally today I have finished all work about programming a C++ program to run a sequential quadratic programming algorithm. 31 | And also I have reached my aim to learning Eigen Library for next tasks on Robot Operating System(ROS). I really appreciate 32 | that I have made a hard working so that I could experience the beauty of this library and C++ and get more confidence to face 33 | the works in my future career. 34 | 35 | ------01.02.2021 36 | -------------------------------------------------------------------------------- /cpp/SQP/SQP.cpp: -------------------------------------------------------------------------------- 1 | /* 2 | *Topic: Implementation of Sequential Quadratic Programming in C++ 3 | *Library: Eigen, a c++ matrix operation library. 4 | *Author: Muqian, Chen 5 | *Current Date: 31.01.2021 6 | *Log: The function to calculate a quadratic optimal programming constrainted by inequation 7 | * conditions are totally implemnted. It did take a long time because I am not familiar 8 | * with lots of implicit mechamism of c++. But I feel so lucky that I really started to 9 | * program with it. I think that I am moving towards to the aim that I really implement 10 | * something based on ROS. The next task is to cover the case of equation constraints 11 | * and add a function which automatically calculates the initial working set. Over. 12 | */ 13 | 14 | #include 15 | #include 16 | #include 17 | #include 18 | 19 | using namespace std; 20 | using namespace Eigen; 21 | VectorXd eqpCalculation(MatrixXd A, VectorXd b, MatrixXd G, 22 | VectorXd d, VectorXd theta); 23 | double calculationAlpha(MatrixXd& A_working, VectorXd& b_working, 24 | MatrixXd& A_deactive,VectorXd& b_deactive, 25 | VectorXd stepLength, VectorXd theta); 26 | 27 | 28 | //这里若是加上&就会变成按引用传递了亲，已经用test检验过了。 29 | VectorXd eqpCalculation(MatrixXd A, VectorXd b, MatrixXd G, 30 | VectorXd d, VectorXd theta) { 31 | VectorXd g, h; 32 | g = d + G * theta; 33 | h = A * theta - b; 34 | Index g_row = g.rows(); 35 | Index h_row = h.rows(); 36 | 37 | //catenate the A and G as KKT Matrix 38 | Index A_row = A.rows(); 39 | Index A_col = A.cols(); 40 | Index G_row = G.rows(); 41 | 42 | //first catenate them in horizont direction 43 | MatrixXd K1(G_row, A_row + G_row); 44 | K1 << G, A.transpose(); 45 | MatrixXd zeros = MatrixXd::Zero(A_row, A_row); 46 | MatrixXd K2(A_row, A_col + A_row); 47 | K2 << A, zeros; 48 | MatrixXd K(A_row + G_row, A_row + G_row); 49 | K << K1, K2; 50 | 51 | //then catenate [g;h] 52 | VectorXd g_h(g_row + h_row, 1); 53 | g_h << g, h; 54 | //for debug 55 | //cout << "[INFO]Current KKT Matrix is:" << endl << K << endl; 56 | //cout << "[INFO]Current g_h is:" << endl << g_h << endl; 57 | 58 | //calculate KKT-Equation 59 | VectorXd result(G_row + A_row, 1); 60 | result = K.inverse() * g_h; 61 | 62 | return result; 63 | }; 64 | 65 | //implementation of a function template for removing a specific row from MatrixXf or VectorXf 66 | //After validation, this function could work at situation MatrixXf or VectorXf 67 | template void removeRow(T1& a, T2 b) { 68 | // a: Matrix or Vector which need to remove the row 69 | // b: the row number of the removed row. 70 | 71 | const int newRow = a.rows() - 1; 72 | const int newCol = a.cols(); 73 | 74 | if (b < a.rows()) { 75 | a.block(b, 0, newRow - b, a.cols()) = 76 | a.block(b + 1, 0, newRow - b, a.cols()); 77 | } 78 | else { 79 | cout << "[INFO] The row to remove is out of boundary of input !" << endl; 80 | }; 81 | a.conservativeResize(newRow, newCol); 82 | } 83 | 84 | //Implementation of a function template for adding a trivial but dimension-matched row into 85 | //MatrixXf or VectorXf 86 | template void addRow(T1& a, T2 b) { 87 | // a: the target Matrix or Vector 88 | // b: added row into target 89 | // column number should be unchanged. 90 | 91 | int newRow = a.rows() + 1; 92 | int newCol = b.cols(); 93 | 94 | a.conservativeResize(newRow, newCol); 95 | a.block(newRow - 1, 0, 1, newCol) = b; 96 | }; 97 | 98 | 99 | //Implementation of a fucntion template 100 | void setInitial(MatrixXd A, MatrixXd& A_working, MatrixXd& A_deactive, VectorXd b, VectorXd& b_working, 101 | VectorXd& b_deactive, VectorXd initialTheta, int f) { 102 | //f: amount of inequations 103 | VectorXd result(f ,1); 104 | result = A * initialTheta - b; 105 | ArrayXd resultArray; 106 | resultArray = result.array().abs(); 107 | for (int i = 0; i < f; i++) { 108 | if ( result(i, 0) < 1e-7) { 109 | //add the matched constraints into working set 110 | addRow(A_working, A.row(i)); 111 | addRow(b_working, b.row(i)); 112 | } 113 | else { 114 | //add the unmatched constraints into deacitve set 115 | addRow(A_deactive, A.row(i)); 116 | addRow(b_deactive, b.row(i)); 117 | } 118 | } 119 | } 120 | 121 | 122 | //Pass by reference could be used here for updating the working set. 123 | double calculationAlpha(MatrixXd& A_working, VectorXd& b_working, 124 | MatrixXd& A_deactive, VectorXd& b_deactive, VectorXd stepLength, VectorXd theta, int i) { 125 | double alpha = -1; 126 | int deactRow = A_deactive.rows(); 127 | int deactCol = A_deactive.cols(); 128 | 129 | VectorXd resultAiP(deactRow, 1); 130 | resultAiP = A_deactive * stepLength; 131 | 132 | VectorXd resultAiTheta(deactRow, 1); 133 | resultAiTheta = b_deactive - A_deactive * theta; 134 | 135 | ArrayXd result(deactRow, 1); 136 | ArrayXd resultUp(deactRow, 1); 137 | ArrayXd resultDown(deactRow, 1); 138 | resultUp = resultAiTheta.array(); 139 | resultDown = resultAiP.array(); 140 | //for debug 141 | //cout << "[INFO]: current resultUp is:" << endl << resultUp << endl; 142 | //cout << "[INFO]: current resultDown is:" << endl << resultDown << endl; 143 | 144 | 145 | for (int i = 0; i < deactRow; i++) { 146 | if (resultDown(i,0)< 0) { 147 | result(i, 0) = resultUp(i, 0) / resultDown(i, 0); 148 | } 149 | else { 150 | result(i, 0) = 1000.0; 151 | } 152 | } 153 | //for debug 154 | //cout << "[INFO]: current result is:" << endl << result << endl; 155 | 156 | 157 | Index minRow, minCol; 158 | double min = 0; 159 | min = result.matrix().minCoeff(&minRow, &minCol); 160 | 161 | //=============================Attention！！！！====================================== 162 | //这里有个坑，就是c++中除号/出来的结果会直接舍弃小数点后面的数，比如这里min应该是1.429 163 | //但是只输出1，但是在这个程序中我可以不用管这个问题，因为之后是要拿去跟1做比较的。 164 | //不对啊，我得管，因为之后alpha若是算出小于一则按照这个规则直接就是零了。。。 165 | //遭重了，得把所有矩阵都改成double类型。 166 | //全部都改成也不行，当你使用mat.array()的时候它会自动将类型换掉（或者有别的我不懂的内 167 | //隐操作），通过实验得出，必须要另外声明ArrayXd变量来储存mat.array()的结果，这样才能在 168 | //相除操作中得到完美的小数点后面的数。 169 | //-----31.01.2021 170 | //ArrayXd result(deactRow, 1); 171 | //ArrayXd resultUp(deactRow, 1); 172 | //ArrayXd resultDown(deactRow, 1); 173 | //resultUp = resultAiTheta.array(); 174 | //resultDown = resultAiP.array(); 175 | //result = resultUp / resultDown; 176 | //cout << "[INFO]Current result is :" << result << endl; 177 | //=============================Attention！！！！====================================== 178 | 179 | if (min < 1) { 180 | alpha = min; 181 | 182 | //add the blocking constraint into working set 183 | addRow(A_working, A_deactive.row(minRow)); 184 | addRow(b_working, b_deactive.row(minRow)); 185 | cout << "[INFO] " << i << ". loop. A constraint is added to working set and deleted from deactive set." << endl; 186 | 187 | //delete the blocking constraint from deactive set 188 | removeRow(A_deactive, minRow); 189 | removeRow(b_deactive, minRow); 190 | 191 | return alpha; 192 | } 193 | else { 194 | alpha = 1; 195 | return alpha; 196 | } 197 | } 198 | 199 | 200 | int main(int argc, char** argv) 201 | { 202 | //define G and d of objective function 203 | MatrixXd G(2,2); 204 | G << 2.0, 0.0, 205 | 0.0, 2.0; 206 | VectorXd d(2,1); 207 | d << -2.0, 208 | -5.0; 209 | 210 | //define initialized theta 211 | VectorXd theta(2, 1); 212 | theta(0) = 2.0; 213 | theta(1) = 0.0; 214 | int ParameterNum = theta.rows(); 215 | 216 | //define equation constraints 217 | MatrixXd AEquation(0, ParameterNum); 218 | VectorXd bEquation(0, 1); 219 | 220 | 221 | //define inequation constraints 222 | MatrixXd AInequation(5, ParameterNum); 223 | AInequation << 1.0, -2.0, 224 | -1.0, -2.0, 225 | -1.0, 2.0, 226 | 1.0, 0.0, 227 | 0.0, 1.0; 228 | 229 | int AInitialRowNum = AInequation.rows(); 230 | 231 | VectorXd bInequation(5, 1); 232 | bInequation << -2.0, 233 | -6.0, 234 | -2.0, 235 | 0.0, 236 | 0.0; 237 | 238 | 239 | 240 | 241 | //define working set 242 | MatrixXd A_working(0, ParameterNum); 243 | VectorXd b_working(0, 1); 244 | 245 | A_working = AEquation; 246 | b_working = bEquation; 247 | 248 | 249 | MatrixXd A_deactive(0, ParameterNum); 250 | VectorXd b_deactive(0, 1); 251 | 252 | setInitial(AInequation, A_working, A_deactive, bInequation, 253 | b_working, b_deactive, theta, AInitialRowNum); 254 | 255 | //for debug 256 | //cout << "[INFO]Current A inequation constraints is:" << endl << A_working << endl; 257 | //cout << "[INFO]Current b inequation constraints is:" << endl << b_working << endl; 258 | //define the main variable during the calculating loop 259 | Index constraintsNumber = b_working.rows(); 260 | Index thetaNumber = theta.rows(); 261 | 262 | VectorXd result(thetaNumber+constraintsNumber, 1); 263 | VectorXd stepLength(thetaNumber, 1); 264 | VectorXd lambdaStar(constraintsNumber, 1); 265 | 266 | int i = 1; 267 | 268 | while (true) { 269 | 270 | 271 | result = eqpCalculation(A_working, b_working, G, d, theta); 272 | 273 | constraintsNumber = b_working.rows(); 274 | stepLength << -result.head(thetaNumber); 275 | lambdaStar = result.tail(constraintsNumber); 276 | //for debug 277 | cout << "[INFO]: current stepLength is:" << endl << stepLength << endl; 278 | cout << "[INFO]: current lambdaStar is:" << endl << lambdaStar << endl; 279 | 280 | if ( ( stepLength.array().abs() < 1e-10 ).all() == 1 ) { 281 | if ((lambdaStar.array() > 0).all() == 1) { 282 | break; 283 | } 284 | else { 285 | VectorXf::Index minRow, minCol; 286 | float min_lambda = lambdaStar.minCoeff(&minRow, &minCol); 287 | 288 | //add the deleted constraints into deactive set 289 | addRow(A_deactive, A_working.row(minRow)); 290 | addRow(b_deactive, b_working.row(minRow)); 291 | 292 | //delete the constraints from working set 293 | removeRow(A_working, minRow); 294 | removeRow(b_working, minRow); 295 | cout << "[INFO] " << i << ". loop. A constraint is deleted from working set and added in deactive set." << endl; 296 | i++; 297 | } 298 | } 299 | else { 300 | float alpha = 0.0; 301 | alpha = calculationAlpha(A_working, b_working, A_deactive, b_deactive, stepLength, theta, i); 302 | cout << "[INFO]" << i << ". loop.Current alpha is :" << endl << alpha << endl; 303 | theta = theta + alpha * stepLength; 304 | cout << "[INFO]" << i << ". loop.Current Theta is :" < 2 | 3 | 4 | inequation 5 | 6 | 7 | -------------------------------------------------------------------------------- /python/.idea/inspectionProfiles/profiles_settings.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 6 | -------------------------------------------------------------------------------- /python/.idea/misc.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 6 | 7 | -------------------------------------------------------------------------------- /python/.idea/modules.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | -------------------------------------------------------------------------------- /python/.idea/python.iml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 11 | -------------------------------------------------------------------------------- /python/.idea/vcs.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | -------------------------------------------------------------------------------- /python/.idea/workspace.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 13 | 14 | 19 | 20 | 21 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 42 | 43 | 44 | 45 | 46 | 66 | 67 | 68 | 88 | 89 | 90 | 110 | 111 | 112 | 132 | 133 | 134 | 135 | 136 | 137 | 138 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 1589796384322 148 | 162 | 163 | 164 | 165 | 167 | 168 | 179 | 180 | 181 | 182 | 183 | 184 | 185 | 186 | -------------------------------------------------------------------------------- /python/EQP_Solution.py: -------------------------------------------------------------------------------- 1 | # A has a dimension of (m,n) m row, n column. 2 | # Each row contain parameters of constraints. There are totally m constraints. 3 | # b has a dimension of (m,1) m row, 1 column 4 | # G has a dimension of (n,n) n row, n column 5 | # d has a dimension of (n,1) n row, 1 column 6 | # theta has a dimension of (n,1) n row, 1 column 7 | 8 | import numpy as np 9 | 10 | def EQP_Solution (A, b, G, d, theta): 11 | g = np.dot(G, theta) + d 12 | h = np.dot(A, theta) - b 13 | 14 | # concatenate the matrix into a KKT-Matrix and [g;h] vector 15 | [row, col] = np.shape(A) 16 | K_temp1 = np.concatenate((G, np.transpose(A)), axis=1) 17 | zeros_matrix = np.zeros([row, row]) 18 | K_temp2 = np.concatenate((A, zeros_matrix), axis=1) 19 | 20 | K = np.concatenate((K_temp1, K_temp2), axis=0) 21 | g_h = np.concatenate((g, h), axis=0) 22 | 23 | # calculation the step length and lambda number 24 | K_inv = np.linalg.inv(K) 25 | result = np.dot(K_inv, g_h) 26 | 27 | # from result separate the step length and lambda value 28 | step_length = -result[0:col, 0].reshape(col, 1) 29 | lambda_star = result[col:, 0].reshape(row, 1) 30 | 31 | return step_length, lambda_star -------------------------------------------------------------------------------- /python/SQP_main.py: -------------------------------------------------------------------------------- 1 | # Author : Muqian Chen 2 | # Data : 18.05.2020 3 | # Purpose: Realisation of algorithms for sequential quadratic programming for python version 4 | # I hope i can success. 5 | 6 | # 27.01.2021 7 | # After cross testing with MATLAB is proved that the algorithm EQP can work fine. 8 | 9 | # 28.01.2021 10 | # Basic functions are totally implemented. The rest work is improvement of the program 11 | # to adapt with equation constraints. 12 | 13 | # 29.01.2021 14 | # All works are finished. This program can solve a linear optimization problem subjected 15 | # to equations and inequations constraints. Maybe it could also be expanded to nonlinear 16 | # situation if taylor expansion is used. I will finish this functionality some days. 17 | 18 | 19 | import numpy as np 20 | from EQP_Solution import EQP_Solution as eqp 21 | from compare_blocking import calculation_alpha 22 | import os 23 | 24 | 25 | def if_working(A_inequation, b_inequation, theta): 26 | [row, col] = np.shape(b_inequation) 27 | A = np.zeros(shape=(0, 2), dtype=float) 28 | b = np.zeros(shape=(0, 1), dtype=float) 29 | 30 | # select all a_i' * theta = b_i then save in temporary variables A and b 31 | for i in range(row): 32 | temp = np.dot(A_inequation[i, :], theta) 33 | if temp == b_inequation[i, :]: 34 | A = np.vstack((A, A_inequation[i, :])) 35 | b = np.vstack((b, b_inequation[i, :])) 36 | return A, b 37 | 38 | 39 | 40 | def main(): 41 | 42 | G = np.array([[2.0, 0.0], 43 | [0.0, 2.0]]) 44 | d = np.array([[-2.0], 45 | [-5.0]]) 46 | theta = np.array([[2.0], 47 | [0.0]]) 48 | 49 | A_equation = np.zeros(shape= (0,2)) 50 | b_equation = np.zeros(shape= (0,1)) 51 | [row_equation, col_equation] = np.shape(b_equation) 52 | 53 | 54 | A_inequation = np.array([[ 1.0, -2.0], 55 | [-1.0, -2.0], 56 | [-1.0, 2.0], 57 | [1.0, 0.0], 58 | [0.0, 1.0]]) 59 | b_inequation = np.array([[-2.0], 60 | [-6.0], 61 | [-2.0], 62 | [ 0.0], 63 | [ 0.0]]) 64 | 65 | 66 | [A, b] = if_working(A_inequation, b_inequation, theta) 67 | 68 | A_working = np.vstack((A_equation, A)) 69 | b_working = np.vstack((b_equation, b)) 70 | 71 | 72 | 73 | while True: 74 | [step_length, lambda_star] = eqp(A_working, b_working, G, d, theta) 75 | 76 | judge_step_length = np.array([step_length < 10e-13]) 77 | if judge_step_length.all(): 78 | # to determine whether all lambdas are all biggest than zeros 79 | judge_lambda = np.array([lambda_star > 0]) 80 | if judge_lambda.all(): 81 | break 82 | else: 83 | lambda_min_index = np.argmin(lambda_star) 84 | b_working = np.delete(b_working, lambda_min_index, axis=0) 85 | A_working = np.delete(A_working, lambda_min_index, axis=0) 86 | print("[INFO] delete the constraint represented by the most negative one ") 87 | else: 88 | [alpha, b_blocked, A_blocked] = calculation_alpha(A_inequation, A_working, 89 | b_inequation, b_working, 90 | step_length, theta, row_equation) 91 | # If alpha is unequal to one, it could just be illustrated as that some 92 | # constraint is blocking the heading direction. So use the returned b 93 | # and A to update the working set. 94 | if alpha != 1: 95 | A_working = np.vstack((A_working, A_blocked)) 96 | b_working = np.vstack((b_working, b_blocked)) 97 | print("[INFO] update the working set with blocking constraint") 98 | 99 | # At the end of the loop, execute gradient descent 100 | theta = theta + alpha * step_length 101 | 102 | print("[INFO] The optimal parameter is :theta_1 = {}, theta_2 = {}".format(theta[0, 0], theta[1, 0])) 103 | 104 | 105 | 106 | 107 | 108 | 109 | if __name__ == '__main__': 110 | main() 111 | 112 | 113 | # use np.empty to prepare an empty array to save 114 | # but first of all is that you should make it clear 115 | # in which dimension you want to expand the array. 116 | # look at this example, actually i want to expand 117 | # the array in row but not in column. So I initialize 118 | # the empty array with 0 row. And because the array 119 | # that I want to add in has a dimension of (n, 1), the 120 | # dimension of the empty array should be also (n, 1). 121 | # test = np.empty((0, 1)) 122 | # print(test) 123 | # test = np.vstack((test, b)) 124 | # print(test) 125 | 126 | #test = np.array([[1]]) 127 | #print(np.shape(test)) 128 | #for i in range(1): 129 | # print(i) 130 | 131 | #b_test = np.array([[0]]) 132 | #A_test = np.array([[0, 1]]) 133 | #b_index = np.where(b_inequation == b_test) 134 | #A_index = np.where(A_inequation == A_test) 135 | 136 | #test = np.zeros(0, dtype=int) 137 | #print(test) 138 | #test = np.hstack((test, b_index[0])) 139 | #print(test) 140 | #print(type(test)) 141 | #print(b_inequation[b_index[0], :]) 142 | #print(b_inequation[test, :]) 143 | 144 | 145 | #print(b_index[0]) 146 | #print(A_index[0]) 147 | 148 | #exist_index = np.array([A_index[0] == b_index[0][1]]) 149 | #print(exist_index) 150 | 151 | #if exist_index.any(): 152 | # print("yes") 153 | #else: 154 | # print("no") 155 | 156 | # test = np.delete(A_inequation, np.array([1, 2]), axis=0) 157 | # print(test) 158 | 159 | #print(index) 160 | #print(index[0]) 161 | #print(type(index[0])) 162 | #print(b_inequation[index])#.reshape(-1, 2)) 163 | #b_test = np.delete(b_inequation, index[0], axis=0) 164 | #print(b_test) 165 | 166 | 167 | 168 | #b_working = np.delete(b_working, 1, axis=0) 169 | #print(b_working) 170 | # use this method to detect the minimal value index 171 | # and follow this format to locate the values with 172 | # the same index of another matrix 173 | # b_working_ineq = b 174 | # A_working_ineq = A 175 | # min_index = np.argmin(b_working) 176 | # print(min_index) 177 | # print(b_working[min_index, :]) 178 | # print(A[min_index, :]) 179 | 180 | # use this method to delete specified row with 181 | # given index of np.array 182 | # axis=0 means that the program uses the index 183 | # as row index to delete the whole row. 184 | # print(A) 185 | # print(b) 186 | # min_index = np.argmin(b) 187 | # A_test = np.delete(A, min_index, axis=0) 188 | # b_test = np.delete(b, min_index, axis=0) 189 | # print(A_test) 190 | # print(b_test) 191 | 192 | 193 | -------------------------------------------------------------------------------- /python/__pycache__/EQP_Solution.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/monstermuqian/optimization_study/49ac8654c5e3a79c4fce513a6c9174bb5f877c3c/python/__pycache__/EQP_Solution.cpython-38.pyc -------------------------------------------------------------------------------- /python/__pycache__/compare_blocking.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/monstermuqian/optimization_study/49ac8654c5e3a79c4fce513a6c9174bb5f877c3c/python/__pycache__/compare_blocking.cpython-38.pyc -------------------------------------------------------------------------------- /python/compare_blocking.py: -------------------------------------------------------------------------------- 1 | ''' 2 | 28.01.2021 by Muqian Chen 3 | I would like to use this package to include one function that: 4 | 5 | calculation_alpha accepts the current parameter in the working set 6 | to calculate alpha parameter used to multiply with step length for 7 | reaching a gradient descent. And simultaneously manage the working 8 | set when calculating the value smaller than one. 9 | ''' 10 | 11 | 12 | 13 | import numpy as np 14 | 15 | def calculation_alpha(A_inequation, A_working, b_inequation, 16 | b_working, step_length, theta, row_equation): 17 | # split the parts of inequation constraints from current working set 18 | b_working = b_working[row_equation:, :] 19 | A_working = A_working[row_equation:, :] 20 | 21 | # firstly thinking about situation with only unequaled constraints 22 | [row, col] = np.shape(b_working) 23 | exist_index = np.zeros(0, dtype=float) 24 | for i in range(row): 25 | b_temp = b_working[i, :] 26 | A_temp = A_working[i, :] 27 | 28 | # find the index of the same parameters inside 29 | # unequaled constraints 30 | b_index = np.where(b_inequation == b_temp) 31 | A_index = np.where(A_inequation == A_temp) 32 | 33 | # determine the right index from the result 34 | for j in range(len(b_index)): 35 | if_exist = np.array([A_index == b_index[i]]) 36 | if if_exist.any(): 37 | exist_index = np.hstack((exist_index, b_index[i])) 38 | # save the right index in np.array exist_index 39 | 40 | # delete all in the working set existed constraints from 41 | # all unequaled constraints so that the rest parameters 42 | # are all not in working set. 43 | A_inequation = np.delete(A_inequation, exist_index, axis=0) 44 | b_inequation = np.delete(b_inequation, exist_index, axis=0) 45 | 46 | # pick out the parameters that satisfy inequation a_i' * p < 0 47 | b_result = np.zeros((0, 1), dtype=float) 48 | A_result = np.zeros((0, 2), dtype=float) 49 | 50 | # please be careful, I would like to make b_result as a column vector 51 | for i in range(len(b_inequation)): 52 | if np.dot(A_inequation[i, :], step_length) < 0: 53 | b_result = np.vstack((b_result, b_inequation[i, :])) 54 | A_result = np.vstack((A_result, A_inequation[i, :])) 55 | 56 | # if there are not any satisfied constraints parameter inside b_result 57 | # It could just be said that the alpha parameter is one. 58 | # else, just save the b_i - a_i'*theta / a_i' * p results in the list 59 | # calc_result and then get the minimal value 60 | calc_result = [] 61 | length = len(b_result) 62 | print("[INFO] There are {} satisfying compare conditions a_i' * p < 0 " 63 | .format(length)) 64 | if length == 0: 65 | alpha = 1 66 | b_blocked = np.zeros((0, 1), dtype=float) 67 | A_blocked = np.zeros((0, 2), dtype=float) 68 | return alpha, b_blocked, A_blocked 69 | else: 70 | for i in range(len(b_result)): 71 | temp = (b_result[i, 0] - np.dot(A_result[i, :], theta)) / \ 72 | np.dot(A_result[i, :], step_length) 73 | calc_result.append(temp) 74 | min_calc = min(calc_result) 75 | min_index = calc_result.index(min_calc) 76 | if min_calc < 1: 77 | print("[INFO] Oops! Touch the boundary of constraint area !") 78 | alpha = min_calc 79 | b_blocked = b_result[min_index, 0] 80 | A_blocked = A_result[min_index, :] 81 | # return the b_i and a_i' of blocking constraint 82 | return alpha, b_blocked, A_blocked 83 | else: 84 | alpha = 1 85 | b_blocked = np.zeros((0, 1), dtype=float) 86 | A_blocked = np.zeros((0, 2), dtype=float) 87 | return alpha, b_blocked, A_blocked 88 | 89 | 90 | -------------------------------------------------------------------------------- /video_scanner/__pycache__/pyimagesearch.cpython-38.pyc: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/monstermuqian/optimization_study/49ac8654c5e3a79c4fce513a6c9174bb5f877c3c/video_scanner/__pycache__/pyimagesearch.cpython-38.pyc -------------------------------------------------------------------------------- /video_scanner/demo.py: -------------------------------------------------------------------------------- 1 | from imutils.video import VideoStream 2 | import time 3 | import cv2 4 | import imutils 5 | from pyimagesearch import four_point_transform 6 | from skimage.filters import threshold_local 7 | import numpy as np 8 | 9 | 10 | vs = VideoStream(str=0).start() 11 | time.sleep(2.0) 12 | 13 | while True: 14 | frame = vs.read() 15 | frame = imutils.resize(frame, width=800) 16 | #print(frame.shape) 17 | 18 | cv2.imshow("Test Video", frame) 19 | key = cv2.waitKey(1) & 0xFF 20 | if key == ord("q"): 21 | break 22 | elif key == ord("l"): 23 | image = frame 24 | #cv2.imshow('test_screen shot', frame) 25 | ratio = image.shape[0] / 500.0 26 | orig = image.copy() 27 | image = imutils.resize(image, height=500) 28 | 29 | # 将原图换成灰度图，高斯模糊去掉高频噪声，最后用canny算法找出所有edge 30 | gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) 31 | gray = cv2.GaussianBlur(gray, (5, 5), 0.4) 32 | cv2.imshow("gray", gray) 33 | cv2.waitKey(0) 34 | edged = cv2.Canny(gray, 50, 125) 35 | cv2.imshow("edged", edged) 36 | cv2.waitKey(0) 37 | 38 | # 开始寻找这个物体的轮廓 39 | cnts = cv2.findContours(edged.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE) 40 | cnts = imutils.grab_contours(cnts) 41 | cnts = sorted(cnts, key=cv2.contourArea, reverse=True)[:5] 42 | 43 | # 来一个个对比我们在image上定位出的所有轮廓 44 | for c in cnts: 45 | epsilon = 0.02*cv2.arcLength(c, True) 46 | approx = cv2.approxPolyDP(c, epsilon, True) 47 | print(approx) 48 | print(approx.shape) 49 | 50 | if len(approx) == 4: 51 | screenCnt = approx 52 | print(approx) 53 | break 54 | 55 | # 最终得到的检测出来的矩形边界的四点坐标被储存在screenCnt中 56 | # 在这里将图片转换成四点检测模式的基础 57 | cv2.drawContours(image, [screenCnt], -1, (0, 0, 255), 2) 58 | cv2.imshow("Outline", image) 59 | cv2.waitKey(0) 60 | 61 | 62 | 63 | warped = four_point_transform(orig, screenCnt.reshape(4, 2) * ratio) 64 | warped = cv2.cvtColor(warped, cv2.COLOR_BGR2GRAY) 65 | T = threshold_local(warped, 25, offset=10, method="gaussian") 66 | warped = (warped > T).astype("uint8") * 255 67 | 68 | 69 | cv2.imshow("Scanned", imutils.resize(warped, height=300)) 70 | cv2.waitKey(0) 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | cv2.destroyAllWindows() 79 | vs.stop() -------------------------------------------------------------------------------- /video_scanner/file.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/monstermuqian/optimization_study/49ac8654c5e3a79c4fce513a6c9174bb5f877c3c/video_scanner/file.jpg -------------------------------------------------------------------------------- /video_scanner/file_scan.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/monstermuqian/optimization_study/49ac8654c5e3a79c4fce513a6c9174bb5f877c3c/video_scanner/file_scan.jpg -------------------------------------------------------------------------------- /video_scanner/pyimagesearch.py: -------------------------------------------------------------------------------- 1 | # 接下来要编写一个4点定位一张图片的工具 2 | 3 | import numpy as np 4 | import cv2 5 | 6 | # 这个首要问题是，这个pts是个什么玩意 7 | # This function takes a single argument, pts , which is a list of four points 8 | # specifying the (x, y) coordinates of each point of the rectangle. 9 | 10 | def order_points(pts): 11 | # 在这一步初始化一个容器，以“左上” “右上” “右下” “左下” 也就是顺时针顺序来记录点的坐标 12 | rect = np.zeros((4, 2), dtype="float32") 13 | 14 | # 故意设置 左上的点有最小值（从pts中提出的axis 1 上的和中的最小值 就是 x+y）， 右下的点有最大值 15 | s = pts.sum(axis=1) 16 | rect[0] = pts[np.argmin(s)] 17 | rect[2] = pts[np.argmax(s)] 18 | 19 | # 在计算了pts中的axis 1 上的各值之间的差后，右上的点储存最小的差值（就是每组x-y），左下储存最大的差值 20 | diff = np.diff(pts, axis=1) 21 | rect[1] = pts[np.argmin(diff)] 22 | rect[3] = pts[np.argmax(diff)] 23 | 24 | # 如是做，则可以返回一组被排好序的坐标，总之因为排好一定顺序的坐标是非常重要的，下面的函数就会指出这点 25 | return rect 26 | 27 | def four_point_transform(image, pts): 28 | 29 | rect = order_points(pts) 30 | (tl, tr, br, bl) = rect 31 | 32 | # 在这里算出新得到的image的宽，就挑最大的宽度 33 | widthA = np.sqrt((bl[0] - br[0]) ** 2 + (bl[1] - br[1]) ** 2) 34 | widthB = np.sqrt((tl[0] - tr[0]) ** 2 + (tl[1] - tr[1]) ** 2) 35 | maxWidth = max(int(widthA), int(widthB)) 36 | 37 | # 按照相同的原则算出新得到的image的长 38 | heightA = np.sqrt((bl[0] - tl[0]) ** 2 + (bl[1] - tl[1]) ** 2) 39 | heightB = np.sqrt((br[0] - tr[0]) ** 2 + (br[1] - tr[1]) ** 2) 40 | maxHeight = max(int(heightA), int(heightB)) 41 | 42 | 43 | # 在得到新图像的尺寸之后储存在max们中后，利用他们去得到新图像的四角坐标，也是按照从左上开始顺时针方向 44 | dst = np.array([ 45 | [0, 0], 46 | [maxWidth-1, 0], 47 | [maxWidth - 1, maxHeight - 1], 48 | [0, maxHeight - 1] 49 | ], dtype="float32") 50 | 51 | # 最后利用cv中的函数，先算出将原图经过透视变换后变成从正上方的俯瞰图所需的Transform Matrix M 52 | M = cv2.getPerspectiveTransform(rect, dst) 53 | # 再利用warpPerspective函数去得到最后的原图换成俯瞰图的成像 54 | warped = cv2.warpPerspective(image, M, (maxWidth, maxHeight)) # 这个应该是指定了最后进行了俯瞰化后的image应该有多大 55 | 56 | return warped 57 | 58 | -------------------------------------------------------------------------------- /video_scanner/scanner.py: -------------------------------------------------------------------------------- 1 | from pyimagesearch import four_point_transform # 别看有红线，但是是可以通过编译的 2 | from skimage.filters import threshold_local 3 | import numpy as np 4 | import argparse 5 | import cv2 6 | import imutils 7 | 8 | if False: 9 | ap = argparse.ArgumentParser() 10 | ap.add_argument("-i", "--image", required=True, help="Path to the image to be scanned") 11 | args = vars(ap.parse_args()) 12 | image = cv2.imread(args["image"]) 13 | else: 14 | image = cv2.imread("file.jpg") 15 | # cv2.imshow("original image", image) 16 | # cv2.waitKey(0) 17 | 18 | 19 | print(image.shape) 20 | ratio = image.shape[0] / 500.0 # 读取了图片的宽 21 | orig = image.copy() 22 | image = imutils.resize(image, height=500) 23 | #cv2.imshow("resize image", image) 24 | #cv2.waitKey(0) 25 | 26 | # 换成灰度图，高斯模糊一波去除高频噪声，然后detect edge 27 | gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) 28 | gray = cv2.GaussianBlur(gray, (5, 5), 1) 29 | edged = cv2.Canny(gray, 20, 400) 30 | 31 | # 展示出处理好的原图跟其边界检测图 32 | print("[INFO] STEP 1: Edge Detection") 33 | cv2.imshow("Original Image", image) 34 | cv2.imshow("Edge Detection", edged) 35 | cv2.waitKey(0) 36 | cv2.destroyAllWindows() 37 | 38 | 39 | # 下面开始找到这个需要检测的物件的轮廓 40 | 41 | cnts = cv2.findContours(edged.copy(), cv2.RETR_LIST, cv2.CHAIN_APPROX_SIMPLE) 42 | cnts = imutils.grab_contours(cnts) 43 | cnts = sorted(cnts, key=cv2.contourArea, reverse=True)[:5] 44 | #screenCnt = np.array([]) 45 | 46 | # 来一个个对比我们找到的image上的所有轮廓 47 | for c in cnts: 48 | # 先拟合出轮廓 49 | peri = cv2.arcLength(c, True) 50 | approx = cv2.approxPolyDP(c, 0.02 * peri, True) 51 | print("object detected") 52 | print(approx) 53 | print(approx.shape) 54 | 55 | 56 | # 如果说拟合出的这个轮廓刚好有4点，那么就说我们找到了我们的目标screen 57 | if len(approx) == 4: 58 | screenCnt = approx 59 | print(approx) 60 | print(approx.shape) 61 | break 62 | 63 | 64 | # 发生了很有趣的现象，我使用的都是同一种参数方法，但是，若原图保持竖直，则可以成功检测出轮廓，若原图横过来，怎么做都不能成功 65 | print("[INFO] STEP 2: Find contours of paper") 66 | cv2.drawContours(image, [screenCnt], -1, (0, 0, 255), 2) 67 | cv2.imshow("Outline", image) 68 | cv2.waitKey(0) 69 | cv2.destroyAllWindows() 70 | 71 | 72 | # 在这里开始使用将图片转变为俯瞰模式的算法 73 | warped = four_point_transform(orig, screenCnt.reshape(4, 2) * ratio) 74 | 75 | # 再将新图片转成灰度图进行 滤波操作 来给他加上 black and white paper effect 76 | warped = cv2.cvtColor(warped, cv2.COLOR_BGR2GRAY) 77 | T = threshold_local(warped, 11, offset=10, method="gaussian") 78 | warped = (warped > T).astype("uint8") * 255 79 | 80 | print("[INFO] STEP 3: Apply perspective transformation") 81 | cv2.imshow("Original", imutils.resize(orig, height=650)) 82 | cv2.imshow("Scanned", imutils.resize(warped, height=650)) 83 | cv2.waitKey(0) 84 | cv2.imwrite("file_scan.jpg", imutils.resize(warped, height=650)) -------------------------------------------------------------------------------- /video_scanner/test_video.py: -------------------------------------------------------------------------------- 1 | from imutils.video import VideoStream 2 | import numpy as np 3 | import imutils 4 | import time 5 | import cv2 6 | 7 | 8 | print("[INFO] Loading the model.....") 9 | net = cv2.dnn.readNetFromCaffe("deploy.prototxt.txt", "res10_300x300_ssd_iter_140000.caffemodel") 10 | 11 | print("[INFO] starting video stream...") 12 | 13 | # 若要使用树莓派摄像头，则在这里进行改动，改动详情参考imutils库 14 | vs = VideoStream(src=0).start() 15 | time.sleep(2.0) 16 | 17 | 18 | while True: 19 | 20 | frame = vs.read() 21 | frame = imutils.resize(frame, width=400) 22 | 23 | (h, w) = frame.shape[:2] # 取数据frame的shape的前两位，来表示高和宽 24 | blob = cv2.dnn.blobFromImage(cv2.resize(frame, (300, 300)), 1.0, (300, 300), (104.0, 177.0, 123.0)) 25 | 26 | net.setInput(blob) 27 | detections = net.forward() 28 | print("[INFO] For validating the result from the net....") 29 | print("[INFO] the data type of it is :") 30 | print(type(detections)) 31 | print("[INFO] the content of it is :") 32 | print(detections[0, 0, 0, 0:3]) 33 | print("[INFO] the shape of it is :") 34 | print(detections.shape) 35 | # 到这里完全弄懂这个 detection 出来的结构是什么，真正储存可以解读的结果的地方在detection的第四维一共七个数字上 36 | # 前两位啥都不是，就单单是0跟1，第三位是置信概率，第四位到第七位是标定检测结构的矩形的对角两个顶点的，一共4个坐标值 37 | 38 | 39 | for i in range(0, detections.shape[2]) : 40 | confidence = detections[0, 0, i, 2] 41 | 42 | if confidence < 0.8: 43 | continue 44 | 45 | box = detections[0, 0, i, 3:7] * np.array([w, h, w, h]) 46 | (startX, startY, endX, endY) = box.astype("int") 47 | cv2.rectangle(frame, (startX, startY), (endX, endY), (0, 255, 0), 2) 48 | 49 | y = startY - 10 if startY - 10 > 10 else startY + 10 50 | cv2.putText(frame, "{:.2f}%".format(confidence*100), (startX, y), cv2.FONT_HERSHEY_COMPLEX, 0.45, 51 | (0, 0, 225), 2) 52 | 53 | 54 | 55 | cv2.imshow("Test of the result", frame) 56 | key = cv2.waitKey(1) & 0xFF 57 | if key == ord("q"): 58 | break 59 | 60 | 61 | cv2.destroyAllWindows() 62 | vs.stop() 63 | --------------------------------------------------------------------------------