├── Calibrartion_sree .ipynb ├── LICENSE ├── README.md ├── misc ├── Images ├── a.png ├── c.png ├── imageedit_1_9820914905.png ├── imageedit_3_4070819195.png └── imageedit_5_5151063270.png ├── pt_corres.mat ├── rubik_2D_pts.mat ├── rubik_3D_pts.mat └── rubik_cube.jpg /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 sreec 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ## Camera Calibration 2 | 3 | 4 | The goal of camera calibration is to find the intrinsic and extrinsic parameters of a camera. 5 | 6 | **Overview** 7 | 8 | 9 | Inorder to calibrate the camera we image a 3D object such as a patterned cube and use the 3D-2D point correspondences between the 3D object and its 2d image to find the camera parameters. 10 | 11 | There are two set of parameters that we need to find: intrinsic parameters and extrinsic parameters. Intrinsic parameters are those parameters which are internal to the camera such as focal length, principal points, etc., whereas extrinsic parameters are those parameters that prescribe the location t (translation vector) and orientation R (rotation matrix) of the camera with respect to an external coordinate system (usually called the world coordinate system). In the first part, we will compute only the intrinsic parameters (assuming that the extrinsic parameters are known) and in the second part we will jointly compute the intrinsic and extrinsic parameters. 12 | 13 | 14 | **Intrinsic parameter computation** 15 | 16 | 17 | The calibrating object that we use is a Rubik’s cube. 18 | 19 | We image the cube as shown in figure below. We then obtain many 3D-2D point correspondences. In this part, we have already computed the point correspondences and all you have to do is to compute the intrinsic parameters from them. The 3D-2D correspondences are given in the data file “pt_corres.mat”. This file contains “pts_2D”, the 2D points and “cam_pts_3D”, all the corresponding 3D points.Now we have to find the K matrix 20 | ![alt text](https://github.com/sreenithy/Camera-Calibration/blob/master/misc/a.png) 21 | **K Matrix** 22 | 23 | 24 | The matrix K which relates the 3D to the 2D points is an upper triangular matrix with the following shape. 25 | ![alt text](https://github.com/sreenithy/Camera-Calibration/blob/master/misc/imageedit_1_9820914905.png "K Matrix") 26 | 27 | where αx, αy represents the focal length in terms of the x and y pixel dimensions,px and py are the principal 28 | points and s is the skew parameter. 29 | 30 | 31 | The relation between the 2D point (x, y) and the corresponding 3D point (X, Y, Z) are given by 32 | 33 | 1. x = αx(X/Z) + s(Y/Z) + px 34 | 35 | 2. y = αy(Y/Z) + py 36 | 37 | **Intrinsic and extrinsic parameter computation** 38 | 39 | In the previous part, we have computed the intrinsic parameter assuming that the extrinsic parameters are known, i.e., we assumed that we know the 3D point correspondences in the camera coordinate system. But this is rarely the case; almost always we know the 3D point correspondences only in the world coordinate system and hence we need to estimate both the intrinsic and extrinsic parameters. But before that we need to obtain the 3D-2D point correspondences. The 3D points are described with respect to a world coordinate system as shown in the figure 1. The figure shows the x,y and z axes of the world coordinate system along with some sample 3D points, which are the corners of the squares. There are 28 such points. 40 | 41 | 1. The 3D points in the world coordinate system are provided in rubik_3D_pts.mat and the corresponding 2D points on the image are provided in rubik_2D_pts.mat 42 | 43 | 2. Next we want to compute the camera projection matrix P = K[R t], where K is the internal/intrinsic calibration matrix, R is the rotation matrix which specifies the orientation of the camera coordinate system w.r.t the world coordinate system and t is the translation vector which species the location of the camera center in the world coordinate system. 44 | 45 | 3. For computing P, the code is in the function “calibrate(x,X)” and is based on the “Direct Linear Transformation (DLT)" 46 | The DLT is an important algorithm to understand and is detailed below. 47 | 48 | **Discrete Linear Transform** 49 | 50 | The Discrete Linear Transorm(DLT) is simple linear algorithm for estimating the camera projection matrix 51 | **P** from corresponding 3-space and image entities. This computation of the camera matrix is known as resectioning. 52 | The simplest such correspondence is that between a 3D point **X** and its image **x** under the unknown 53 | camera mapping. Given sufficiently many such correspondences the camera matrix may be determined. 54 | 55 | **Algorithm** 56 | 57 | Let’s assume a number of point correspondences between 3D points and 2D image points are given. The 58 | camera matrix is a 3x4 matrix which relates the points by, xi = P.Xi For each correspondence Xi ↔ xi 59 | , we get three equations of which two are linearly independent and is described below 60 | 61 | 62 | **Steps** 63 | 64 | 1. From a set of n point correspondences, we obtain a 2nx12 matrix A by stacking up the equations of the 65 | above form for each correspondence 66 | 67 | 2. Obtain the SVD of A. The unit singular vector corresponding to the smallest singular value is the solution 68 | p. Specifically, if A = UDVT with D diagonal with positive diagonal entries, arranged in descending order 69 | down the diagonal, then p is the last column of V 70 | 71 | 3. Obtain p and write it in matrix form to get the matrix P 72 | 73 | ![alt text](https://github.com/sreenithy/Camera-Calibration/blob/master/misc/imageedit_5_5151063270.png ) 74 | 75 | The projection matrix P is computed by solving the set of equations Ap = 0, where p is the vector containing 76 | the entries of the matrix P. 77 | 78 | 79 | **Minimum number of point correspondences needed for computing P** 80 | 81 | The 3x4 matrix P has 12 elements but the scale is arbitrary and therefore 11 degrees of freedom. Since each 82 | point correspondence gives 2 equations, at a minimum 5.5 such correspondences are required to solve for 83 | P. The 0.5 indicates that only one of the equations is used from the sixth point, that is we choose either the 84 | x-coordinate or the y-coordinate of the sixth image point. 85 | 86 | 87 | With this minimum number of correspondences, the solution is exact and is obtained by solving Ap = 0 where 88 | A is an 11x12 matrix in this case. 89 | 90 | 91 | If the data is not exact,the n ≥ 6 point correspondences are given, then there will not be an exact solution and 92 | we solve by minimizing an algebraic or geometric error. 93 | 94 | **Obtaining parameters K, R and t from the projection matrix P** 95 | 96 | The decomposition of P into K,R,t is done as by RQ decomposition. It involves calculating the decomposition A 97 | = R Q where Q is unitary/orthogonal and R upper triangular. 98 | 99 | **Verify the accuracy of the computed parameters** 100 | 101 | For this we will compute the re-projection error, which is a measure of the distance between the 2D points and the 2D points obtained by projecting the 3D points using the computed camera parameters. 102 | ![alt text](https://github.com/sreenithy/Camera-Calibration/blob/master/misc/c.png "Plot of the given 2D points and the re-projected 2D points in the original figure of the Rubik’s cube, 103 | here the red circles denote the given 2D points and the green are the re-projected points") 104 | 105 | 106 | **References** 107 | 108 | 109 | [1] R. Hartley and A. Zissermann, Multiview geometry, 2nd edition, Cambridge University Press. 110 | 111 | 112 | [2] Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11):1330-1334, 2000. 113 | -------------------------------------------------------------------------------- /misc/Images: -------------------------------------------------------------------------------- 1 | 2 | -------------------------------------------------------------------------------- /misc/a.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sreenithy/Camera-Calibration/64a28470fb2dfcabb3e4078d30a5d5c496222852/misc/a.png -------------------------------------------------------------------------------- /misc/c.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sreenithy/Camera-Calibration/64a28470fb2dfcabb3e4078d30a5d5c496222852/misc/c.png -------------------------------------------------------------------------------- /misc/imageedit_1_9820914905.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sreenithy/Camera-Calibration/64a28470fb2dfcabb3e4078d30a5d5c496222852/misc/imageedit_1_9820914905.png -------------------------------------------------------------------------------- /misc/imageedit_3_4070819195.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sreenithy/Camera-Calibration/64a28470fb2dfcabb3e4078d30a5d5c496222852/misc/imageedit_3_4070819195.png -------------------------------------------------------------------------------- /misc/imageedit_5_5151063270.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sreenithy/Camera-Calibration/64a28470fb2dfcabb3e4078d30a5d5c496222852/misc/imageedit_5_5151063270.png -------------------------------------------------------------------------------- /pt_corres.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sreenithy/Camera-Calibration/64a28470fb2dfcabb3e4078d30a5d5c496222852/pt_corres.mat -------------------------------------------------------------------------------- /rubik_2D_pts.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sreenithy/Camera-Calibration/64a28470fb2dfcabb3e4078d30a5d5c496222852/rubik_2D_pts.mat -------------------------------------------------------------------------------- /rubik_3D_pts.mat: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sreenithy/Camera-Calibration/64a28470fb2dfcabb3e4078d30a5d5c496222852/rubik_3D_pts.mat -------------------------------------------------------------------------------- /rubik_cube.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sreenithy/Camera-Calibration/64a28470fb2dfcabb3e4078d30a5d5c496222852/rubik_cube.jpg --------------------------------------------------------------------------------