├── .gitignore ├── README.md ├── blog.md ├── camera_calibration ├── camera_calibration.sln └── camera_calibration │ ├── camera_calobration.vcxproj │ ├── camera_calobration.vcxproj.filters │ └── camera_calobration.vcxproj.user ├── images ├── 1.png ├── left01.jpg ├── left02.jpg ├── left03.jpg ├── left04.jpg ├── left05.jpg ├── left06.jpg ├── left07.jpg ├── left08.jpg ├── left09.jpg ├── left11.jpg ├── left12.jpg ├── left13.jpg ├── left14.jpg └── pinhole_camera_model.png ├── modules ├── class_camera_calibrator.hpp └── class_nonlinear_optimizer.hpp ├── props ├── boost1.71.props ├── ceres_release_x64.props ├── console_debug_release_x64 .props ├── dll_debug_release_x64.props ├── eigen3.props ├── glog_release_x64.props ├── opencv412_release_x64.props └── pcl180_release_x64.props └── samples └── main.cpp /.gitignore: -------------------------------------------------------------------------------- 1 | build/ 2 | *.cfg 3 | *.weights 4 | *.engine 5 | *.pdb 6 | x64/ 7 | ipch/ 8 | data/ 9 | *.sdf 10 | *.opensdf 11 | *.exp 12 | *.db 13 | *.ipdb 14 | *.dll 15 | *.lib 16 | *.table 17 | api/ 18 | *.iobj 19 | *.opendb 20 | *.exe 21 | .vs/ 22 | .vscode/ 23 | *.log -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | 2 | # camera calibration cpp 3 | 4 | ### Introduction 5 | 6 | The implementation of camera calibration based on zhang's method.There is a chinese tutorial about this implementaion at current directory. 7 | 8 | ### Dependency 9 | 10 | - opencv(just get corners) 11 | - Eigen3 12 | - ceres 13 | 14 | If you want to build this project on windows, you perhaps need [windows-ceres](https://github.com/tbennun/ceres-windows) project. 15 | 16 | ### Reference 17 | 18 | [1] Burger, Wilhelm. "Zhang’s camera calibration algorithm: in-depth tutorial and implementation." Hagenberg, Austria (2016). 19 | 20 | [2] Hartley, Richard, and Andrew Zisserman. Multiple view geometry in computer vision. Cambridge university press, 2003. 21 | 22 | 23 | [3] https://zhuanlan.zhihu.com/p/24651968 24 | 25 | [4] http://ceres-solver.org/index.htm 26 | 27 | [5] Zhang, Zhengyou. "A flexible new technique for camera calibration." IEEE Transactions on pattern analysis and machine intelligence 22.11 (2000): 1330-1334. -------------------------------------------------------------------------------- /blog.md: -------------------------------------------------------------------------------- 1 | # 相机标定原理与c++实现（张氏标定法） 2 | 3 | https://github.com/enazoe/camera_calibration_cpp 4 | ## 前言 5 | 6 | 最近在做相机标定方面的工作，虽然以前多次进行相机标定，但是多数时候是调用opencv的函数，过程相对简单。虽然对标定过程有一定的了解，但是掌握的不是很扎实，所以趁这次机会，对相机标定过程做了一下深入的了解，并且用c++重新实现了下整个标定过程（opencv 中有源码，但是有点乱我没太看懂，有可能我比较渣渣），所以这是篇学习笔记。其中有些自己的理解，所以难免有错误的地方，还望指正，抱拳。 7 | 8 | 小学时候大家都做过小孔成像实验，当孔足够小的时候，在小孔的另一端我们能看到物体的倒立成像，但是当小孔变大的时候图像的不同部分发出的光线会在孔另一端的屏幕上重叠成像就不清晰了。现代相机正是依据小孔成像原理制作的，由于亮度等其他原因，使用镜头代替了小孔。同时镜头又给成像带来了其他问题，比如说畸变。 9 | 10 | ## 标定的目的 11 | 12 | 相机的作用是把看到的3D世界转换成2D图像，相当于我的输入是三维数据经过一个函数把他变成二维数据。我们想找到这个理想的函数，那么从数学角度先对这个问题进行建模，也就是相机的成像模型+畸变模型，而标定的目的就是通过观察到的数据，去无线逼近这个函数，也就是逼近这个理想模型的参数。 13 | 14 | ## 相机模型（透视投影模型，成像过程） 15 | 16 | 在看相机模型前先明确几个坐标系 17 | 18 | __世界坐标系：__ 单位m，三维世界的坐标系，用户在描述真实世界位置而引入的坐标系，后面我们把棋盘板坐标系看作世界坐标系。 19 | 20 | __相机坐标系：__ 单位m，通常原点在光心，z轴与光轴平行，为了描述物体和相机的相对位置而引入。 21 | 22 | __图像坐标系：__ 单位m，原点位于sensor中心，比如ccd相机的sensor中心，xy轴分别平行于sensor的两个边。 23 | 24 | __像素坐标系：__ 单位像素，这个是我们最常见的用于描述图片的，比如从相机读取出来的图片，原点在图片左上角。 25 | 26 | ![](images/1.png) 27 | 28 | 图1 29 | 30 | [source:mathworks](https://ww2.mathworks.cn/help/vision/ug/camera-calibration.html) 31 | 32 | 上图是各坐标系间的关系，$O_w$为世界坐标系，$O_c$（图中$\alpha$处）为相机坐标系，$O_i$为图像坐标系，$O_p$为像素坐标系，下面逐步分析坐标系间的转换关系。 33 | 34 | ### 世界坐标系 -> 相机坐标系 35 | 36 | 世界坐标系到相机坐标系是一个刚体变换过程，刚体变换可以用一个旋转矩阵（_也可以为旋转向量、欧拉角、四元数,程序中对外参进行优化时就是利用旋转向量这种紧凑的表示方法进行优化，后文会提到_）和平移向量来表示。 37 | 38 | $$ \left[\begin{array}{c} 39 | x_c \\ 40 | y_c\\ 41 | z_c\\ 42 | 1\\ 43 | \end{array}\right]=\underbrace{\left[\begin{array}{cccc} 44 | r_{11} & r_{12} & r_{13} & t_{x} \\ 45 | r_{21} & r_{22} & r_{23} & t_{y} \\ 46 | r_{31} & r_{32} & r_{33} & t_{z} \\ 47 | 0 & 0 & 0 & 1 48 | \end{array}\right]}_W \cdot\left[\begin{array}{l} 49 | x_w \\ 50 | y_w \\ 51 | z_w \\ 52 | 1 53 | \end{array}\right] \quad \cdots 式1$$ 54 | 55 | $r_{3\times3}$为旋转矩阵，$t_{3\times1}$为平移向量。这个刚体变换矩阵$W$就是我们常说的相机外参，和相机本身没有太大关系。至此我们得到了世界坐标系到相机坐标系下的变换关系： 56 | $$P_c = W \cdot P_w$$ 57 | 本文中以标定版坐标系为世界坐标系，所以$z_w=0$,所以$W$的第三列可以省略，用更紧凑的形式表示。 58 | 59 | 60 | ### 相机坐标系 -> 理想无畸变的图像坐标系 61 | 62 | ![](images/pinhole_camera_model.png) 63 | 64 | 图2 65 | 66 | [source:opencv](https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html) 67 | 68 | 图中蓝色的坐标轴（xy）是图像坐标系，橘黄色的坐标轴（uv）为像素坐标系，两个坐标系重合并且和相机坐标系（$F_c,$）处在同一侧,这里可能会造成一些疑惑，实际相机的成像是在光心（相机坐标系原点）的另一侧（对称），也就是在sensor成倒立图像，但是为了数学上的方便描述和计算方便，所以对图像坐标系做了个对调。 69 | 70 | 根据相似三角形原理，图像坐标系下点$P$，焦距$f$,可得到对应的相机坐标系下的点为： 71 | 72 | $$x=f \cdot \frac{X}{Z} \quad y=f \cdot \frac{Y}{Z} \quad \cdots 式2$$ 73 | 74 | 向量话形式为： 75 | 76 | $$p=\left[\begin{array}{l} 77 | x \\ 78 | y 79 | \end{array}\right]=\frac{f}{Z} \cdot\left[\begin{array}{l} 80 | X \\ 81 | Y 82 | \end{array}\right] \quad \cdots 式3$$ 83 | 84 | 式2，3是在笛卡尔坐标系下对透视变换的描述（非线性变换），我们也可以在齐次坐标系下对这个过程进行线性描述。 85 | $$\underbrace{\left[\begin{array}{l} 86 | x \\ 87 | y 88 | \end{array}\right]}_p = \frac{f}{Z} \cdot\left[\begin{array}{c} 89 | X \\ 90 | Y 91 | \end{array}\right] \equiv\left[\begin{array}{c} 92 | f X / Z \\ 93 | f Y / Z \\ 94 | 1 95 | \end{array}\right] \equiv\left[\begin{array}{c} 96 | f X \\ 97 | f Y \\ 98 | Z 99 | \end{array}\right]=\underbrace{\left[\begin{array}{cccc} 100 | f & 0 & 0 & 0 \\ 101 | 0 & f & 0 & 0 \\ 102 | 0 & 0 & 1 & 0 103 | \end{array}\right]}_{\mathbf{M}_{\mathrm{P}}} \cdot \underbrace{\left[\begin{array}{l} 104 | X \\ 105 | Y \\ 106 | Z \\ 107 | 1 108 | \end{array}\right]}_{P_c}$$ 109 | 110 | 其中： 111 | 112 | $$\mathbf{M}_{\mathrm{P}}=\left(\begin{array}{cccc} 113 | f & 0 & 0 & 0 \\ 114 | 0 & f & 0 & 0 \\ 115 | 0 & 0 & 1 & 0 116 | \end{array}\right)=\underbrace{\left(\begin{array}{ccc} 117 | f & 0 & 0 \\ 118 | 0 & f & 0 \\ 119 | 0 & 0 & 1 120 | \end{array}\right)}_{\mathbf{M}_{f}} \cdot \underbrace{\left(\begin{array}{cccc} 121 | 1 & 0 & 0 & 0 \\ 122 | 0 & 1 & 0 & 0 \\ 123 | 0 & 0 & 1 & 0 124 | \end{array}\right)}_{\mathbf{M}_{0}}$$ 125 | 126 | $M_f$为理想的焦距为$f$的针孔相机模型，且$M_0$通常被称为standard (or 127 | canonical) projection matrix.至此，我们得到了从相机坐标系到理想图像坐标系的转化关系： 128 | $$ 129 | p = M_p \cdot P_c 130 | $$ 131 | 132 | ### 图像坐标系 -> 像素坐标系 133 | 134 | 上面我们得到了点在相机坐标系下的坐标，要想得到实际图片的坐标的话，还要进行进一步的变换，可以理解为仿射变换。那么我们需要知道，xy方向上两个坐标系的尺度变换系数；由于像素坐标系的原点在图像的左上角，而图像坐标系的原点在图像的中心$(u_c,v_c)$。所以，我们引入尺度因子$s$,$s_x=1/d_x$，$d_x$为$x$方向上像素的尺寸（最小感光元件的尺寸，单位为$m$）,那么我们可以得到像素坐标系下的坐标为： 135 | 136 | $$ 137 | \left[ \begin{array}{c} 138 | u\\v\\1 139 | \end{array} 140 | \right]= 141 | \left[\begin{array}{ccc} 142 | s_{x} & 0 & u_{c} \\ 143 | 0 & s_{y} & v_{c} \\ 144 | 0 & 0 & 1 145 | \end{array}\right] \cdot \underbrace{ \left[ 146 | \begin{array}{c} 147 | x\\y\\1 148 | \end{array} \right]}_p 149 | $$ 150 | 151 | ### 相机坐标系->实际的图像坐标系 152 | 153 | __相机的畸变过程就发生在相机坐标系到图像坐标系这个变换过程__，所以我们也是在这个过程中引入畸变模型。相机畸变主要分为两类，径向畸变和切向畸变。径向畸变是由于透镜形状的制造工艺导致。且越向透镜边缘移动径向畸变越严重。 154 | 155 | $$\left\{\begin{array}{l} 156 | \mathrm{x}_{rcorr}=x_{p}\left(1+k_{1} r^{2}+k_{2} r^{4}+k_{3} r^{6}\right) \\ 157 | y_{rcorr}=y_{p}\left(1+k_{1} r^{2}+k_{2} r^{4}+k_{3} r^{6}\right) 158 | \end{array}\right.$$ 159 | 切向畸变是由于透镜和CMOS或者CCD的安装位置误差导致。切向畸变需要两个额外的畸变参数来描述，矫正前后的坐标关系为： 160 | $$\left\{\begin{array}{l} 161 | \mathrm{x}_{tcorr}=x_{p}+\left[2 p_{1} x_{p} y_{p}+p_{2}\left(r^{2}+2 x_{p}^{2}\right)\right] \\ 162 | \mathrm{y}_{tcorr}=y_{p}+\left[p_{1}\left(r^{2}+2 y_{p}^{2}\right)+2 p_{2} x_{p} y_{p}\right] 163 | \end{array}\right.$$ 164 | 165 | 所以，现在我们需要五个参数来描述相机的畸变，我们可以用下式来表示： 166 | $$\tilde{\boldsymbol{x}}=\operatorname{warp}(\boldsymbol{x}, \boldsymbol{k},\boldsymbol{p})$$ 167 | 168 | ### 投影过程总结 169 | 170 | $$\left[\begin{array}{l} 171 | u \\ 172 | v \\ 173 | 1 174 | \end{array}\right]= 175 | \underbrace{\left[\begin{array}{lll} 176 | s_{x} & s_{\theta} & u_{c} \\ 177 | 0 & s_{y} & v_{c} \\ 178 | 0 & 0 & 1 179 | \end{array}\right] 180 | \cdot\underbrace{\left(\begin{array}{lll} 181 | f & 0 & 0 \\ 182 | 0 & f & 0 \\ 183 | 0 & 0 & 1 184 | \end{array}\right)}_{\mathbf{M}_{f}}}_A \cdot 185 | \operatorname{warp}(\boldsymbol{x}, \boldsymbol{k}, \boldsymbol{p})\cdot 186 | M_0\cdot 187 | W \cdot\left[\begin{array}{l} 188 | x_{w} \\ 189 | y_{w} \\ 190 | z_{w} \\ 191 | 1 192 | \end{array}\right]$$ 193 | 194 | 其中： 195 | $$\mathbf{A}=\left(\begin{array}{ccc} 196 | f s_{x} & f s_{\theta} & u_{c} \\ 197 | 0 & f s_{y} & v_{c} \\ 198 | 0 & 0 & 1 199 | \end{array}\right)=\left(\begin{array}{ccc} 200 | \alpha & \gamma & u_{c} \\ 201 | 0 & \beta & v_{c} \\ 202 | 0 & 0 & 1 203 | \end{array}\right)$$ 204 | 上式中不是严格的矩阵乘法，其中涉及极坐标化等。其中$A$即为内参矩阵，$k,p$为畸变系数，$W$为外参。 205 | 206 | ## 标定过程 207 | 208 | 我们已经了解了相机投影模型，下面我们来看下怎么对未知参数进行标定。首先我们用相机采集不同姿态标定板的图像，标定板上带有明显视觉特征点。其中我们以标定板左上角角点为原点，垂直于标定板平面方向为z轴，建立世界坐标系，这样我们就能得到特征点在世界坐标系下的坐标和其在像素坐标系下的坐标。整个过程可分为五部分。 209 | 210 | #### 1. 求单映性矩阵（homography） 211 | 212 | 单映性矩阵用来描述两个平面上的点的映射关系，结合前文提到的图像坐标系到像素坐标系的映射关系，我们有： 213 | 214 | $$ 215 | \left[\begin{array}{l} 216 | u \\ 217 | v \\ 218 | 1 219 | \end{array}\right] = 220 | \lambda 221 | \underbrace{\left[\begin{array}{lll} 222 | f\cdot s_{x} & f\cdot s_{\theta} & u_c \\ 223 | 0 & f\cdotp s_{y} & v_c \\ 224 | 0 & 0 & 1 225 | \end{array}\right]}_A\left[\begin{array}{lll} 226 | r_{11} & r_{12} & t_x \\ 227 | r_{21} & r_{22} & t_y \\ 228 | r_{31} & r_{32} & t_z \\ 229 | \end{array}\right]\left[\begin{array}{l} 230 | x_w \\ 231 | y_w \\ 232 | 1 233 | \end{array}\right] 234 | $$ 235 | 设$H=\left[h1,h2,h3\right]=\lambda\cdotp A\cdot[r_1,r_2,t]$，因为H为3x3矩阵，且参与计算的坐标为其次坐标，所以H有八个自由度，所以一个点对对应两个方程，所以大斯鱼等于四个点即可计算出H矩阵。标定版的特征点一般大于四个，有利于H的优化和数值的稳定；求解H一般情况下使用normalized DLT 算法[1][2]；每个姿态的图像都能计算出一个单映性矩阵，我们可以用已知的点对对H进一步优化，本文使用ceres[4]非线性优化库进行求解。 236 | 237 | 238 | #### 2.计算内参矩阵 239 | H已知，H可以分解为下式： 240 | $h_1=\lambda \cdot A \cdot r_1 \Rightarrow r_1=s\cdot A^{-1}\cdot h_1$ 241 | $h_2=\lambda \cdot A \cdot r_2 \Rightarrow r_2=s\cdot A^{-1}\cdot h_2$ 242 | $h_3=\lambda \cdot A \cdot t \Rightarrow t=s\cdot A^{-1}\cdot h_3$ 243 | 244 | 由于$r_1$,$r_2$为旋转矩阵的两列，所以可以引入两个约束： 245 | - $r_1,r_2$正交:$r_1^T\cdot r_2=r_2^T\cdot r_1=0$ 246 | - $r_1,r_2$的模为1:$|r_1|=|r_2|=1$ 247 | 248 | 用h的展开式替换约束条件中的h： 249 | 250 | $$\begin{aligned} 251 | &r_{1}^{T} \cdot r_{2}=h_{1}^{T}\left(A^{-1}\right)^{T} A^{-1} h_{2}=0\\ 252 | &h_{1}^{T}\left(A^{-1}\right)^{T} A^{-1} h_{1}=h_{2}^{T}\left(A^{-1}\right)^{T} A^{-1} h_{2} 253 | \end{aligned}$$ 254 | 255 | 令 256 | $$ 257 | \mathbf{B}=\left(\mathbf{A}^{-1}\right)^{\top} \cdot \mathbf{A}^{-1}=\left(\begin{array}{lll}B_{0} & B_{1} & B_{3} \\ B_{1} & B_{2} & B_{4} \\ B_{3} & B_{4} & B_{5}\end{array}\right)$$ 258 | 259 | 根据上文得到的$A$，展开$B$可知$B$为对称矩阵，且两个约束条件可变为： 260 | $$\begin{array}{l} 261 | \boldsymbol{h}_{0}^{\top} \cdot \mathbf{B} \cdot \boldsymbol{h}_{1}=0 \\ 262 | \boldsymbol{h}_{0}^{\top} \cdot \mathbf{B} \cdot \boldsymbol{h}_{0}-\boldsymbol{h}_{1}^{\top} \cdot \mathbf{B} \cdot \boldsymbol{h}_{1}=0 263 | \end{array}$$ 264 | 下面我们用一个六维的向量$b$对$B$进行表示： 265 | $$\boldsymbol{b}=\left(B_{0}, B_{1}, B_{2}, B_{3}, B_{4}, B_{5}\right)$$ 266 | 对约束条件完全展开可以得到： 267 | $$\boldsymbol{h}_{p}^{\top} \cdot \mathbf{B} \cdot \boldsymbol{h}_{q}=\boldsymbol{v}_{p, q}(\mathbf{H}) \cdot \boldsymbol{b}$$ 268 | 其中： 269 | $$\boldsymbol{v}_{p, q}(\mathbf{H})=\left(\begin{array}{c} 270 | H_{0, p} \cdot H_{0, q} \\ 271 | H_{0, p} \cdot H_{1, q}+H_{1, p} \cdot H_{0, q} \\ 272 | H_{1, p} \cdot H_{1, q} \\ 273 | H_{2, p} \cdot H_{0, q}+H_{0, p} \cdot H_{2, q} \\ 274 | H_{2, p} \cdot H_{1, q}+H_{1, p} \cdot H_{2, q} \\ 275 | H_{2, p} \cdot H_{2, q} 276 | \end{array}\right)$$ 277 | 278 | 所以，我们的约束条件变为： 279 | $$\left(\begin{array}{c} 280 | \boldsymbol{v}_{0,1}(\mathbf{H}) \\ 281 | \boldsymbol{v}_{0,0}(\mathbf{H})-\boldsymbol{v}_{1,1}(\mathbf{H}) 282 | \end{array}\right) \cdot \boldsymbol{b}=\left(\begin{array}{l} 283 | 0 \\ 284 | 0 285 | \end{array}\right ) \text{or}\mathbf{V} \cdot \mathbf{b}=\mathbf{0}$$ 286 | $b$为六维向量，我们想解出$b$需要至少六个方程，现在一个视角下的$H$对应两个方程，所以如果我们有大于等于三个视角的数据，就可以解出$b$,本文的c++实现采用svd求解$\mathbf{V} \cdot \mathbf{b}=\mathbf{0}$。 287 | 288 | 因为$B$为内参矩阵$A$计算得来，通过cholesky分解，可得到$A$的闭式解： 289 | $$\begin{aligned} 290 | \alpha &=\sqrt{w /\left(d \cdot B_{0}\right)} \\ 291 | \beta &=\sqrt{w / d^{2} \cdot B_{0}} \\ 292 | \gamma &=\sqrt{w /\left(d^{2} \cdot B_{0}\right)} \cdot B_{1} \\ 293 | u_{c} &=\left(B_{1} B_{4}-B_{2} B_{3}\right) / d \\ 294 | v_{c} &=\left(B_{1} B_{3}-B_{0} B_{4}\right) / d 295 | \end{aligned}$$ 296 | 其中， 297 | $$\begin{array}{l} 298 | w=B_{0} B_{2} B_{5}-B_{1}^{2} B_{5}-B_{0} B_{4}^{2}+2 B_{1} B_{3} B_{4}-B_{2} B_{3}^{2} \\ 299 | d=B_{0} B_{2}-B_{1}^{2} 300 | \end{array}$$ 301 | 302 | #### 3.计算外参矩阵 303 | 上面我们计算出了内参矩阵$A$,和单映性矩阵$H$,所以我们可以得到： 304 | $$ 305 | \begin{aligned} 306 | r_0 &=\lambda \cdot A^{-1}\cdot h_0\\ 307 | r_1 &=\lambda \cdot A^{-1}\cdot h_1 \\ 308 | t &=\lambda \cdot A^{-1}\cdot h_2 \\ 309 | \end{aligned} 310 | $$ 311 | 其中，$r$为旋转矩阵的向量，所以列向量正交且模为1，所以： 312 | $$\lambda=\frac{1}{\left\|\mathbf{A}^{-1} \cdot h_{0}\right\|}=\frac{1}{\left\|\mathbf{A}^{-1} \cdot h_{1}\right\|}$$ 313 | $$\boldsymbol{r}_{2}=\boldsymbol{r}_{0} \times \boldsymbol{r}_{1}$$ 314 | 315 | #### 4.计算畸变参数 316 | 本文对畸变参数的计算只考虑二元径向畸变的情况来介绍畸变参数的求解过程。畸变模型可以表示为： 317 | $$\tilde{\boldsymbol{x}}_{i}=\operatorname{warp}\left(\boldsymbol{x}_{i}, \boldsymbol{k}\right) = \boldsymbol{x}_{i} \cdot\left[1+D\left(\left\|\boldsymbol{x}_{i}\right\|, \boldsymbol{k}\right)\right]$$ 318 | 其中，$\tilde{\boldsymbol{x}}_i$为畸变后的点，$x_i$为未发生畸变的点 319 | $$D(r, \boldsymbol{k})=k_{0} \cdot r^{2}+k_{1} \cdot r^{4}=\boldsymbol{k} \cdot\left(\begin{array}{c} 320 | r^{2} \\ 321 | r^{4} 322 | \end{array}\right)$$ 323 | $$r_{i}=\left\|\boldsymbol{x}_{i}-\boldsymbol{x}_{c}\right\|=\left\|\boldsymbol{x}_{i}\right\|=\sqrt{x_{i}^{2}+y_{i}^{2}}$$ 324 | 325 | 所以，我们可以对观察到的点进行建模： 326 | $$ 327 | \tilde{\boldsymbol{u}}_{i, j}-\boldsymbol{u}_{c}=\left(\boldsymbol{u}_{i, j}-\boldsymbol{u}_{c}\right) \cdot\left[1+D\left(r_{i, j}, \boldsymbol{k}\right)\right] 328 | $$ 329 | 化简得到： 330 | $$ 331 | \tilde{\boldsymbol{u}}_{i, j}-u_{i,j}=(u_{i,j}-u_c)\cdot D(r_{i,j},k) 332 | $$ 333 | 其中$\tilde{\boldsymbol{u}}_{i, j}$为观察到的畸变的像素坐标系下的坐标，$u_{i,j}$为理想无畸变下的像素坐标，$u_c$为投影中心。我们的目的是通过拟合畸变参数使我们计算的坐标无线逼近我们观察到的坐标，也就是上式最小化为0。所以你和畸变参数问题就变成了解方程组： 334 | $$\left(\begin{array}{cc} 335 | \left(\dot{u}_{i, j}-u_{c}\right) \cdot r_{i, j}^{2} & \left(\dot{u}_{i, j}-u_{c}\right) \cdot r_{i, j}^{4} \\ 336 | \left(\dot{v}_{i, j}-v_{c}\right) \cdot r_{i, j}^{2} & \left(\dot{v}_{i, j}-v_{c}\right) \cdot r_{i, j}^{4} 337 | \end{array}\right) \cdot\left(\begin{array}{c} 338 | k_{0} \\ 339 | k_{1} 340 | \end{array}\right)=\left(\begin{array}{c} 341 | \dot{u}_{i, j}-u_{i, j} \\ 342 | \dot{v}_{i, j}-v_{i, j} 343 | \end{array}\right)$$ 344 | 可使用SVD，或者QR分解，对参数进行求解。 345 | #### 5.全局微调 346 | 347 | 前面的步骤我们已经得到所有参数，包括内参，外参和畸变参数，但是这些值都是初值，我们需要利用所有数据对参数进行进一步优化。有话的方法是最小化投影误差，即计算到的点和观察到的点的偏差。因为这个过程中涉及到投影变换，畸变模型等，所以这是非线性优化问题，本文的实现使用ceres库对各参数进行优化。其中，将外参中的旋转矩阵转化为更为紧凑的旋转向量进行表示，降低优化难度。 348 | 349 | ## 结论 350 | 351 | 相同样本图片的前提下，本文实现的相机标定结果为： 352 | $$ 353 | \left[ 354 | \begin{array}{lll} 355 | 533.397 & 0.389316 & 342.54 \\ 356 | 0 &533.819 & 233.221\\ 357 | 0 & 0 & 1 \\ 358 | \end{array} 359 | \right] 360 | $$ 361 | opencv calibrateCamera 结果： 362 | $$ 363 | \left[ 364 | \begin{array}{lll} 365 | 532.795 & 0 & 342.4584 \\ 366 | 0 &532.919 &233.9011\\ 367 | 0 & 0 & 1 \\ 368 | \end{array} 369 | \right] 370 | $$ 371 | ## 参考文献 372 | 373 | [1] Burger, Wilhelm. "Zhang’s camera calibration algorithm: in-depth tutorial and implementation." Hagenberg, Austria (2016). 374 | 375 | [2] Hartley, Richard, and Andrew Zisserman. Multiple view geometry in computer vision. Cambridge university press, 2003. 376 | 377 | 378 | [3] https://zhuanlan.zhihu.com/p/24651968 379 | 380 | [4] http://ceres-solver.org/index.htm 381 | 382 | [5] Zhang, Zhengyou. "A flexible new technique for camera calibration." IEEE Transactions on pattern analysis and machine intelligence 22.11 (2000): 1330-1334. -------------------------------------------------------------------------------- /camera_calibration/camera_calibration.sln: -------------------------------------------------------------------------------- 1 |  2 | Microsoft Visual Studio Solution File, Format Version 12.00 3 | # Visual Studio 14 4 | VisualStudioVersion = 14.0.25420.1 5 | MinimumVisualStudioVersion = 10.0.40219.1 6 | Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "camera_calibration", "camera_calibration\camera_calobration.vcxproj", "{BE8C29FF-988A-40DF-AE37-440810A68AB1}" 7 | EndProject 8 | Global 9 | GlobalSection(SolutionConfigurationPlatforms) = preSolution 10 | Debug|x64 = Debug|x64 11 | Debug|x86 = Debug|x86 12 | Release|x64 = Release|x64 13 | Release|x86 = Release|x86 14 | EndGlobalSection 15 | GlobalSection(ProjectConfigurationPlatforms) = postSolution 16 | {BE8C29FF-988A-40DF-AE37-440810A68AB1}.Debug|x64.ActiveCfg = Debug|x64 17 | {BE8C29FF-988A-40DF-AE37-440810A68AB1}.Debug|x64.Build.0 = Debug|x64 18 | {BE8C29FF-988A-40DF-AE37-440810A68AB1}.Debug|x86.ActiveCfg = Debug|Win32 19 | {BE8C29FF-988A-40DF-AE37-440810A68AB1}.Debug|x86.Build.0 = Debug|Win32 20 | {BE8C29FF-988A-40DF-AE37-440810A68AB1}.Release|x64.ActiveCfg = Release|x64 21 | {BE8C29FF-988A-40DF-AE37-440810A68AB1}.Release|x64.Build.0 = Release|x64 22 | {BE8C29FF-988A-40DF-AE37-440810A68AB1}.Release|x86.ActiveCfg = Release|Win32 23 | {BE8C29FF-988A-40DF-AE37-440810A68AB1}.Release|x86.Build.0 = Release|Win32 24 | EndGlobalSection 25 | GlobalSection(SolutionProperties) = preSolution 26 | HideSolutionNode = FALSE 27 | EndGlobalSection 28 | EndGlobal 29 | -------------------------------------------------------------------------------- /camera_calibration/camera_calibration/camera_calobration.vcxproj: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | Debug 6 | Win32 7 | 8 | 9 | Release 10 | Win32 11 | 12 | 13 | Debug 14 | x64 15 | 16 | 17 | Release 18 | x64 19 | 20 | 21 | 22 | {BE8C29FF-988A-40DF-AE37-440810A68AB1} 23 | camera_calobration 24 | 8.1 25 | camera_calibration 26 | 27 | 28 | 29 | Application 30 | true 31 | v140 32 | MultiByte 33 | 34 | 35 | Application 36 | false 37 | v140 38 | true 39 | MultiByte 40 | 41 | 42 | Application 43 | true 44 | v140 45 | MultiByte 46 | 47 | 48 | Application 49 | false 50 | v140 51 | true 52 | MultiByte 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | Level3 82 | Disabled 83 | true 84 | 85 | 86 | 87 | 88 | Level3 89 | Disabled 90 | true 91 | 92 | 93 | 94 | 95 | Level3 96 | MaxSpeed 97 | true 98 | true 99 | true 100 | 101 | 102 | true 103 | true 104 | 105 | 106 | 107 | 108 | Level3 109 | MaxSpeed 110 | true 111 | true 112 | true 113 | 114 | 115 | true 116 | true 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | false 126 | 127 | 128 | 129 | 130 | 131 | -------------------------------------------------------------------------------- /camera_calibration/camera_calibration/camera_calobration.vcxproj.filters: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | {4FC737F1-C7A5-4376-A066-2A32D752A2FF} 6 | cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx 7 | 8 | 9 | {93995380-89BD-4b04-88EB-625FBE52EBFB} 10 | h;hh;hpp;hxx;hm;inl;inc;xsd 11 | 12 | 13 | {67DA6AB6-F800-4c08-8B7A-83BB121AAD01} 14 | rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx;tiff;tif;png;wav;mfcribbon-ms 15 | 16 | 17 | 18 | 19 | Header Files 20 | 21 | 22 | Header Files 23 | 24 | 25 | 26 | 27 | Source Files 28 | 29 | 30 | -------------------------------------------------------------------------------- /camera_calibration/camera_calibration/camera_calobration.vcxproj.user: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | $(SolutionDir)$(Configuration)\ 5 | WindowsLocalDebugger 6 | 7 | -------------------------------------------------------------------------------- /images/1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/1.png -------------------------------------------------------------------------------- /images/left01.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/left01.jpg -------------------------------------------------------------------------------- /images/left02.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/left02.jpg -------------------------------------------------------------------------------- /images/left03.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/left03.jpg -------------------------------------------------------------------------------- /images/left04.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/left04.jpg -------------------------------------------------------------------------------- /images/left05.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/left05.jpg -------------------------------------------------------------------------------- /images/left06.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/left06.jpg -------------------------------------------------------------------------------- /images/left07.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/left07.jpg -------------------------------------------------------------------------------- /images/left08.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/left08.jpg -------------------------------------------------------------------------------- /images/left09.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/left09.jpg -------------------------------------------------------------------------------- /images/left11.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/left11.jpg -------------------------------------------------------------------------------- /images/left12.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/left12.jpg -------------------------------------------------------------------------------- /images/left13.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/left13.jpg -------------------------------------------------------------------------------- /images/left14.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/left14.jpg -------------------------------------------------------------------------------- /images/pinhole_camera_model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/images/pinhole_camera_model.png -------------------------------------------------------------------------------- /modules/class_camera_calibrator.hpp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/enazoe/camera_calibration_cpp/45a1d9de083b5bebdd22398c1ff5e6e54c4b446d/modules/class_camera_calibrator.hpp -------------------------------------------------------------------------------- /modules/class_nonlinear_optimizer.hpp: -------------------------------------------------------------------------------- 1 | #ifndef CLASS_NONLINEAR_OPTIMIZER_HPP_ 2 | #define CLASS_NONLINEAR_OPTIMIZER_HPP_ 3 | 4 | #include 5 | #include 6 | #include 7 | 8 | struct Params 9 | { 10 | Eigen::Matrix3d camera_matrix; 11 | Eigen::VectorXd k; 12 | std::vector vec_rt; 13 | }; 14 | 15 | template 16 | void SymmetricGeometricDistanceTerms(const Eigen::Matrix &H, 17 | const Eigen::Matrix &x1, 18 | const Eigen::Matrix &x2, 19 | T forward_error[2], 20 | T backward_error[2]) 21 | { 22 | typedef Eigen::Matrix Vec3; 23 | Vec3 x(x1(0), x1(1), T(1.0)); 24 | Vec3 y(x2(0), x2(1), T(1.0)); 25 | 26 | Vec3 H_x = H * x; 27 | Vec3 Hinv_y = H.inverse() * y; 28 | 29 | H_x /= H_x(2); 30 | Hinv_y /= Hinv_y(2); 31 | 32 | forward_error[0] = H_x(0) - y(0); 33 | forward_error[1] = H_x(1) - y(1); 34 | backward_error[0] = Hinv_y(0) - x(0); 35 | backward_error[1] = Hinv_y(1) - x(1); 36 | } 37 | double SymmetricGeometricDistance(const Eigen::Matrix3d &H, 38 | const Eigen::Vector2d &x1, 39 | const Eigen::Vector2d &x2) 40 | { 41 | Eigen::Vector2d forward_error, backward_error; 42 | SymmetricGeometricDistanceTerms(H, 43 | x1, 44 | x2, 45 | forward_error.data(), 46 | backward_error.data()); 47 | return forward_error.squaredNorm() + 48 | backward_error.squaredNorm(); 49 | } 50 | 51 | 52 | struct EstimateHomographyOptions { 53 | // Default settings for homography estimation which should be suitable 54 | // for a wide range of use cases. 55 | EstimateHomographyOptions() 56 | : max_num_iterations(50), 57 | expected_average_symmetric_distance(1e-16) {} 58 | 59 | int max_num_iterations; 60 | double expected_average_symmetric_distance; 61 | }; 62 | 63 | // Termination checking callback. This is needed to finish the 64 | // optimization when an absolute error threshold is met, as opposed 65 | // to Ceres's function_tolerance, which provides for finishing when 66 | // successful steps reduce the cost function by a fractional amount. 67 | // In this case, the callback checks for the absolute average reprojection 68 | // error and terminates when it's below a threshold (for example all 69 | // points < 0.5px error). 70 | class TerminationCheckingCallback : public ceres::IterationCallback { 71 | public: 72 | TerminationCheckingCallback(const Eigen::MatrixXd &x1, const Eigen::MatrixXd &x2, 73 | const EstimateHomographyOptions &options, 74 | Eigen::Matrix3d *H) 75 | : options_(options), x1_(x1), x2_(x2), H_(H) {} 76 | 77 | virtual ceres::CallbackReturnType operator()( 78 | const ceres::IterationSummary& summary) { 79 | // If the step wasn't successful, there's nothing to do. 80 | if (!summary.step_is_successful) { 81 | return ceres::SOLVER_CONTINUE; 82 | } 83 | 84 | // Calculate average of symmetric geometric distance. 85 | double average_distance = 0.0; 86 | for (int i = 0; i < x1_.cols(); i++) { 87 | average_distance += SymmetricGeometricDistance(*H_, 88 | x1_.col(i), 89 | x2_.col(i)); 90 | } 91 | average_distance /= x1_.cols(); 92 | 93 | if (average_distance <= options_.expected_average_symmetric_distance) { 94 | return ceres::SOLVER_TERMINATE_SUCCESSFULLY; 95 | } 96 | return ceres::SOLVER_CONTINUE; 97 | } 98 | 99 | private: 100 | const EstimateHomographyOptions &options_; 101 | const Eigen::MatrixXd &x1_; 102 | const Eigen::MatrixXd &x2_; 103 | Eigen::Matrix3d *H_; 104 | }; 105 | class NonlinearOptimizer 106 | { 107 | public: 108 | NonlinearOptimizer() 109 | { 110 | } 111 | ~NonlinearOptimizer() 112 | { 113 | } 114 | 115 | void refine_H(const std::vector &img_pts_, 116 | const std::vector &board_pts_, 117 | const Eigen::Matrix3d &matrix_H_, 118 | Eigen::Matrix3d &refined_H_) 119 | { 120 | Eigen::MatrixXd x1(2, board_pts_.size()); 121 | Eigen::MatrixXd x2(2, img_pts_.size()); 122 | 123 | for (int i = 0; i < board_pts_.size(); ++i) 124 | { 125 | x1(0, i) = board_pts_[i].x; 126 | x1(1, i) = board_pts_[i].y; 127 | } 128 | for (int i = 0; i < img_pts_.size(); ++i) 129 | { 130 | x2(0, i) = img_pts_[i].x; 131 | x2(1, i) = img_pts_[i].y; 132 | } 133 | 134 | 135 | Eigen::Matrix3d H = matrix_H_; 136 | //std::cout << "H:" << H<< std::endl; 137 | // Step 2: Refine matrix using Ceres minimizer. 138 | ceres::Problem problem; 139 | for (int i = 0; i < x1.cols(); i++) 140 | { 141 | HomographySymmetricGeometricCostFunctor 142 | *homography_symmetric_geometric_cost_function = 143 | new HomographySymmetricGeometricCostFunctor(x1.col(i), 144 | x2.col(i)); 145 | 146 | problem.AddResidualBlock( 147 | new ceres::AutoDiffCostFunction< 148 | HomographySymmetricGeometricCostFunctor, 149 | 4, // num_residuals 150 | 9>(homography_symmetric_geometric_cost_function), 151 | NULL, 152 | H.data()); 153 | } 154 | EstimateHomographyOptions options; 155 | options.expected_average_symmetric_distance = 0.02; 156 | // Configure the solve. 157 | ceres::Solver::Options solver_options; 158 | solver_options.linear_solver_type = ceres::DENSE_QR; 159 | solver_options.max_num_iterations = options.max_num_iterations; 160 | solver_options.update_state_every_iteration = true; 161 | 162 | // Terminate if the average symmetric distance is good enough. 163 | TerminationCheckingCallback callback(x1, x2, options, &H); 164 | solver_options.callbacks.push_back(&callback); 165 | 166 | // Run the solve. 167 | ceres::Solver::Summary summary; 168 | ceres::Solve(solver_options, &problem, &summary); 169 | 170 | refined_H_ = H / H(2, 2); 171 | } 172 | 173 | 174 | void refine_all_camera_params(const Params ¶ms_, 175 | const std::vector> &imgs_pts_, 176 | const std::vector> &bords_pts_, 177 | Params &refined_params_) 178 | { 179 | ceres::Problem problem; 180 | Eigen::Matrix3d camera_matrix = params_.camera_matrix; 181 | Eigen::VectorXd k = params_.k; 182 | std::vector vec_rt = params_.vec_rt; 183 | Eigen::VectorXd v_camera_matrix(5); 184 | v_camera_matrix << camera_matrix(0,0), 185 | camera_matrix(0, 1), 186 | camera_matrix(0, 2), 187 | camera_matrix(1, 1), 188 | camera_matrix(1, 2); 189 | double *p_camera = v_camera_matrix.data(); 190 | double *p_k = k.data(); 191 | 192 | //package all rt 193 | std::vector packet_rt; 194 | for (int n = 0; n < params_.vec_rt.size(); ++n) 195 | { 196 | Eigen::AngleAxisd r(vec_rt[n].block<3, 3>(0, 0)); 197 | Eigen::VectorXd rot_vec(r.axis()*r.angle()); 198 | Eigen::VectorXd rt(6); 199 | rt << rot_vec(0), rot_vec(1), rot_vec(2), 200 | vec_rt[n](0, 3), vec_rt[n](1, 3), vec_rt[n](2, 3); 201 | packet_rt.push_back(rt); 202 | } 203 | for (int n = 0; n < params_.vec_rt.size(); ++n) 204 | { 205 | Eigen::MatrixXd x1(2, bords_pts_[n].size()); 206 | Eigen::MatrixXd x2(2, imgs_pts_[n].size()); 207 | 208 | for (int i = 0; i < bords_pts_[n].size(); ++i) 209 | { 210 | x1(0, i) = bords_pts_[n][i].x; 211 | x1(1, i) = bords_pts_[n][i].y; 212 | } 213 | for (int i = 0; i < imgs_pts_[n].size(); ++i) 214 | { 215 | x2(0, i) = imgs_pts_[n][i].x; 216 | x2(1, i) = imgs_pts_[n][i].y; 217 | } 218 | 219 | double *p_rt = &packet_rt[n](0); 220 | for (int i = 0; i < x1.cols(); i++) 221 | { 222 | ReprojectionError *cost_function = 223 | new ReprojectionError(x2.col(i),x1.col(i)); 224 | 225 | problem.AddResidualBlock( 226 | new ceres::AutoDiffCostFunction< 227 | ReprojectionError, 228 | 2, // num_residuals 229 | 5,2,6>(cost_function), 230 | NULL, 231 | p_camera, 232 | p_k, 233 | p_rt); 234 | } 235 | } 236 | // Configure the solver. 237 | ceres::Solver::Options options; 238 | options.linear_solver_type = ceres::DENSE_SCHUR; 239 | options.minimizer_progress_to_stdout =false; 240 | 241 | // Solve! 242 | ceres::Solver::Summary summary; 243 | ceres::Solve(options, &problem, &summary); 244 | 245 | DLOG(INFO)<< "Final Brief Report:\n" << summary.BriefReport()<formate_data(v_camera_matrix, k, packet_rt, refined_params_); 247 | } 248 | private: 249 | 250 | void formate_data(const Eigen::VectorXd &v_camera_matrix_, 251 | const Eigen::VectorXd &v_dist_, 252 | const std::vector &v_rt_, 253 | Params ¶ms_) 254 | { 255 | params_.camera_matrix << v_camera_matrix_(0), v_camera_matrix_(1), v_camera_matrix_(2), 256 | 0., v_camera_matrix_(3), v_camera_matrix_(4), 257 | 0, 0, 1.; 258 | params_.k = v_dist_; 259 | params_.vec_rt.clear(); 260 | for (const auto &rt:v_rt_) 261 | { 262 | Eigen::Vector3d rv(rt(0), rt(1), rt(2)); 263 | Eigen::AngleAxisd r_v(rv.norm(), rv/rv.norm()); 264 | Eigen::Matrix rt; 265 | rt.block<3, 3>(0, 0) = r_v.toRotationMatrix(); 266 | rt.block<3, 1>(0, 3) = Eigen::Vector3d(rt(3), rt(4), rt(5)); 267 | params_.vec_rt.push_back(rt); 268 | } 269 | } 270 | 271 | // Cost functor which computes symmetric geometric distance 272 | // used for homography matrix refinement. 273 | struct HomographySymmetricGeometricCostFunctor 274 | { 275 | HomographySymmetricGeometricCostFunctor(const Eigen::Vector2d &x, 276 | const Eigen::Vector2d &y) 277 | : x_(x), y_(y) { } 278 | 279 | template 280 | bool operator()(const T* homography_parameters, T* residuals) const { 281 | typedef Eigen::Matrix Mat3; 282 | typedef Eigen::Matrix Vec2; 283 | 284 | Mat3 H(homography_parameters); 285 | Vec2 x(T(x_(0)), T(x_(1))); 286 | Vec2 y(T(y_(0)), T(y_(1))); 287 | 288 | SymmetricGeometricDistanceTerms(H, 289 | x, 290 | y, 291 | &residuals[0], 292 | &residuals[2]); 293 | return true; 294 | } 295 | 296 | const Eigen::Vector2d x_; 297 | const Eigen::Vector2d y_; 298 | }; 299 | 300 | struct ReprojectionError 301 | { 302 | ReprojectionError(const Eigen::Vector2d &img_pts_, const Eigen::Vector2d &board_pts_) 303 | :_img_pts(img_pts_), _board_pts(board_pts_) 304 | { 305 | } 306 | 307 | template 308 | bool operator()(const T* const instrinsics_, 309 | const T* const k_, 310 | const T* const rt_,//6 : angle axis and translation 311 | T* residuls)const 312 | { 313 | // Eigen::Vector3d hom_w(_board_pts(0), _board_pts(1), T(1.)); 314 | T hom_w_t[3]; 315 | hom_w_t[0] = T(_board_pts(0)); 316 | hom_w_t[1] = T(_board_pts(1)); 317 | hom_w_t[2] = T(1.); 318 | T hom_w_trans[3]; 319 | ceres::AngleAxisRotatePoint(rt_, hom_w_t, hom_w_trans); 320 | hom_w_trans[0] += rt_[3]; 321 | hom_w_trans[1] += rt_[4]; 322 | hom_w_trans[2] += rt_[5]; 323 | 324 | T c_x = hom_w_trans[0] / hom_w_trans[2]; 325 | T c_y = hom_w_trans[1] / hom_w_trans[2]; 326 | 327 | //distortion 328 | T r2 = c_x*c_x + c_y*c_y; 329 | T r4 = r2*r2; 330 | T r_coeff = (T(1) + k_[0] * r2 + k_[1] * r4); 331 | T xd = c_x*r_coeff; 332 | T yd = c_y*r_coeff; 333 | 334 | //camera coord => image coord 335 | T predict_x = instrinsics_[0] * xd + instrinsics_[1] * yd + instrinsics_[2]; 336 | T predict_y = instrinsics_[3] * yd + instrinsics_[4]; 337 | 338 | //residus 339 | 340 | residuls[0] = _img_pts(0) - predict_x; 341 | residuls[1] = _img_pts(1) - predict_y; 342 | return true; 343 | } 344 | const Eigen::Vector2d _img_pts; 345 | const Eigen::Vector2d _board_pts; 346 | }; 347 | 348 | }; 349 | 350 | #endif 351 | -------------------------------------------------------------------------------- /props/boost1.71.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | D:\dependency\boost_1_71_0;%(AdditionalIncludeDirectories) 9 | 10 | 11 | D:\dependency\boost_1_71_0\stage\lib;%(AdditionalLibraryDirectories) 12 | libboost_thread-vc140-mt-x64-1_71.lib;%(AdditionalDependencies) 13 | 14 | 15 | 16 | -------------------------------------------------------------------------------- /props/ceres_release_x64.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | D:\hzr\lib\ceres-windows\ceres-solver\include;D:\hzr\lib\ceres-windows\ceres-solver\config;%(AdditionalIncludeDirectories) 9 | %(PreprocessorDefinitions);GOOGLE_GLOG_DLL_DECL=;_MBCS;CERES_MSVC_USE_UNDERSCORE_PREFIXED_BESSEL_FUNCTIONS 10 | 11 | 12 | D:\hzr\lib\ceres-windows\x64\Release;%(AdditionalLibraryDirectories) 13 | ceres.lib;libglog_static.lib;%(AdditionalDependencies) 14 | 15 | 16 | 17 | -------------------------------------------------------------------------------- /props/console_debug_release_x64 .props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | <_PropertySheetDisplayName>console_debug_release_x64 7 | $(SolutionDir)$(Configuration)\ 8 | $(SolutionDir)$(Platform)\$(Configuration)\ 9 | 10 | 11 | 12 | ..\..\modules;..\..\extra\linx;..\..\extra\linx_timer;%(AdditionalIncludeDirectories) 13 | 14 | 15 | $(SolutionDir)$(Configuration)\;..\..\libs;%(AdditionalLibraryDirectories) 16 | 17 | 18 | 19 | -------------------------------------------------------------------------------- /props/dll_debug_release_x64.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | <_PropertySheetDisplayName>linx_dll_debug_release_x64 7 | $(SolutionDir)$(Configuration)\ 8 | $(SolutionDir)$(Platform)\$(Configuration)\ 9 | 10 | 11 | 12 | ..\..\modules;..\..\extra\linx;%(AdditionalIncludeDirectories) 13 | LINX_API_EXPORTS;%(PreprocessorDefinitions) 14 | 15 | 16 | $(SolutionDir)$(Configuration)\;%(AdditionalLibraryDirectories) 17 | 18 | 19 | 20 | -------------------------------------------------------------------------------- /props/eigen3.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | D:\dependency\Eigen3\include\eigen3;%(AdditionalIncludeDirectories) 9 | 10 | 11 | 12 | -------------------------------------------------------------------------------- /props/glog_release_x64.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | D:\dependency\glog\include;D:\dependency\gflag\include;%(AdditionalIncludeDirectories) 9 | 10 | 11 | D:\dependency\glog\lib;D:\dependency\gflag\lib;%(AdditionalLibraryDirectories) 12 | glog.lib;gflags_static.lib;%(AdditionalDependencies) 13 | 14 | 15 | 16 | -------------------------------------------------------------------------------- /props/opencv412_release_x64.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | D:\dependency\OPENCV412\include;%(AdditionalIncludeDirectories) 9 | 10 | 11 | D:\dependency\OPENCV412\x64\vc14\lib;%(AdditionalLibraryDirectories) 12 | opencv_world412.lib;%(AdditionalDependencies) 13 | 14 | 15 | 16 | -------------------------------------------------------------------------------- /props/pcl180_release_x64.props: -------------------------------------------------------------------------------- 1 |  2 | 3 | 4 | 5 | 6 | 7 | 8 | D:\dependency\PCL180\3rdParty\VTK\include\vtk-7.0;D:\dependency\PCL180\include\pcl-1.8;D:\dependency\PCL180\3rdParty\Eigen\eigen3;D:\dependency\PCL180\3rdParty\Boost\include\boost-1_61;D:\dependency\OpenNI2\Include;D:\dependency\PCL180\3rdParty\FLANN\include;D:\dependency\PCL180\3rdParty\Qhull\include;D:\dependency\PCL180\3rdParty\VTK\include;%(AdditionalIncludeDirectories) 9 | 10 | 11 | D:\dependency\PCL180\3rdParty\Boost\lib\libboost_system-vc120-mt-1_61.lib;D:\dependency\PCL180\3rdParty\Boost\lib\libboost_filesystem-vc120-mt-1_61.lib;D:\dependency\PCL180\3rdParty\Boost\lib\libboost_thread-vc120-mt-1_61.lib;D:\dependency\PCL180\3rdParty\Boost\lib\libboost_date_time-vc120-mt-1_61.lib;D:\dependency\PCL180\3rdParty\Boost\lib\libboost_iostreams-vc120-mt-1_61.lib;D:\dependency\PCL180\3rdParty\Boost\lib\libboost_serialization-vc120-mt-1_61.lib;D:\dependency\PCL180\3rdParty\Boost\lib\libboost_chrono-vc120-mt-1_61.lib;D:\dependency\PCL180\3rdParty\Boost\lib\libboost_atomic-vc120-mt-1_61.lib;D:\dependency\PCL180\3rdParty\Boost\lib\libboost_regex-vc120-mt-1_61.lib;D:\dependency\PCL180\lib\pcl_common_release.lib;D:\dependency\PCL180\lib\pcl_octree_release.lib;D:\dependency\OpenNI2\Lib\OpenNI2.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtksqlite-7.0.lib;D:\dependency\PCL180\lib\pcl_io_release.lib;D:\dependency\PCL180\3rdParty\FLANN\lib\flann_cpp_s.lib;D:\dependency\PCL180\lib\pcl_kdtree_release.lib;D:\dependency\PCL180\lib\pcl_search_release.lib;D:\dependency\PCL180\lib\pcl_sample_consensus_release.lib;D:\dependency\PCL180\lib\pcl_filters_release.lib;D:\dependency\PCL180\lib\pcl_features_release.lib;D:\dependency\PCL180\lib\pcl_ml_release.lib;D:\dependency\PCL180\lib\pcl_segmentation_release.lib;D:\dependency\PCL180\lib\pcl_visualization_release.lib;D:\dependency\PCL180\3rdParty\Qhull\lib\qhullstatic.lib;D:\dependency\PCL180\lib\pcl_surface_release.lib;D:\dependency\PCL180\lib\pcl_registration_release.lib;D:\dependency\PCL180\lib\pcl_keypoints_release.lib;D:\dependency\PCL180\lib\pcl_tracking_release.lib;D:\dependency\PCL180\lib\pcl_recognition_release.lib;D:\dependency\PCL180\lib\pcl_stereo_release.lib;D:\dependency\PCL180\lib\pcl_cuda_features_release.lib;D:\dependency\PCL180\lib\pcl_cuda_segmentation_release.lib;D:\dependency\PCL180\lib\pcl_cuda_sample_consensus_release.lib;D:\dependency\PCL180\lib\pcl_outofcore_release.lib;D:\dependency\PCL180\lib\pcl_gpu_containers_release.lib;D:\dependency\PCL180\lib\pcl_gpu_utils_release.lib;D:\dependency\PCL180\lib\pcl_gpu_octree_release.lib;D:\dependency\PCL180\lib\pcl_gpu_features_release.lib;D:\dependency\PCL180\lib\pcl_gpu_kinfu_release.lib;D:\dependency\PCL180\lib\pcl_gpu_kinfu_large_scale_release.lib;D:\dependency\PCL180\lib\pcl_gpu_segmentation_release.lib;D:\dependency\PCL180\lib\pcl_gpu_surface_release.lib;D:\dependency\PCL180\lib\pcl_people_release.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkDomainsChemistryOpenGL2-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersFlowPaths-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersGeneric-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersHyperTree-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersParallelImaging-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersProgrammable-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersSMP-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersSelection-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersTexture-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersVerdict-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkverdict-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkGeovisCore-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkproj4-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOAMR-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOEnSight-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOExodus-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOExport-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOImport-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOInfovis-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtklibxml2-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOLSDyna-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOMINC-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOMovie-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkoggtheora-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOPLY-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOParallel-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkjsoncpp-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOParallelXML-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOSQL-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOVideo-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkImagingMath-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkImagingMorphological-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkImagingStatistics-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkImagingStencil-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkInteractionImage-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkRenderingContextOpenGL2-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkRenderingImage-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkRenderingLOD-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkRenderingVolumeOpenGL2-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkViewsContext2D-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkViewsInfovis-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkDomainsChemistry-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersAMR-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersParallel-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkexoIIc-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIONetCDF-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkNetCDF_cxx-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkNetCDF-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkhdf5_hl-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkhdf5-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOXML-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOGeometry-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOXMLParser-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkexpat-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkParallelCore-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOLegacy-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkRenderingOpenGL2-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkglew-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkChartsCore-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkRenderingContext2D-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersImaging-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkInfovisLayout-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkInfovisCore-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkViewsCore-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkInteractionWidgets-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkImagingHybrid-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOImage-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkDICOMParser-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkIOCore-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkmetaio-7.0.lib;comctl32.lib;wsock32.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkpng-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtktiff-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkjpeg-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersHybrid-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkImagingGeneral-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkImagingSources-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersModeling-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkInteractionStyle-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkRenderingAnnotation-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkImagingColor-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkRenderingVolume-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkRenderingLabel-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkRenderingFreeType-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkRenderingCore-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkCommonColor-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersExtraction-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersStatistics-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkImagingFourier-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkImagingCore-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkalglib-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersGeometry-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersSources-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersGeneral-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkFiltersCore-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkCommonExecutionModel-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkCommonComputationalGeometry-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkCommonDataModel-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkCommonMisc-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkCommonTransforms-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkCommonMath-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkCommonSystem-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkCommonCore-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtksys-7.0.lib;ws2_32.lib;Psapi.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkfreetype-7.0.lib;D:\dependency\PCL180\3rdParty\VTK\lib\vtkzlib-7.0.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;comdlg32.lib;advapi32.lib;%(AdditionalDependencies) 12 | 13 | 14 | 15 | -------------------------------------------------------------------------------- /samples/main.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include "class_camera_calibrator.hpp" 3 | 4 | 5 | int main() 6 | { 7 | FLAGS_log_dir = "./"; 8 | FLAGS_colorlogtostderr = true; 9 | google::InitGoogleLogging("calibrator"); 10 | google::LogToStderr(); 11 | std::vector images; 12 | cv::glob("../../images/*.jpg", images); 13 | std::vector vec_mat; 14 | for (const auto &path:images) 15 | { 16 | cv::Mat img = cv::imread(path, cv::IMREAD_GRAYSCALE); 17 | vec_mat.push_back(img); 18 | } 19 | 20 | CameraCalibrator m; 21 | Eigen::Matrix3d camera_matrix; 22 | Eigen::VectorXd k; 23 | std::vector vec_extrinsics; 24 | 25 | m.set_input(vec_mat, cv::Size{ 9,6 }); 26 | m.get_result(camera_matrix,k,vec_extrinsics); 27 | 28 | std::cout << "camera_matrix:\n" << camera_matrix << std::endl; 29 | std::cout << "k:\n" << k << std::endl; 30 | //for (int i=0;i