├── .DS_Store ├── README.md ├── base.jpg ├── baseLandmarks.txt ├── delaunay.cpp ├── delaunay.h ├── doc ├── .DS_Store ├── affine.png ├── alignment.jpg ├── avgface.jpg ├── cosmetic.jpg ├── dt.jpg ├── face_morphing.gif ├── faceppICCV.pdf ├── facepp_iccv_ppt.ppt ├── math.jpg └── sdm.pdf ├── imgwarp_mls.cpp ├── imgwarp_mls.h ├── imgwarp_piecewiseaffine.cpp ├── imgwarp_piecewiseaffine.h ├── main.cpp ├── makefile ├── pitch.jpg └── pitchLandmarks.txt /.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/.DS_Store -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ###**Background** 2 | 严格来说，无论是仿射变化(affine transformation)还是图像变形(image warping)其实都不能算在计算机视觉(computer vision)的范畴内，而应该属于计算几何(computational geometry)。自己学习计算几何已经是研究生时候的事情，课程更偏向于对诸如Convex hull, Delaunay triangulation, Voronoi diagram等问题的formulation, algorithm design和complexity analysis，几乎没有涉及任何实际的应用场景。最近研究了Brown University的Computational Photography课程，才发现利用简单的geometry便可以做出各种有意思的图像应用，比如在人脸方面： 3 | 4 | * **Face morphing** 5 | Face morphing翻译过来叫做脸部变形，其实大部分的变形都是指从一个人的脸渐渐变形成另一个人的脸，具体效果如下图。Morphing技术在CG工业界的运用很广泛，比如迪士尼就大量使用morphing来制造廉价的3D造型。 6 | 7 | ![drawing](doc/face_morphing.gif) 8 | (Image from ebnelson's project page, Brown University) 9 | 10 | * **3D Face alignment** 11 | 人脸矫正(face alignment)是指把脸调整到标准的大小，位置和姿态，是人脸识别过程中的第一步，也是最关键的一步。人的面部姿态一般会从roll(平面旋转), pitch(左右侧脸)和yaw(抬头低头)三个维度来描述。平面旋转很容易处理，只需将图片旋转一个角度调整至水平即可。而侧脸和低头处理起来比较有挑战，但通过放射变化也可以较好的解决。 12 | 13 | ![drawing](doc/alignment.jpg) 14 | 15 | * **Average face** 16 | 平均脸是指将许多人脸照片叠在一起，得到一张平均长相的照片。可以用同一个人的多张照片求得此人的标准照，也可以将多个人的照片混合在一起，得到平均的长相。如果使用普通的像素叠加，由于五官无法对齐，图片会变得模糊，而通过仿射将脸调整至标准后，效果便好得多。 17 | 18 | ![drawing](doc/avgface.jpg) 19 | 20 | * **美容应用** 21 | 大部分变形类的图片处理都是通过仿射变换实现的。一个经典的用法就是可以通过改变五官的大小和形状达到美容的效果。 22 | 23 | ![drawing](doc/cosmetic.jpg) 24 | 其实在现实中还有很多例子，比如替换广告牌上的内容（仿射可以将标准的广告牌变成带有透视形变的效果）等等。 25 | 26 | 27 | ###**人脸变形简介** 28 | 简单来说，Affine transformation做的事情就是就是将一个图形映射到另一个位置，在映射过程中图形本身的大小，形状可能发生变化，但仍然保留着图形内部点与点之间的位置关系。如下图所示，正方形ABCD通过变换成为梯形ABCD，而内部的两点x,y之间的相对位置并没有发生改变。 29 | ![drawing](doc/affine.png) 30 | 利用这一点，我们可以将人脸分割成许多小的三角形（因为三角形是可以定义闭合空间的拥有最少顶点的形状），然后利用改变这些三角形及其邻近三角形的位置和大小，达到变形的效果。因此，大部分的人脸变形都可以分为以下几个步骤： 31 | 1. **Landmark detection**: 检测并且标定脸部得landmarks，如眉毛，眼睛，鼻子，嘴，轮廓等等。 32 | 2. **Triangulation**: 基于检测出的landmarks将人脸区域分割为多个三角形。 33 | 3. **Affine transformation**: 对这些三角形进行仿射变换，投影到新的位置。 34 | * *Face morphing, 3D矫正*：将参考人脸(reference)以及需要变换的目标人脸(target)都进行triangulation，将目标人脸上的三角形与参考人脸上的三角形的大小与位置一一对应，再一个一个通过仿射变换映射过去。 35 | * *平均脸*： 和矫正相似，首先对所有照片上的人脸标定，然后求出每个landmark的平均位置，再对这些平均位置做triangulation得到参考脸(reference)。最后将所有照片进行triangulation，全部统一映射到参考脸上三角形的位置。 36 | 37 | 38 | ###Step 1：Landmark detection 39 | 我对Landmark detection的技巧还没有深入研究，最传统的做法是基于Active Appearance Model (AAM)或是Active Shape Model(ASM)的。现在对landmark detection作的比较好的几个机构有： 40 | 41 | **Face++** 42 | * paper: *Extensive Facial Landmark Localization with Coarse-to-fine Convolutional Network Cascade* [ [pdf](doc/faceppICCV.pdf) ] [ [ppt](doc/facepp_iccv_ppt.ppt) ] 43 | * software: [ [matlab](https://github.com/t0nyren/landmarkpp) ] 44 | 45 | **CMU Human Sensing Lab: Intraface** 46 | * paper: *Supervised Descent Method and its Applications to Face Alignment* [ [pdf](doc/sdm.pdf) ] 47 | * software: [ [matlab](http://www.humansensing.cs.cmu.edu/intraface/download_functions_matlab.html) ] [ [C++](http://www.humansensing.cs.cmu.edu/intraface/download_functions_cpp.html) ] 48 | 49 | 50 | ###Step 2：Delaunay Triangulation 51 | Delaunay Triangulation是一种最常见的利用参考点(reference point)将平面分割成三角形的方法。Delaunay Triangulation的标准定义如下： 52 | *"A Delaunay triangulation for a set P of points in a plane is a triangulation DT(P) such that no point in P is inside the circumcircle of any triangle in DT(P). "* 53 | DT的一个重要特性是，它最大化了所有三角形中最小的角度(maximized the minimum angles in all triangles)，因此在使用DT得到的三角形做仿射变换可以防止过大的角度变化而造成的失真。 54 | 计算DT的算法：DT并不是一个NP-hard的问题，有许多高效的算法可以解DT，其中最著名的是Lee and Schachter's algorithm，达到了O(nlogn)的time complexity。值得一提的是，由于DT是Voronoi Diagram的dual graph，所以著名的Fortune's algorithm也可以用来求解DT。 55 | ![drawing](doc/dt.jpg) 56 | 57 | ###Step 3：Affine Transformation 58 | ![math](doc/math.jpg) 59 | 60 | 61 | 62 | ###**代码** 63 | 64 | 65 | ####基于OpenCV的Delaunay Triangulation [ [C++](https://github.com/t0nyren/DelaunayTriangulation) ] 66 | 67 | 68 | ####3D Alignment [ [C++](https://github.com/t0nyren/piecewiseAffine) ] 69 | 70 | 71 | ####平均脸：[ [matlab](https://github.com/t0nyren/AverageFace) ] 72 | -------------------------------------------------------------------------------- /base.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/base.jpg -------------------------------------------------------------------------------- /baseLandmarks.txt: -------------------------------------------------------------------------------- 1 | 19.547222 64.632812 23.477361 69.822396 28.459722 75.054687 34.693958 79.195833 41.756319 83.075000 87.497917 39.029010 86.439583 45.590625 84.828472 51.675937 82.558333 57.461458 79.827778 62.908333 76.981944 68.748438 72.211806 73.953125 67.113958 78.884896 60.319792 82.808333 31.378403 42.277448 31.466597 39.753021 23.170000 40.044427 26.567917 41.439844 35.852639 41.751042 32.504306 39.237500 40.112569 41.381406 31.583819 36.941771 26.993542 37.852865 36.423403 38.255208 18.314514 32.221354 23.328333 30.748125 29.997083 30.977187 36.316250 31.978333 42.426528 32.300052 22.893889 28.956563 29.756944 28.485000 36.756944 29.491927 38.628542 69.097396 51.573750 73.928125 45.524097 69.257812 42.175417 71.818750 46.652569 73.830729 57.029514 69.126042 60.848403 71.032292 55.817639 73.264063 51.651042 69.100521 63.528681 68.873437 51.479167 68.536979 47.641111 65.223958 42.240486 66.608854 46.485972 68.325521 54.793750 65.059896 59.243542 66.510937 57.428125 68.456250 51.472222 65.800000 46.414028 40.185260 43.615903 53.156771 46.224444 60.478125 51.223333 60.588542 56.022500 40.477031 58.036736 52.973958 56.864306 60.202083 41.336875 58.344271 60.733542 58.258333 50.817292 56.018750 71.595833 41.785729 71.007639 39.373333 62.511389 41.164115 66.498264 41.470208 75.704167 40.786458 69.297917 38.910208 78.865278 39.163646 70.797917 36.808125 66.032361 38.076823 75.415278 37.312708 59.937708 32.178490 66.434861 31.985937 72.659722 30.618854 79.025694 29.940365 83.725694 31.459531 65.982639 29.175156 71.968750 28.024271 79.785417 28.015625 -------------------------------------------------------------------------------- /delaunay.cpp: -------------------------------------------------------------------------------- 1 | /* 2 | * delaunay.cpp 3 | * aamlib-opencv 4 | * 5 | * Created by Chen Xing on 10-2-12. 6 | * Copyright 2010 __MyCompanyName__. All rights reserved. 7 | * 8 | */ 9 | 10 | #include "delaunay.h" 11 | using cv::Point2i; 12 | 13 | bool operator <(const Triangle &t1, const Triangle &t2){ 14 | for (int i=0;i<3;i++){ 15 | if ((t1.v[i].xt2.v[i].x) 18 | return false; 19 | else if (t1.v[i].x==t2.v[i].x && t1.v[i].y>t2.v[i].y) 20 | return false; 21 | } 22 | return false; 23 | } 24 | 25 | bool operator< (const TriangleInID &a, const TriangleInID &b) 26 | { 27 | return (a.v[0] *)p1)->x<((Point_ *)p2)->x || 35 | (((Point_ *)p1)->x == ((Point_ *)p2)->x && ((Point_ *)p1)->y<((Point_ *)p2)->y)) 36 | return -1; 37 | else 38 | return 1; 39 | } 40 | 41 | bool FindTriangleFromEdge(CvSubdiv2DEdge e, set< Triangle > &V) 42 | { 43 | CvSubdiv2DEdge t = e; 44 | Triangle triT; 45 | int iPointNum = 3; 46 | int j; 47 | 48 | for(j = 0; j < iPointNum; j++ ){ 49 | CvSubdiv2DPoint* pt = cvSubdiv2DEdgeOrg( t ); 50 | if( !pt ) break; 51 | triT.v[j] = cvPoint( cvRound(pt->pt.x), cvRound(pt->pt.y)); 52 | t = cvSubdiv2DGetEdge( t, CV_NEXT_AROUND_LEFT ); 53 | } 54 | if (j == iPointNum) { 55 | qsort(triT.v,3, sizeof(Point_), pComp); 56 | V.insert(triT); 57 | return true; 58 | } 59 | 60 | return false; 61 | } 62 | -------------------------------------------------------------------------------- /delaunay.h: -------------------------------------------------------------------------------- 1 | /* 2 | * delaunay.h 3 | * aamlib-opencv 4 | * 5 | * Created by Chen Xing on 10-2-12. 6 | * Copyright 2010 __MyCompanyName__. All rights reserved. 7 | * 8 | */ 9 | 10 | #include 11 | #include "opencv2/legacy/legacy.hpp" 12 | #include 13 | #include 14 | #include 15 | using cv::Point_; 16 | using cv::Mat_; 17 | using std::vector; 18 | using std::set; 19 | using std::map; 20 | #include 21 | #include 22 | using std::sort; 23 | 24 | class Delaunay { 25 | }; 26 | 27 | struct Triangle{ 28 | Point_< int > v[3]; 29 | }; 30 | 31 | struct TriangleInID{ 32 | int v[3]; 33 | }; 34 | 35 | int pComp(const void *p1, const void *p2); 36 | 37 | bool FindTriangleFromEdge(CvSubdiv2DEdge e, set< Triangle > &V); 38 | 39 | 40 | //! Find the Delaunay division for given points(Return in int coordinates). 41 | template 42 | vector< Triangle > delaunayDiv(const vector< Point_ > & vP, cv::Rect boundRect) 43 | { 44 | CvSubdiv2D* subdiv; 45 | 46 | CvMemStorage *storage; 47 | storage = cvCreateMemStorage(0); 48 | subdiv = cvCreateSubdivDelaunay2D( boundRect, storage ); 49 | for (size_t e = 0; eedges->total; 55 | int elem_size = subdiv->edges->elem_size; 56 | 57 | cvStartReadSeq( (CvSeq*)(subdiv->edges), &reader, 0 ); 58 | 59 | set< Triangle > V; 60 | 61 | for( i = 0; i < total; i++ ) 62 | { 63 | CvQuadEdge2D* edge = (CvQuadEdge2D*)(reader.ptr); 64 | 65 | if( CV_IS_SET_ELEM( edge )) 66 | { 67 | CvSubdiv2DEdge e = (CvSubdiv2DEdge)edge; 68 | FindTriangleFromEdge(e, V); 69 | 70 | CvSubdiv2DEdge e1 = (CvSubdiv2DEdge)edge+2; //=next[2] 71 | FindTriangleFromEdge(e1, V); 72 | } 73 | CV_NEXT_SEQ_ELEM( elem_size, reader ); 74 | } 75 | cvReleaseMemStorage(&storage); 76 | vector< Triangle > ans; 77 | ans.resize(V.size()); 78 | std::copy(V.begin(), V.end(), ans.begin()); 79 | return ans; 80 | } 81 | 82 | template < class T > 83 | struct PointLess{ 84 | bool operator ()(const Point_ &pa, const Point_ &pb ) const 85 | { 86 | return (pa.x 93 | bool FindTriangleIDFromEdge(CvSubdiv2DEdge e, set< TriangleInID >& V, 94 | map< Point_, int, PointLess >& pMap) 95 | { 96 | CvSubdiv2DEdge t = e; 97 | TriangleInID triT; 98 | int iPointNum = 3; 99 | int j; 100 | 101 | for(j = 0; j < iPointNum; j++ ){ 102 | CvSubdiv2DPoint* pt = cvSubdiv2DEdgeOrg( t ); 103 | if( !pt ) break; 104 | //map< Point_, int, PointLess >::iterator itM= 105 | ; 106 | if (pMap.find(Point_ ( pt->pt.x, pt->pt.y))!=pMap.end()) 107 | triT.v[j] = pMap.find(Point_ ( pt->pt.x, pt->pt.y))->second; 108 | else 109 | return false; 110 | t = cvSubdiv2DGetEdge( t, CV_NEXT_AROUND_LEFT ); 111 | } 112 | if (j == iPointNum) { 113 | sort(triT.v, triT.v+3); 114 | V.insert(triT); 115 | return true; 116 | } 117 | 118 | return false; 119 | } 120 | 121 | //! Find the Delaunay division for given points(Return in point id). 122 | template 123 | vector< TriangleInID > delaunayDivInID(const vector< Point_ > & vP, cv::Rect boundRect) 124 | { 125 | map< Point_, int, PointLess > pMap; 126 | 127 | CvSubdiv2D* subdiv; 128 | 129 | CvMemStorage *storage; 130 | storage = cvCreateMemStorage(0); 131 | subdiv = cvCreateSubdivDelaunay2D( boundRect, storage ); 132 | for (size_t e = 0; eedges->total; 139 | int elem_size = subdiv->edges->elem_size; 140 | 141 | cvStartReadSeq( (CvSeq*)(subdiv->edges), &reader, 0 ); 142 | 143 | set< TriangleInID > V; 144 | 145 | for( i = 0; i < total; i++ ) 146 | { 147 | CvQuadEdge2D* edge = (CvQuadEdge2D*)(reader.ptr); 148 | 149 | if( CV_IS_SET_ELEM( edge )) 150 | { 151 | CvSubdiv2DEdge e = (CvSubdiv2DEdge)edge; 152 | FindTriangleIDFromEdge(e, V, pMap); 153 | 154 | CvSubdiv2DEdge e1 = (CvSubdiv2DEdge)edge+2; //=next[2] 155 | FindTriangleIDFromEdge(e1, V, pMap); 156 | } 157 | CV_NEXT_SEQ_ELEM( elem_size, reader ); 158 | } 159 | cvReleaseMemStorage(&storage); 160 | vector< TriangleInID > ans; 161 | ans.resize(V.size()); 162 | std::copy(V.begin(), V.end(), ans.begin()); 163 | return ans; 164 | } 165 | 166 | template< typename T > 167 | void labelMatByTriInID(const vector< Point_ > & vP, 168 | vector< TriangleInID > &triList, Mat_ &mapMat, 169 | cv::Size labelSize) 170 | { 171 | mapMat.create(labelSize); 172 | mapMat.setTo(triList.size()); 173 | 174 | vector< TriangleInID >::iterator it; 175 | Point_ v[3]; 176 | for (it=triList.begin(); it!=triList.end(); it++){ 177 | //// Not interested in points outside the region. 178 | v[0] = vP[it->v[0]]; 179 | v[1] = vP[it->v[1]]; 180 | v[2] = vP[it->v[2]]; 181 | 182 | cv::fillConvexPoly(mapMat, v, 3, it-triList.begin()); 183 | } 184 | } 185 | -------------------------------------------------------------------------------- /doc/.DS_Store: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/doc/.DS_Store -------------------------------------------------------------------------------- /doc/affine.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/doc/affine.png -------------------------------------------------------------------------------- /doc/alignment.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/doc/alignment.jpg -------------------------------------------------------------------------------- /doc/avgface.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/doc/avgface.jpg -------------------------------------------------------------------------------- /doc/cosmetic.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/doc/cosmetic.jpg -------------------------------------------------------------------------------- /doc/dt.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/doc/dt.jpg -------------------------------------------------------------------------------- /doc/face_morphing.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/doc/face_morphing.gif -------------------------------------------------------------------------------- /doc/faceppICCV.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/doc/faceppICCV.pdf -------------------------------------------------------------------------------- /doc/facepp_iccv_ppt.ppt: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/doc/facepp_iccv_ppt.ppt -------------------------------------------------------------------------------- /doc/math.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/doc/math.jpg -------------------------------------------------------------------------------- /doc/sdm.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/doc/sdm.pdf -------------------------------------------------------------------------------- /imgwarp_mls.cpp: -------------------------------------------------------------------------------- 1 | #include "imgwarp_mls.h" 2 | #include 3 | 4 | using cv::Vec3b; 5 | 6 | ImgWarp_MLS::ImgWarp_MLS() 7 | { 8 | gridSize = 5; 9 | } 10 | 11 | inline double bilinear_interp(double x, double y, 12 | double v11, double v12, double v21, double v22){ 13 | return (v11*(1-y) + v12*y) * (1-x) + (v21*(1-y) + v22*y) * x; 14 | } 15 | 16 | Mat ImgWarp_MLS::setAllAndGenerate(const Mat & oriImg, 17 | const vector< Point_ > &qsrc, 18 | const vector< Point_ > &qdst, 19 | const int outW, const int outH, 20 | const double transRatio){ 21 | setSize(oriImg.cols, oriImg.rows); 22 | setTargetSize(outW, outH); 23 | setSrcPoints(qsrc); 24 | setDstPoints(qdst); 25 | calcDelta(); 26 | return genNewImg(oriImg, transRatio); 27 | } 28 | 29 | 30 | Mat ImgWarp_MLS::genNewImg(const Mat & oriImg, double transRatio){ 31 | int i,j; 32 | double di, dj; 33 | double nx, ny; 34 | int nxi, nyi, nxi1, nyi1; 35 | double deltaX, deltaY; 36 | double w, h; 37 | int ni, nj; 38 | 39 | Mat newImg(tarH, tarW, oriImg.type()); 40 | for (i=0;i=tarH) ni = tarH-1, h = ni-i+1; 45 | if (nj>=tarW) nj = tarW-1, w = nj-j+1; 46 | for (di=0; disrcW-1) nx = srcW-1; 59 | if (ny>srcH-1) ny = srcH-1; 60 | // if (nx>srcH-1) nx = srcH-1; 61 | // if (ny>srcW-1) ny = srcW-1; 62 | if (nx<0) nx = 0; 63 | if (ny<0) ny = 0; 64 | nxi = int(nx); 65 | nyi = int(ny); 66 | nxi1 = ceil(nx); 67 | nyi1 = ceil(ny); 68 | 69 | if (oriImg.channels() == 1) 70 | newImg.at(i+di, j+dj) = 71 | bilinear_interp(ny-nyi, nx-nxi, 72 | oriImg.at(nyi, nxi), oriImg.at(nyi, nxi1), 73 | oriImg.at(nyi1, nxi), oriImg.at(nyi1, nxi1)); 74 | else { 75 | for (int ll=0; ll<3; ll++) 76 | newImg.at(i+di, j+dj)[ll] = 77 | bilinear_interp(ny-nyi, nx-nxi, 78 | oriImg.at(nyi, nxi)[ll], oriImg.at(nyi, nxi1)[ll], 79 | oriImg.at(nyi1, nxi)[ll], oriImg.at(nyi1, nxi1)[ll]); 80 | } 81 | } 82 | } 83 | return newImg; 84 | } 85 | 86 | // Set source points and prepare transformation matrices 87 | void ImgWarp_MLS::setSrcPoints(const vector< Point_< int > > &qsrc) 88 | { 89 | nPoint = qsrc.size(); 90 | 91 | newDotL.clear(); 92 | newDotL.reserve(nPoint); 93 | 94 | 95 | for (size_t i = 0; i > &qdst) 102 | { 103 | nPoint = qdst.size(); 104 | //oldDotL = Mat(nPoint, 2, CV_32FC1); 105 | oldDotL.clear(); 106 | oldDotL.reserve(nPoint); 107 | 108 | for (size_t i = 0; i 5 | #include 6 | using std::vector; 7 | 8 | using cv::Mat; 9 | using cv::Mat_; 10 | using cv::Point_; 11 | using cv::Point; 12 | 13 | //! The base class for Moving Least Square image warping. 14 | /*! 15 | * Choose one of the subclasses, the easiest interface to generate 16 | * an output is to use setAllAndGenerate function. 17 | */ 18 | class ImgWarp_MLS 19 | { 20 | public: 21 | ImgWarp_MLS(); 22 | virtual ~ImgWarp_MLS(){} 23 | 24 | //! Set all and generate an output. 25 | /*! 26 | \param oriImg the image to be warped. 27 | \param qsrc A list of "from" points. 28 | \param qdst A list of "target" points. 29 | \param outW The width of the output image. 30 | \param outH The height of the output image. 31 | \param transRatio 1 means warp to target points, 0 means no warping 32 | 33 | This will do all the initialization and generate a warped image. 34 | After calling this, one can later call genNewImg with different 35 | transRatios to generate a warping animation. 36 | */ 37 | Mat setAllAndGenerate(const Mat & oriImg, 38 | const vector< Point_ > &qsrc, 39 | const vector< Point_ > &qdst, 40 | const int outW, const int outH, 41 | const double transRatio = 1); 42 | 43 | //! Generate the warped image. 44 | /*! This function generate a warped image using PRE-CALCULATED data. 45 | * DO NOT CALL THIS AT FIRST! Call this after at least one call of 46 | * setAllAndGenerate. 47 | */ 48 | Mat genNewImg(const Mat & oriImg, double transRatio); 49 | 50 | //! Calculate delta value which will be used for generating the warped image. 51 | virtual void calcDelta()=0; 52 | 53 | //! Parameter for MLS. 54 | double alpha; 55 | 56 | //! Parameter for MLS. 57 | int gridSize; 58 | 59 | //! Set the list of target points 60 | void setDstPoints(const vector< Point_< int > > &qdst); 61 | 62 | //! Set the list of source points 63 | void setSrcPoints(const vector< Point_< int > > &qsrc); 64 | 65 | //! The size of the original image. For precalculation. 66 | void setSize(int w, int h){srcW=w, srcH=h;} 67 | 68 | //! The size of output image 69 | void setTargetSize(const int outW, const int outH){ 70 | tarW = outW; 71 | tarH = outH; 72 | } 73 | 74 | protected: 75 | 76 | vector< Point_< double > > oldDotL, newDotL; 77 | 78 | int nPoint; 79 | 80 | Mat_ /*! \brief delta_x */rDx, /*! \brief delta_y */rDy; 81 | 82 | int srcW, srcH; 83 | int tarW, tarH; 84 | }; 85 | 86 | #endif // IMGTRANS_MLS_H 87 | -------------------------------------------------------------------------------- /imgwarp_piecewiseaffine.cpp: -------------------------------------------------------------------------------- 1 | #include "imgwarp_piecewiseaffine.h" 2 | #include "delaunay.h" 3 | 4 | #include 5 | 6 | using cv::Point2d; 7 | 8 | ImgWarp_PieceWiseAffine::ImgWarp_PieceWiseAffine(void) 9 | { 10 | backGroundFillAlg = BGNone; 11 | } 12 | 13 | ImgWarp_PieceWiseAffine::~ImgWarp_PieceWiseAffine(void) 14 | { 15 | } 16 | 17 | Point_ ImgWarp_PieceWiseAffine::getMLSDelta(int x, int y) { 18 | static Point_< double > swq, qstar, newP, tmpP; 19 | double sw; 20 | 21 | static vector< double > w; 22 | w.resize(nPoint); 23 | 24 | static Point_< double > swp, pstar, curV, curVJ, Pi, PiJ; 25 | double miu_s; 26 | 27 | int i = x; 28 | int j = y; 29 | int k; 30 | 31 | sw = 0; 32 | swp.x = swp.y = 0; 33 | swq.x = swq.y = 0; 34 | newP.x = newP.y = 0; 35 | curV.x = i; 36 | curV.y = j; 37 | for (k = 0; k < nPoint; k++) { 38 | if ((i==oldDotL[k].x) && j==oldDotL[k].y) 39 | break; 40 | /* w[k] = pow((i-oldDotL[k].x)*(i-oldDotL[k].x)+ 41 | (j-oldDotL[k].y)*(j-oldDotL[k].y), -alpha);*/ 42 | w[k] = 1/((i-oldDotL[k].x)*(i-oldDotL[k].x)+ 43 | (j-oldDotL[k].y)*(j-oldDotL[k].y)); 44 | sw = sw + w[k]; 45 | swp = swp + w[k] * oldDotL[k]; 46 | swq = swq + w[k] * newDotL[k]; 47 | } 48 | if ( k == nPoint ) { 49 | pstar = (1 / sw) * swp ; 50 | qstar = 1/sw * swq; 51 | // qDebug("pstar: (%f, %f)", pstar[0], pstar[1]); 52 | 53 | // Calc miu_s 54 | miu_s = 0; 55 | for (k = 0; k < nPoint; k++) { 56 | if (i==oldDotL[k].x && j==oldDotL[k].y) 57 | continue; 58 | 59 | Pi = oldDotL[k] - pstar; 60 | miu_s += w[k] * Pi.dot(Pi); 61 | } 62 | 63 | curV -= pstar; 64 | curVJ.x = -curV.y, curVJ.y = curV.x; 65 | 66 | for (k = 0; k < nPoint; k++) { 67 | if (i==oldDotL[k].x && j==oldDotL[k].y) 68 | continue; 69 | 70 | Pi = oldDotL[k] - pstar; 71 | PiJ.x = -Pi.y, PiJ.y = Pi.x; 72 | 73 | tmpP.x = Pi.dot(curV) * newDotL[k].x 74 | - PiJ.dot(curV) * newDotL[k].y; 75 | tmpP.y = -Pi.dot(curVJ) * newDotL[k].x 76 | + PiJ.dot(curVJ) * newDotL[k].y; 77 | tmpP *= w[k]/miu_s; 78 | newP += tmpP; 79 | } 80 | newP += qstar; 81 | } 82 | else { 83 | newP = newDotL[k]; 84 | } 85 | 86 | newP.x -= i; 87 | newP.y -= j; 88 | return newP; 89 | } 90 | 91 | void ImgWarp_PieceWiseAffine::calcDelta(){ 92 | Mat_< int > imgLabel = Mat_< int >::zeros(tarH, tarW); 93 | 94 | rDx = rDx.zeros(tarH, tarW); 95 | rDy = rDy.zeros(tarH, tarW); 96 | for (int i=0;inPoint;i++){ 97 | //! Ignore points outside the target image 98 | if (oldDotL[i].x<0) 99 | oldDotL[i].x = 0; 100 | if (oldDotL[i].y<0) 101 | oldDotL[i].y = 0; 102 | if (oldDotL[i].x >= tarW) 103 | oldDotL[i].x = tarW - 1; 104 | if (oldDotL[i].y >= tarH) 105 | oldDotL[i].y = tarH - 1; 106 | 107 | rDx(oldDotL[i]) = newDotL[i].x-oldDotL[i].x; 108 | rDy(oldDotL[i]) = newDotL[i].y-oldDotL[i].y; 109 | } 110 | rDx(0, 0) = rDy(0, 0) = 0; 111 | rDx(tarH-1, 0) = rDy(0, tarW-1) = 0; 112 | rDy(tarH-1, 0) = rDy(tarH-1, tarW-1) = srcH-tarH; 113 | rDx(0, tarW-1) = rDx(tarH-1, tarW-1) = srcW-tarW; 114 | 115 | 116 | vector< Triangle > V; 117 | vector< Triangle >::iterator it; 118 | cv::Rect_ boundRect(0, 0, tarW, tarH); 119 | vector< Point_< double > > oL1 = oldDotL; 120 | if (backGroundFillAlg == BGPieceWise){ 121 | oL1.push_back(Point2d(0,0)); 122 | oL1.push_back(Point2d(0,tarH-1)); 123 | oL1.push_back(Point2d(tarW-1, 0)); 124 | oL1.push_back(Point2d(tarW-1,tarH-1)); 125 | } 126 | // In order preserv the background 127 | V = ::delaunayDiv(oL1, boundRect); 128 | 129 | // vector< TriangleInID > Vt; 130 | // // vector< Triangle >::iterator it; 131 | // // cv::Rect_ boundRect(0, 0, tarW, tarH); 132 | // Vt = ::delaunayDivInID(oldDotL, boundRect); 133 | Mat_ imgTmp = Mat_::zeros(tarH, tarW); 134 | for (it=V.begin(); it!=V.end(); it++){ 135 | cv::line(imgTmp, it->v[0], it->v[1], 255, 1, CV_AA); 136 | cv::line(imgTmp, it->v[0], it->v[2], 255, 1, CV_AA); 137 | cv::line(imgTmp, it->v[2], it->v[1], 255, 1, CV_AA); 138 | 139 | // Not interested in points outside the region. 140 | if (!(it->v[0].inside(boundRect) && it->v[1].inside(boundRect) && it->v[2].inside(boundRect))) 141 | continue; 142 | 143 | cv::fillConvexPoly(imgLabel, it->v, 3, cv::Scalar_(it-V.begin()+1)); 144 | } 145 | //imshow("imgTmp", imgTmp); 146 | //cvWaitKey(10); 147 | 148 | 149 | int i, j; 150 | 151 | Point_< int > v1, v2, curV; 152 | 153 | for (i = 0; ; i+=gridSize){ 154 | if (i>=tarW && i=tarW) 157 | break; 158 | for (j = 0; ; j+=gridSize){ 159 | if (j>=tarH && j=tarH) 162 | break; 163 | int tId = imgLabel(j, i) - 1; 164 | if (tId<0){ 165 | if (backGroundFillAlg == BGMLS){ 166 | Point_ dV = getMLSDelta(i, j); 167 | rDx(j, i) = dV.x; 168 | rDy(j, i) = dV.y; 169 | } 170 | else{ 171 | rDx(j, i) = -i; 172 | rDy(j, i) = -j; 173 | } 174 | continue; 175 | } 176 | v1 = V[tId].v[1] - V[tId].v[0]; 177 | v2 = V[tId].v[2] - V[tId].v[0]; 178 | curV.x = i, curV.y = j; 179 | curV -= V[tId].v[0]; 180 | 181 | double d0, d1, d2; 182 | d2 = double(v1.x * curV.y - curV.x * v1.y)/(v1.x*v2.y-v2.x*v1.y); 183 | d1 = double(v2.x * curV.y - curV.x * v2.y)/(v2.x*v1.y-v1.x*v2.y); 184 | //d1=d2=0; 185 | d0 = 1-d1-d2; 186 | rDx(j, i) = d0*rDx(V[tId].v[0]) + d1*rDx(V[tId].v[1]) + d2*rDx(V[tId].v[2]); 187 | rDy(j, i) = d0*rDy(V[tId].v[0]) + d1*rDy(V[tId].v[1]) + d2*rDy(V[tId].v[2]); 188 | } 189 | } 190 | // qDebug("Calc OK"); 191 | // cout< getMLSDelta(int x, int y); 28 | }; 29 | 30 | #endif //IMGTRANSPIECEWISEAFFINE_H -------------------------------------------------------------------------------- /main.cpp: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include "imgwarp_piecewiseaffine.h" 6 | using namespace std; 7 | using namespace cv; 8 | 9 | #define numLandmarks 83 10 | 11 | int main(){ 12 | ifstream finSrc, finDst; 13 | //read in dst img 14 | Mat src = imread("pitch.jpg", 1); 15 | Rect rect(0, 0, src.cols, src.rows); 16 | Subdiv2D subdiv(rect); 17 | 18 | //parse landmarks 19 | finSrc.open("pitchLandmarks.txt"); 20 | finDst.open("baseLandmarks.txt"); 21 | if (finSrc.fail() || finDst.fail()){ 22 | cout<<"Couldn't open landmarks files"< > srcPoints; 25 | vector > dstPoints; 26 | for (int i = 0; i < numLandmarks; i++){ 27 | float x,y; 28 | finSrc>>x; 29 | finSrc>>y; 30 | srcPoints.push_back(Point_( cvRound(x/100*src.cols),cvRound(y/100*src.rows) )); 31 | finDst>>x; 32 | finDst>>y; 33 | dstPoints.push_back(Point_( cvRound(x/100*src.cols),cvRound(y/100*src.rows) )); 34 | } 35 | 36 | //piecewise affine 37 | ImgWarp_PieceWiseAffine warpper; 38 | warpper.backGroundFillAlg = ImgWarp_PieceWiseAffine::BGMLS; 39 | Mat warpDst = warpper.setAllAndGenerate(src, srcPoints, dstPoints, src.cols, src.rows, 1); 40 | imwrite("affine.jpg", warpDst); 41 | return 0; 42 | } -------------------------------------------------------------------------------- /makefile: -------------------------------------------------------------------------------- 1 | all: 2 | g++ -c main.cpp 3 | g++ -c delaunay.cpp 4 | g++ -c imgwarp_mls.cpp 5 | g++ -c imgwarp_piecewiseaffine.cpp 6 | g++ -o affine main.o delaunay.o imgwarp_mls.o imgwarp_piecewiseaffine.o -lopencv_core -lopencv_highgui -lopencv_imgproc 7 | clean: 8 | rm *.o 9 | rm affine 10 | rm affine.jpg -------------------------------------------------------------------------------- /pitch.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/t0nyren/piecewiseAffine/db43065f8d8808e0ce95dd9020480edf9301ad65/pitch.jpg -------------------------------------------------------------------------------- /pitchLandmarks.txt: -------------------------------------------------------------------------------- 1 | 21.044722 63.191146 23.758264 68.672917 27.498958 74.743229 31.431458 79.632292 36.400556 84.397396 94.956250 40.506615 95.143056 48.390937 94.600694 56.316146 93.288889 63.488021 89.401389 70.504688 83.500000 76.325000 75.053472 80.535938 65.719306 84.241667 55.722222 86.705729 27.759583 41.121510 28.614306 38.364844 21.698333 38.251719 23.935000 40.120417 32.653542 40.832448 31.639028 37.718438 36.759722 41.029896 28.746111 35.343542 24.971319 36.032917 33.620347 36.738333 17.958611 28.977708 22.293542 28.659271 27.232569 29.379062 32.028611 30.397135 37.410139 31.133646 21.900833 26.626927 27.723472 26.848437 32.794028 27.831563 34.082500 69.613021 43.329236 74.599479 38.926875 69.652604 35.914444 71.985937 39.439028 74.016146 51.240833 70.140104 54.133681 72.809896 48.370903 74.284896 44.125903 69.717188 59.022708 70.623958 43.692639 69.026042 40.044028 66.145833 35.869514 67.173438 39.091181 68.802083 46.593819 66.191667 52.618264 68.201042 50.616250 69.357813 43.252431 66.686458 40.779583 39.988646 36.380000 52.817188 38.110972 60.200521 42.591389 60.673958 50.668819 40.314948 51.057778 52.571354 48.363750 60.440104 34.183611 57.881250 54.724306 59.039583 40.228611 55.460417 67.446042 41.963021 66.903889 39.293385 57.480764 40.678802 61.822569 41.514531 71.812500 41.009062 67.307222 39.033229 75.709722 39.678229 67.101458 36.339740 61.706389 37.581823 71.534722 37.352187 53.583611 30.203490 61.337014 30.582708 68.663125 29.804323 76.098611 29.766927 81.895833 31.213490 61.585139 27.452344 68.705139 26.947344 76.450694 28.278594 --------------------------------------------------------------------------------