├── python ├── README.md └── npzread.py ├── syntheticdata ├── town_1_frame_1268.npz ├── town_1_frame_1268.png ├── town_1_frame_2098.npz └── town_1_frame_2098.png ├── matlab ├── computeKviaIACTest.m ├── computeVanishingPointsviaRandK.m ├── computeRviaVanishingPointsandK.m ├── getPitchByIAC.m ├── computeCameraHeight.m └── computeKviaIAC.m ├── external ├── getKfromVanishingPoints.m └── isalmost.m ├── camheightCriminisi.m ├── wagner_test.m ├── vpinvestigation.m ├── README.md ├── abbas_hom.m └── LICENSE /python/README.md: -------------------------------------------------------------------------------- 1 | The idea is to have here the python version of the library 2 | -------------------------------------------------------------------------------- /syntheticdata/town_1_frame_1268.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/carlosbeltran/EyeLib/HEAD/syntheticdata/town_1_frame_1268.npz -------------------------------------------------------------------------------- /syntheticdata/town_1_frame_1268.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/carlosbeltran/EyeLib/HEAD/syntheticdata/town_1_frame_1268.png -------------------------------------------------------------------------------- /syntheticdata/town_1_frame_2098.npz: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/carlosbeltran/EyeLib/HEAD/syntheticdata/town_1_frame_2098.npz -------------------------------------------------------------------------------- /syntheticdata/town_1_frame_2098.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/carlosbeltran/EyeLib/HEAD/syntheticdata/town_1_frame_2098.png -------------------------------------------------------------------------------- /python/npzread.py: -------------------------------------------------------------------------------- 1 | import numpy as np 2 | 3 | data = np.load('town_1_frame_2098.npz', allow_pickle=True) 4 | lst = data.files 5 | 6 | for item in lst: 7 | print(item) 8 | print(data[item]) 9 | -------------------------------------------------------------------------------- /matlab/computeKviaIACTest.m: -------------------------------------------------------------------------------- 1 | 2 | vps_gt = [[ 1195.99492678 192.67503264]; ... 3 | [ 104.98138221 80.65389521]; ... 4 | [ 144.38825531 1614.68566071]]; 5 | 6 | K_gt = [[403.76110881 0. 288.]; ... 7 | [ 0. 403.76110881 216.]; ... 8 | [ 0. 0. 1. ]]; 9 | 10 | v_1 = [vps_gt(1,:),1]; 11 | v_2 = [vps_gt(2,:),1]; 12 | v_3 = [vps_gt(3,:),1]; 13 | 14 | [K,W] = computeKviaIAC(v_1,v_2,v_3); 15 | 16 | K_gt 17 | K -------------------------------------------------------------------------------- /matlab/computeVanishingPointsviaRandK.m: -------------------------------------------------------------------------------- 1 | %% Function Name: computeVanishingPointsviaRandK 2 | % 3 | % Get vanishing point provided the rotation matrix cRw and intrinsics K 4 | % 5 | % $Revision: v1.0$ 6 | % $Author: Carlos Beltran-Gonzalez$ 7 | % $Email: carlos.beltran@iit.it$ 8 | % Istituto Italiano di Tecnologia 9 | % Pattern Analysis and Computer Vision 10 | % $Date: May 16, 2020$ 11 | % Copyright (C) 2020 Carlos Beltran-Gonzalez 12 | % CopyPolicy: GNU Lesser General Public License v3 13 | 14 | function [v1,v2,v3] = computeVanishingPointsviaRandK(R,K) 15 | 16 | v1 = K * R(:,1); 17 | v1 = v1/v1(3) 18 | v2 = K * R(:,2); 19 | v2 = v2/v2(3) 20 | v3 = K * R(:,3); 21 | v3 = v3/v3(3) 22 | 23 | end 24 | 25 | -------------------------------------------------------------------------------- /matlab/computeRviaVanishingPointsandK.m: -------------------------------------------------------------------------------- 1 | %% Function Name: computeRviaVanishingPointsandK 2 | % 3 | % Solves for cRw provided the vanishg points and the intrinsics matrix K 4 | % vanishingpoints are expected as column vectors in normalized homogeneous 5 | % coordinates 6 | % 7 | % $Revision: v1.0$ 8 | % $Author: Carlos Beltran-Gonzalez$ 9 | % $Email: carlos.beltran@iit.it$ 10 | % Istituto Italiano di Tecnologia 11 | % Pattern Analysis and Computer Vision 12 | % $Date: May 16, 2020$ 13 | % Copyright (C) 2020 Carlos Beltran-Gonzalez 14 | % CopyPolicy: GNU Lesser General Public License v3 15 | 16 | function R = computeRviaVanishingPointsandK(K,vp1,vp2) 17 | 18 | r1 = (inv(K)*vp1)/norm((inv(K)*vp1)); 19 | r2 = (inv(K)*vp2)/norm((inv(K)*vp2)); 20 | r3 = cross(r1,r2); 21 | R = horzcat(r1,r2,r3); 22 | 23 | end 24 | 25 | -------------------------------------------------------------------------------- /matlab/getPitchByIAC.m: -------------------------------------------------------------------------------- 1 | %% Function Name: getPitchByIAC 2 | % 3 | % Gets the angle between the optical axis and the vertical vanishg point 4 | % by means of the image of the absolute conic 5 | % K is expected to be column wise 6 | % verticalvp in image coordinates 7 | % angle is returned in degrees 8 | % Note: check page 210 of Hartley and Zisserman " Multiple View Geometry in 9 | % computer vision" [second edition] 10 | % 11 | % $Revision: v1.0$ 12 | % $Author: Carlos Beltran-Gonzalez$ 13 | % $Email: carlos.beltran@iit.it$ 14 | % Istituto Italiano di Tecnologia 15 | % Pattern Analysis and Computer Vision 16 | % $Date: May 16, 2020$ 17 | % Copyright (C) 2020 Carlos Beltran-Gonzalez 18 | % CopyPolicy: GNU Lesser General Public License v3 19 | 20 | function pitchDeg = getPitchByIAC(K,verticalvp) 21 | 22 | x1 = K(:,3); % Image centre (first image point in homogeneus form) 23 | iac = inv(K*K'); %Image of the absolute conic 24 | x2 = [verticalvp,1]'; % Vanishing point (second image point) 25 | costheta = (x1'*iac*x2)/(sqrt(x1'*iac*x1)*sqrt(x2'*iac*x2)); 26 | pitch_iac = rad2deg(pi/2 - acos(costheta)); 27 | pitchDeg = pitch_iac; 28 | 29 | end 30 | 31 | -------------------------------------------------------------------------------- /external/getKfromVanishingPoints.m: -------------------------------------------------------------------------------- 1 | 2 | function K = getKfromVanishingPoints(vp_x,vp_y,vp_z) 3 | %GETKFROMVANISHINGPOINTS Summary of this function goes here 4 | % Get intrinsics matrix from three hortogonal finite vanishing points 5 | 6 | temp_1_x = vp_x(1)+vp_y(1); 7 | temp_2_x = vp_y(1)+vp_z(1); 8 | temp_3_x = vp_z(1)+vp_x(1); 9 | temp_1_y = vp_x(2)+vp_y(2); 10 | temp_2_y = vp_y(2)+vp_z(2); 11 | temp_3_y = vp_z(2)+vp_x(2); 12 | temp_xy = vp_x(1)*vp_y(1)+vp_x(2)*vp_y(2); 13 | temp_yz = vp_z(1)*vp_y(1)+vp_z(2)*vp_y(2); 14 | temp_xz = vp_x(1)*vp_z(1)+vp_x(2)*vp_z(2); 15 | 16 | %solve for image center ax+by+c=0 17 | syms u_0 v_0; 18 | [final_u, final_v] = solve( ... 19 | -(temp_1_x)*u_0 - (temp_1_y)*v_0 + temp_xy == -(temp_2_x)*u_0 - (temp_2_y)*v_0 + temp_yz,... 20 | -(temp_3_x)*u_0 - (temp_3_y)*v_0 + temp_xz == -(temp_2_x)*u_0 - (temp_2_y)*v_0 + temp_yz... 21 | ); 22 | 23 | u = double(final_u); 24 | v = double(final_v); 25 | 26 | %focal length solving 27 | eq_f = (u - vp_x(1))*(u - vp_y(1)) + (v - vp_x(2))*(v - vp_y(2)); 28 | f = sqrt(-eq_f); 29 | f = double(f); 30 | 31 | 32 | K = [f 0 u; 33 | 0 f v; 34 | 0 0 1]; 35 | 36 | end 37 | 38 | -------------------------------------------------------------------------------- /camheightCriminisi.m: -------------------------------------------------------------------------------- 1 | clear all; close all; 2 | addpath('./matlab'); 3 | 4 | Zc_gt = 5.2; 5 | Zt = 1.8; 6 | 7 | % Plot word coordinate frame 8 | Tcam = transl(0,0,Zc_gt)*trotx(deg2rad(-90))*troty(deg2rad(45))*trotx(deg2rad(-35)); 9 | 10 | cam = CentralCamera('focal', 0.015,'pixel',10e-6, ... 11 | 'resolution',[1280 1024],'centre',[640 512],'pose',Tcam); 12 | 13 | world = SE3(); 14 | trplot(world,'frame','0','color','b'); 15 | 16 | xlim([-1 5]); 17 | ylim([-1 5]); 18 | zlim([-0.1 5.5]); 19 | 20 | % plot camera 21 | cam.plot_camera(); 22 | 23 | % Plot points 24 | hold on; 25 | %%P = mkgrid(2,0.2,'pose',transl(0.7,0.7,0)); 26 | %P = mkcube(1.8,'pose',transl([4,4,0.9])); 27 | %P = P(:,[1,5]); 28 | P = [ 4 4; 4 4; 0 Zt]; 29 | plot3(P(1,:),P(2,:),P(3,:),'LineWidth',7); 30 | cam.plot(P); 31 | 32 | B = P(:,1); 33 | T = P(:,2); 34 | 35 | xy = cam.project(P); 36 | 37 | b = [xy(:,1);1]; 38 | t = [xy(:,2);1]; 39 | 40 | R = cam.T.R; 41 | R = R'; 42 | K = cam.K 43 | [v1,v2,v3] = computeVanishingPointsviaRandK(R,K); 44 | 45 | Zc = computeCameraHeight(v1,v2,v3,b,t,Zt); 46 | 47 | % hoz = cross(v1,v2); 48 | % btLine = cross(b,t); 49 | % i = cross(btLine,hoz); 50 | % i = i/i(3); 51 | % 52 | % hoznor = hoz/norm(hoz); 53 | % 54 | % alpha = (-norm(cross(b,t)))/(dot(hoznor,b)*norm(cross(v3,t))*Zt) 55 | % Zc = (-norm(cross(b,i)))/(dot(hoznor,b)*norm(cross(v3,i))*alpha) 56 | % Zc2 = -(inv(dot(hoznor,v3)))/alpha -------------------------------------------------------------------------------- /matlab/computeCameraHeight.m: -------------------------------------------------------------------------------- 1 | %% Function Name: computeCameraHeight 2 | % 3 | % Gets the camera height from the vanishing points and the height of a 4 | % reference object with known botton and top image projections 5 | % REFERENCE: Criminisi, Reid and Zisserman. Single View Metrology. 1999 6 | % 7 | % $Revision: v1.0$ 8 | % $Author: Carlos Beltran-Gonzalez$ 9 | % $Email: carlos.beltran@iit.it$ 10 | % Istituto Italiano di Tecnologia 11 | % Pattern Analysis and Computer Vision 12 | % $Date: May 26, 2020$ 13 | % Copyright (C) 2020 Carlos Beltran-Gonzalez 14 | % CopyPolicy: GNU Lesser General Public License v3 15 | 16 | function Z = computeCameraHeight(v1,v2,v3,b,t,Zt) 17 | 18 | 19 | hoz = cross(v1,v2); % Get the horizon line 20 | 21 | %% eventualy needed for method 1 22 | %btLine = cross(b,t); % Get the line connecting b and t 23 | %%Get the point interception between b->t line and horizon line 24 | %i = cross(btLine,hoz); 25 | %i = i/i(3); % Normalize i 26 | %%%%%% 27 | 28 | hoznor = hoz/norm(hoz); % Normalize the horizon line 29 | 30 | % Compute the scaling factor using the known target height 31 | alpha = (-norm(cross(b,t)))/(dot(hoznor,b)*norm(cross(v3,t))*Zt) 32 | 33 | %Method 1 needs i 34 | %Z = (-norm(cross(b,i)))/(dot(hoznor,b)*norm(cross(v3,i))*alpha) 35 | 36 | % Compute the camara heigh using the previusly compute scaling factor 37 | % Method 2 doesn't need i 38 | Z = -(inv(dot(hoznor,v3)))/alpha 39 | 40 | end 41 | 42 | -------------------------------------------------------------------------------- /external/isalmost.m: -------------------------------------------------------------------------------- 1 | function test = isalmost(a,b,tol) 2 | % 3 | % usage: test = isalmost(a,b,tol) 4 | % 5 | % tests if matrix a is approximately equal to b within a specified 6 | % tolerance interval (b-tol <= a <= b+tol) 7 | % 8 | % note: if b is given as a scalar, all values in a are compared against 9 | % the scalar value b 10 | % 11 | % calls: none 12 | % 13 | % inputs: 14 | % 15 | % a(nr,nc) = matrix of data values to test 16 | % b(nr,nc) = matrix of data values for comparison (or a single scalar value) 17 | % tol = tolerance used in computation 18 | % 19 | % outputs: 20 | % 21 | % test(nr,nc) = matrix of test results: 22 | % 23 | % test(i,j) = 0 -> a(i,j) is not equal to b(i,j) (or is NaN) 24 | % test(i,j) = 1 -> a(i,j) is approximately equal to b(i,j) 25 | % 26 | % author : James Crawford 27 | % created 01/08/2007 28 | % modified by: Carlos Beltran-Gonzalez 29 | % 30 | % 31 | % history: v1.0 (01/08/2007) 32 | % history v2.0 (19/05/2020) 33 | % 34 | % get length of input matrix a 35 | [nr,nc] = size(a); 36 | % check input for consistency 37 | if ~all(size(a) == size(b)) 38 | if all(size(b) == [1 1]) 39 | % convert scalar value b to a matrix of size(a) 40 | b = b*ones(size(a)); 41 | else 42 | disp('error: input arguments are inconsistent (isalmost.m)') 43 | disp('(b) must be a matrix of same size as (a) or a single value') 44 | end 45 | end 46 | one = ones(size(b)); 47 | % perform test 48 | test = (a <= b+tol*one)&(a >= b-tol*one); 49 | test = all(all(test)); 50 | % -------------------------------------------------------------------------------- /wagner_test.m: -------------------------------------------------------------------------------- 1 | clear all; close all; 2 | 3 | % Plot word coordinate frame 4 | Tcam = transl(0,0,4)*trotx(deg2rad(-90))*troty(deg2rad(45))*trotx(deg2rad(-35)); 5 | 6 | cam = CentralCamera('focal', 0.015,'pixel',10e-6, ... 7 | 'resolution',[1280 1024],'centre',[640 512],'pose',Tcam); 8 | 9 | world = SE3(); 10 | trplot(world,'frame','0','color','b'); 11 | 12 | xlim([-1 5]); 13 | ylim([-1 5]); 14 | zlim([-1 5]); 15 | 16 | % plot camera 17 | cam.plot_camera(); 18 | 19 | % Plot points 20 | hold on; 21 | %%P = mkgrid(2,0.2,'pose',transl(0.7,0.7,0)); 22 | P = mkcube(1.8,'pose',transl([4,4,0.9])); 23 | P = P(:,[1,5]); 24 | plot3(P(1,:),P(2,:),P(3,:)); 25 | cam.plot(P); 26 | 27 | xy = cam.project(P); 28 | 29 | R = cam.T.R; 30 | R = R'; 31 | K = cam.K 32 | % 33 | flen = K(1,1); 34 | u0 = K(1,3); 35 | v0 = K(2,3); 36 | r00 = R(1,1); 37 | r01 = R(1,2); 38 | r02 = R(1,3); 39 | r10 = R(2,1); 40 | r11 = R(2,2); 41 | r12 = R(2,3); 42 | r20 = R(3,1); 43 | r21 = R(3,2); 44 | r22 = R(3,3); 45 | 46 | a = flen*r00+u0*r20; 47 | b = flen*r01+u0*r21; 48 | c = flen*r02+u0*r22; 49 | d = (-flen*r02-u0*r22); 50 | e = flen*r10+v0*r20; 51 | f = flen*r11+v0*r21; 52 | g = flen*r12+v0*r22; 53 | h = (-flen*r12-v0*r22); 54 | i = r20; 55 | j = r21; 56 | k = r22; 57 | l = -r22; 58 | 59 | % image plane coordinates for the head 60 | u_g = xy(1,1); 61 | v_g = xy(2,1); 62 | u_t = xy(1,2); 63 | v_t = xy(2,2); 64 | Z_t = 1.8; 65 | 66 | %Ax = b 67 | A = [ (d-u_g*l) (a-u_g*i) (b-u_g*j) 0 0; 68 | (h-v_g*l) (e-v_g*i) (f-v_g*j) 0 0; 69 | (d-u_t*l) 0 0 (a-u_t*i) (b-u_t*j); 70 | (h-v_t*l) 0 0 (e-v_t*i) (f-v_t*j)]; 71 | 72 | B = [ 0 ; 73 | 0 ; 74 | Z_t*(u_t*k-c); 75 | Z_t*(v_t*k-g)]; 76 | 77 | A\B 78 | mldivide(A,B) 79 | -------------------------------------------------------------------------------- /vpinvestigation.m: -------------------------------------------------------------------------------- 1 | clear all; close all; 2 | addpath('./external'); 3 | addpath('./matlab'); 4 | 5 | % Plot word coordinate frame 6 | Tcam = transl(0,0,1)*trotx(deg2rad(-90))*troty(deg2rad(45))*trotx(deg2rad(-45)); 7 | 8 | cam = CentralCamera('focal', 0.015,'pixel',10e-6, ... 9 | 'resolution',[1280 1024],'centre',[640 512],'pose',Tcam); 10 | 11 | world = SE3(); 12 | trplot(world,'frame','0','color','b'); 13 | 14 | % plot camera 15 | cam.plot_camera(); 16 | 17 | % Plot points 18 | hold on; 19 | P = mkgrid(2,0.2,'pose',transl(0.7,0.7,0)); 20 | plot3(P(1,:),P(2,:),P(3,:)); 21 | %figure; 22 | 23 | cam.plot(P); 24 | xy = cam.project(P); 25 | xx = xy(1,:); 26 | yy = xy(2,:); 27 | 28 | figure; 29 | set(gca,'Ydir','reverse') 30 | p1 = [xx(1),yy(1)]; 31 | p2 = [xx(2),yy(2)]; 32 | p3 = [xx(3),yy(3)]; 33 | p4 = [xx(4),yy(4)]; 34 | hold on 35 | plot(p1(1),p1(2),'*r'); 36 | plot(p2(1),p2(2),'*r'); 37 | plot(p3(1),p3(2),'*y'); 38 | plot(p4(1),p4(2),'*y'); 39 | 40 | line1 = cross([p1,1],[p2,1]); 41 | plot(xx(1:2),yy(1:2),'-r'); 42 | line2 = cross([p4,1],[p3,1]); 43 | plot([xx(4);xx(3)],[yy(4);yy(3)],'-r'); 44 | 45 | line3 = cross([p4,1],[p1,1]); 46 | plot([xx(4);xx(1)],[yy(4);yy(1)],'-y'); 47 | line4 = cross([p3,1],[p2,1]); 48 | plot([xx(3);xx(2)],[yy(3);yy(2)],'-y'); 49 | 50 | vp1 = cross(line1,line2); 51 | vp2 = cross(line3,line4); 52 | vp1 = vp1/vp1(3) 53 | vp2 = vp2/vp2(3) 54 | plot(vp1(1)/vp1(3),vp1(2)/vp1(3),'*r'); 55 | plot(vp2(1)/vp2(3),vp2(2)/vp2(3),'*y'); 56 | 57 | rot = cam.T.R; % rotaton part of the "pose" of the camera wRc 58 | rot = rot';% getting cRw 59 | cRw = rot; 60 | [vv1,vv2,vv3] = computeVanishingPointsviaRandK(cRw,cam.K); 61 | vv1 62 | vv2 63 | vv3 64 | 65 | cRw_gt = cam.T.R' 66 | Rest = computeRviaVanishingPointsandK(cam.K,vv1,vv2) 67 | 68 | %% Calculate focal length 69 | 70 | K = computeKviaIAC(vv1,vv2,vv3) 71 | cam.K 72 | assert(isalmost(K,cam.K,1e-10),"Comparison of K matrices") 73 | 74 | -------------------------------------------------------------------------------- /matlab/computeKviaIAC.m: -------------------------------------------------------------------------------- 1 | %% Function Name: computeKviaIAC 2 | % 3 | % Description: Computes the Image of the Absolute Conic (IAC) and the 4 | % intrinsics matrix (K) from three orthogonal finite vanishing points. 5 | % Implements algorithm 8.2 "Computing K from scene and internal 6 | % constraints" in the case of three vanishing points with zero skew and 7 | % square pixels (Example 8.27) from the book Multiple View Geometry in 8 | % Computer Vision by Richard Hartley and Andrew Zisserman (Second Edition). 9 | % Pag.225 and pag 226. 10 | % 11 | % Assumptions: 12 | % - vanishing points are orthogonal and finite representing x y and z 13 | % directions. No-noise is assumed. 14 | % - Constraints (needs 5 constraints to solve W): 15 | % - zero skew (first constraint) (w21 = w12 = 0) 16 | % - square pixels (second constraint) (w11 = w22) 17 | % - three other contraints come from combining pairs of orothogonal 18 | % vanishing points by u'wv = 0 19 | % - IAC form: 20 | % W = [w1 0 w2; 21 | % 0 w1 w3; 22 | % w2 w3 w4] 23 | % - K form: 24 | % K = [f 0 u0; 25 | % 0 f v0; 26 | % 0 0 1] 27 | % f = focal lenght in pixels 28 | % u0,v0 = image center 29 | % 30 | % Inputs: 31 | % The three vanishing points in homogeneous coordinates 32 | % Outputs: 33 | % W Image of the Absolute Conic 34 | % K intrinsics matrix 35 | % 36 | % $Revision: v1.0$ 37 | % $Author: Carlos Beltran-Gonzalez$ 38 | % $Email: carlos.beltran@iit.it$ 39 | % Istituto Italiano di Tecnologia 40 | % Pattern Analysis and Computer Vision 41 | % $Date: May 16, 2020$ 42 | % Copyright (C) 2020 Carlos Beltran-Gonzalez 43 | % CopyPolicy: GNU Lesser General Public License v3 44 | %--------------------------------------------------------- 45 | 46 | function [K,W] = computeKviaIAC(vp1,vp2,vp3) 47 | 48 | % Get vectorization of pairs of vanishing points 49 | a1 = getvpVectorization(vp1,vp2); 50 | a2 = getvpVectorization(vp1,vp3); 51 | a3 = getvpVectorization(vp3,vp2); 52 | 53 | A = vertcat(a1',a2',a3'); % Build up matrix A 54 | w = null(A); % the w vector is the null space of A 55 | W = [ w(1) 0 w(2);0 w(1) w(3);w(2) w(3) w(4)]; % Build the IAC from w 56 | K = inv(chol(W)); % Get intrinsic matrix (K) by Cholesky factorization of W 57 | K = K./K(9); % normalize K by forcing k33 to 1 58 | 59 | end 60 | 61 | function a = getvpVectorization(u,v) 62 | % input two vanishing points u and v 63 | % u'wv = 64 | % [u1 u2 u3] [w1 0 w2;0 w1 w3;w2 w3 w4] [v1 v2 v3]' = 65 | % [u1 u2 u3] [w1v1+w2v3; w1v2+w3v3;w2v1+w3v2+w4v3] = 66 | % u1(w1v1+w2v3) + u2(w1v2+w3v3) + u3(w2v1+w3v2+w4v3)= 67 | % w1(v1u1+v2u2) + w2(v3u1+v1u3) + w3(v3u2+v2u3) + w4v3u3 68 | % then considering a vectorization a'w = 0 we have: 69 | % a = (v1u1+v2u2,v3u1+v1u3,v3u2+v2u3,v3u3) 70 | % w = (w1,w2,w3,w4) 71 | 72 | a = [v(1)*u(1)+v(2)*u(2), 73 | v(3)*u(1)+v(1)*u(3), 74 | v(3)*u(2)+v(2)*u(3), 75 | v(3)*u(3)]; 76 | end 77 | 78 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # The EYELib 2 | 3 | A library for unifying a clean and straight forward implementation -in matlab 4 | and python- of the most common (and not so common) algorithms and techniques 5 | used for Single View Metrology and one image self/auto calibration. 6 | 7 | These algorithms are broadly explained in advanced courses of computer 8 | vision, books and papers, however, open source implementations of such 9 | algorithms are very scarce or simply don't exist. 10 | 11 | The library depends heavily on vanishing points. The stress is on what can 12 | be done with the vanishing points not on obtaining them. The EYELib assumes 13 | that vanishing points are provided by a third party library. 14 | 15 | Follows a list of stuff implemented so far (trying to keep this updated): 16 | - Implementation of the first part of Abbas and Zisserman ICCVW19 paper 17 | "A Geometric Approach to Obtain a Bird’s Eye View from an Image" 18 | - Attempt to implement 4-D SCENE ALIGNMENT IN SURVEILLANCE VIDEO by 19 | RobertWagner, Daniel Crispell, Patrick Feeney, Joe Mundy 20 | - Implementation of pitch angle angle calculation using the Image of the 21 | Absolute Conic. See "Multiple view Geometry in Computer Vision" by Harley 22 | and Zissermanequation 8.9 pag 210 Second Edition. 23 | - Implementation of algorithm "Calibration from three orthogonal vanishing 24 | points". See "Multiple view Geometry in Computer Vision" by Harley and 25 | Zisserman Example 8.27 pag 225-226 Second Edition. 26 | - Implementation of camera height calculation. SeeCriminisi, A., Reid, I. & 27 | Zisserman, A. Single View Metrology. International Journal of Computer 28 | Vision 40, 123–148 (2000). https://doi.org/10.1023/A:1026598000963 29 | - Implementation of rotation matrix from vanishing points and viceversa. 30 | 31 | ## Dependencies 32 | 33 | - The library depends frequently on Peter Corke toolboxes 34 | P.I. Corke, “Robotics, Vision & Control”, Springer 2011, ISBN 978-3-642-20143-1. **Important: notice that the paths to these toolboxes must be inserted on top of your matlab "setpath". Otherwise, there are some conflics with preessinting matlab rutines. Tested with Machine Vision toolbox on top followed by Robotics Toolbox** 35 | - The library also depends on the David Legland geom3d library: 36 | David Legland (2020). geom3d (https://www.mathworks.com/matlabcentral/fileexchange/24484-geom3d), MATLAB Central File Exchange. Retrieved May 26, 2020. 37 | 38 | ## Disclaimer 39 | 40 | Notice that, in principle, the library is not dealing with distortion or image noise issues. For testing, the library will be using synthetic images or ideal (very low distortion-noisy) images. 41 | Actually, the final purpose of the library is to provide a semi-automatic or eventually automatic way of extracting the geometry from a single image. Notice that this will highly depend on the quality of vanishing points or distortion estimation based on third party libraries. Ideally, in some near future the library may converge into deep learning approaches. 42 | -------------------------------------------------------------------------------- /abbas_hom.m: -------------------------------------------------------------------------------- 1 | clear all; close all; 2 | addpath('./matlab'); 3 | 4 | % Dependency on Peter Corke Toolboxes (Machine Vision and Robotics) 5 | 6 | %% Groundtruth 7 | vps_gt = [[ 1195.99492678 192.67503264]; ... 8 | [ 104.98138221 80.65389521]; ... 9 | [ 144.38825531 1614.68566071]]; 10 | 11 | K_gt = [[403.76110881 0. 288.]; ... 12 | [ 0. 403.76110881 216.]; ... 13 | [ 0. 0. 1. ]]; 14 | 15 | fov_gt = 71.0; 16 | camera_height = 7.692022323608398; 17 | pitch_gt = -16.022058486938477; 18 | yaw_gt = 24.999956130981445; 19 | roll_gt = -5.862370014190674; 20 | image_width = 576; 21 | image_height = 432; 22 | 23 | Rot_gt = rotx(-90,'deg')... 24 | *roty(yaw_gt,'deg')... 25 | *rotx(pitch_gt,'deg')... 26 | *rotz(roll_gt,'deg'); 27 | Rot_gt_inv = Rot_gt'; 28 | 29 | Tcam = transl(0,0,camera_height)... 30 | *trotx(-90,'deg')... 31 | *troty(yaw_gt,'deg')... 32 | *trotx(pitch_gt,'deg')... 33 | *trotz(roll_gt,'deg'); 34 | 35 | cam = CentralCamera('focal', 0.015,'pixel',10e-6, ... 36 | 'resolution',[1280 1024],'centre',[640 512],'pose',Tcam); 37 | world = SE3(); 38 | trplot(world,'frame','0','color','b'); 39 | cam.plot_camera(); 40 | 41 | xlim([-1 5]); 42 | ylim([-1 5]); 43 | zlim([-1 10]); 44 | 45 | % Get intrinsics matrix K from vanishing points 46 | vpx = vps_gt(1,:); 47 | vpy = vps_gt(2,:); 48 | vpz = vps_gt(3,:); 49 | %K = getKfromVanishingPoints(vpx,vpy,vpz); 50 | K = computeKviaIAC([vpx,1],[vpy,1],[vpz,1]); 51 | 52 | %check vanishing points from rotation matrix 53 | vvv1 = K_gt*Rot_gt_inv(:,1); 54 | vvv1 = h2e(vvv1)' 55 | gt_vp1 = vps_gt(1,:) 56 | 57 | vvv2 = K_gt*Rot_gt_inv(:,2); 58 | vvv2 = h2e(vvv2)' 59 | gt_vp2 = vps_gt(2,:) 60 | 61 | vvv3 = K_gt*Rot_gt_inv(:,3); 62 | vvv3 = h2e(vvv3) 63 | gt_vp3 = vps_gt(3,:) 64 | 65 | % open image 66 | I = imread('./syntheticdata/town_1_frame_1268.png'); 67 | I = insertShape(I,'Line',[vps_gt(1,1) vps_gt(1,2) vps_gt(2,1) vps_gt(2,2)],... 68 | 'LineWidth',2,'Color','blue'); 69 | imshow(I); 70 | % hold on 71 | 72 | % Plot vanishing points 73 | % plot(vps_gt(1,1),vps_gt(1,2),'*r'); 74 | % plot(vps_gt(2,1),vps_gt(2,2),'*g'); 75 | % plot(vps_gt(3,1),vps_gt(3,2),'*b'); 76 | 77 | % compute horizon line 78 | horizon = cross([vps_gt(1,:),1],[vps_gt(2,:),1]); 79 | 80 | %computer roll around z axis 81 | theta_z = atan(-horizon(1)/horizon(2)); 82 | roll_deg = rad2deg(theta_z); 83 | roll_gt; 84 | 85 | % %Rotation matrix on z axis 86 | % Rz = [ cos(theta_z) -sin(theta_z) 0; 87 | % sin(theta_z) cos(theta_z) 0; 88 | % 0 0 1]; 89 | 90 | % % compute x rotation (tilt) [ this doesn't seem to work, error in stimated 91 | % % angle ] 92 | % theta_x = (pi/2.0) - atan(norm([vps_gt(3,:)])/K_gt(1)); 93 | % tild_deg = rad2deg(theta_x); 94 | % pitch_gt; 95 | 96 | % Get Pitch using the IAC by means of the intrinsics matrix K and the 97 | % vertical vanishg point 98 | pitch_iac = getPitchByIAC(K,vps_gt(3,:)); 99 | 100 | % Check the rotation matrix using the vanishing points 101 | r1 = (inv(K_gt)*[vps_gt(1,:),1]')/norm(inv(K_gt)*[vps_gt(1,:),1]'); 102 | r2 = (inv(K_gt)*[vps_gt(2,:),1]')/norm(inv(K_gt)*[vps_gt(2,:),1]'); 103 | r3 = cross(r1,r2); 104 | RotM_2 = [r1,r2,r3]' 105 | 106 | % Rot_gt 107 | % cam.T 108 | 109 | 110 | %% compute homograpy 111 | Hrot_gt = K_gt ... 112 | *rotx(90+pitch_gt,'deg')... 113 | *inv(K_gt)... 114 | *rotz(roll_gt,'deg'); 115 | 116 | Hrot = K ... 117 | *rotx(90-pitch_iac,'deg')... %% Pitch 118 | *inv(K)... 119 | *rotz(-roll_deg,'deg'); 120 | 121 | % compute traslation 122 | frame_width = image_width; 123 | frame_height = image_height; 124 | 125 | tlc = [0, 0]; 126 | trc = [frame_width, 0]; 127 | blc = [0, frame_height]; 128 | brc = [frame_width, frame_height]; 129 | 130 | P1 = horzcat(tlc',trc',blc',brc'); 131 | P2 = homtrans(inv(Hrot), P1); 132 | 133 | xmin = min(P2(1,:)); 134 | xmax = max(P2(1,:)); 135 | ymin = min(P2(2,:)); 136 | ymax = max(P2(2,:)); 137 | 138 | Tscene = [[1, 0, -xmin]; 139 | [0, 1, -ymin]; 140 | [0, 0, 1]]; 141 | 142 | Ralign = rotz(20,'deg'); 143 | Hrot_final = Ralign * Tscene * Hrot; 144 | 145 | cb_ref = imref2d(size(I)) 146 | cb_translated_ref = cb_ref; 147 | cb_translated_ref.XWorldLimits(1) = cb_translated_ref.XWorldLimits(1)-500; 148 | cb_translated_ref.YWorldLimits(1) = cb_translated_ref.YWorldLimits(1)-2000; 149 | cb_translated_ref.XWorldLimits(2) = cb_translated_ref.XWorldLimits(2)+1500; 150 | cb_translated_ref.YWorldLimits(2) = cb_translated_ref.YWorldLimits(2)+1500; 151 | % % % % apply rototranslation to the image plane 152 | figure; 153 | tform = projective2d(Hrot_final'); 154 | [imout,trans_ref] = imwarp(I,tform,'OutputView',cb_translated_ref); 155 | % % figure; 156 | imshow(imout,trans_ref); 157 | hold on; 158 | % plot(P2(1,:),P2(2,:),'*r'); 159 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | GNU LESSER GENERAL PUBLIC LICENSE 2 | Version 3, 29 June 2007 3 | 4 | Copyright (C) 2007 Free Software Foundation, Inc. 5 | Everyone is permitted to copy and distribute verbatim copies 6 | of this license document, but changing it is not allowed. 7 | 8 | 9 | This version of the GNU Lesser General Public License incorporates 10 | the terms and conditions of version 3 of the GNU General Public 11 | License, supplemented by the additional permissions listed below. 12 | 13 | 0. Additional Definitions. 14 | 15 | As used herein, "this License" refers to version 3 of the GNU Lesser 16 | General Public License, and the "GNU GPL" refers to version 3 of the GNU 17 | General Public License. 18 | 19 | "The Library" refers to a covered work governed by this License, 20 | other than an Application or a Combined Work as defined below. 21 | 22 | An "Application" is any work that makes use of an interface provided 23 | by the Library, but which is not otherwise based on the Library. 24 | Defining a subclass of a class defined by the Library is deemed a mode 25 | of using an interface provided by the Library. 26 | 27 | A "Combined Work" is a work produced by combining or linking an 28 | Application with the Library. The particular version of the Library 29 | with which the Combined Work was made is also called the "Linked 30 | Version". 31 | 32 | The "Minimal Corresponding Source" for a Combined Work means the 33 | Corresponding Source for the Combined Work, excluding any source code 34 | for portions of the Combined Work that, considered in isolation, are 35 | based on the Application, and not on the Linked Version. 36 | 37 | The "Corresponding Application Code" for a Combined Work means the 38 | object code and/or source code for the Application, including any data 39 | and utility programs needed for reproducing the Combined Work from the 40 | Application, but excluding the System Libraries of the Combined Work. 41 | 42 | 1. Exception to Section 3 of the GNU GPL. 43 | 44 | You may convey a covered work under sections 3 and 4 of this License 45 | without being bound by section 3 of the GNU GPL. 46 | 47 | 2. Conveying Modified Versions. 48 | 49 | If you modify a copy of the Library, and, in your modifications, a 50 | facility refers to a function or data to be supplied by an Application 51 | that uses the facility (other than as an argument passed when the 52 | facility is invoked), then you may convey a copy of the modified 53 | version: 54 | 55 | a) under this License, provided that you make a good faith effort to 56 | ensure that, in the event an Application does not supply the 57 | function or data, the facility still operates, and performs 58 | whatever part of its purpose remains meaningful, or 59 | 60 | b) under the GNU GPL, with none of the additional permissions of 61 | this License applicable to that copy. 62 | 63 | 3. Object Code Incorporating Material from Library Header Files. 64 | 65 | The object code form of an Application may incorporate material from 66 | a header file that is part of the Library. You may convey such object 67 | code under terms of your choice, provided that, if the incorporated 68 | material is not limited to numerical parameters, data structure 69 | layouts and accessors, or small macros, inline functions and templates 70 | (ten or fewer lines in length), you do both of the following: 71 | 72 | a) Give prominent notice with each copy of the object code that the 73 | Library is used in it and that the Library and its use are 74 | covered by this License. 75 | 76 | b) Accompany the object code with a copy of the GNU GPL and this license 77 | document. 78 | 79 | 4. Combined Works. 80 | 81 | You may convey a Combined Work under terms of your choice that, 82 | taken together, effectively do not restrict modification of the 83 | portions of the Library contained in the Combined Work and reverse 84 | engineering for debugging such modifications, if you also do each of 85 | the following: 86 | 87 | a) Give prominent notice with each copy of the Combined Work that 88 | the Library is used in it and that the Library and its use are 89 | covered by this License. 90 | 91 | b) Accompany the Combined Work with a copy of the GNU GPL and this license 92 | document. 93 | 94 | c) For a Combined Work that displays copyright notices during 95 | execution, include the copyright notice for the Library among 96 | these notices, as well as a reference directing the user to the 97 | copies of the GNU GPL and this license document. 98 | 99 | d) Do one of the following: 100 | 101 | 0) Convey the Minimal Corresponding Source under the terms of this 102 | License, and the Corresponding Application Code in a form 103 | suitable for, and under terms that permit, the user to 104 | recombine or relink the Application with a modified version of 105 | the Linked Version to produce a modified Combined Work, in the 106 | manner specified by section 6 of the GNU GPL for conveying 107 | Corresponding Source. 108 | 109 | 1) Use a suitable shared library mechanism for linking with the 110 | Library. A suitable mechanism is one that (a) uses at run time 111 | a copy of the Library already present on the user's computer 112 | system, and (b) will operate properly with a modified version 113 | of the Library that is interface-compatible with the Linked 114 | Version. 115 | 116 | e) Provide Installation Information, but only if you would otherwise 117 | be required to provide such information under section 6 of the 118 | GNU GPL, and only to the extent that such information is 119 | necessary to install and execute a modified version of the 120 | Combined Work produced by recombining or relinking the 121 | Application with a modified version of the Linked Version. (If 122 | you use option 4d0, the Installation Information must accompany 123 | the Minimal Corresponding Source and Corresponding Application 124 | Code. If you use option 4d1, you must provide the Installation 125 | Information in the manner specified by section 6 of the GNU GPL 126 | for conveying Corresponding Source.) 127 | 128 | 5. Combined Libraries. 129 | 130 | You may place library facilities that are a work based on the 131 | Library side by side in a single library together with other library 132 | facilities that are not Applications and are not covered by this 133 | License, and convey such a combined library under terms of your 134 | choice, if you do both of the following: 135 | 136 | a) Accompany the combined library with a copy of the same work based 137 | on the Library, uncombined with any other library facilities, 138 | conveyed under the terms of this License. 139 | 140 | b) Give prominent notice with the combined library that part of it 141 | is a work based on the Library, and explaining where to find the 142 | accompanying uncombined form of the same work. 143 | 144 | 6. Revised Versions of the GNU Lesser General Public License. 145 | 146 | The Free Software Foundation may publish revised and/or new versions 147 | of the GNU Lesser General Public License from time to time. Such new 148 | versions will be similar in spirit to the present version, but may 149 | differ in detail to address new problems or concerns. 150 | 151 | Each version is given a distinguishing version number. If the 152 | Library as you received it specifies that a certain numbered version 153 | of the GNU Lesser General Public License "or any later version" 154 | applies to it, you have the option of following the terms and 155 | conditions either of that published version or of any later version 156 | published by the Free Software Foundation. If the Library as you 157 | received it does not specify a version number of the GNU Lesser 158 | General Public License, you may choose any version of the GNU Lesser 159 | General Public License ever published by the Free Software Foundation. 160 | 161 | If the Library as you received it specifies that a proxy can decide 162 | whether future versions of the GNU Lesser General Public License shall 163 | apply, that proxy's public statement of acceptance of any version is 164 | permanent authorization for you to choose that version for the 165 | Library. 166 | --------------------------------------------------------------------------------