Lecture 6: Camera Computation and the Essential Matrix

Pages 5
Views 12

Please download to get full document.

View again

of 5
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Lecture 6: Camera Computation and the Essential Matrix 1 Computing Cameras From the Fundamental Matrix In Lecture 5 we considered the two view structure from motion problem, that is, given a number of
Lecture 6: Camera Computation and the Essential Matrix 1 Computing Cameras From the Fundamental Matrix In Lecture 5 we considered the two view structure from motion problem, that is, given a number of measured points in two images we want to compute both camera matrices and 3D points such that they project to the measurements. We showed that the 3D points can be eliminated from the problem by considering the fundamental matrix F. If x is an image point belonging to the fist image and x belongs to the second then there is a 3D point that projects to to these if and only if the epipolar constraint x T F x = 0 (1) is fulfilled. Using the projections of 8-scene points we can compute the fundamental matrix by solving a homogeneous least squares problem (the 8-point algorithm). What remains in order to find a solution to the two view structure from motion camera is to compute cameras from F and finally compute the 3D-points. In general we may assume (see Lecture 5) that the cameras are of the form P 1 = [I 0] and P 2 = [A e 2 ] where e 2 is the epipole in the second image. Since we know that F T e 2 = 0 we can find e 2 by computing the null space of F T. A solution for the second camera is then given by P 2 = [[e 2 ] F e 2 ]. (2) To see that these [ cameras ] have the fundamental matrix F we verify that their projections fulfill the epipolar constraint. If X =, where X R X ρ 3 and ρ R then the projections in the two cameras are given by x [ I 0 ] [ ] X ρ x [ [e 2 ] F e 2 ] [ X ρ = X, (3) ] = [e 2 ] F X + e 2 ρ = e 2 (F X) + e 2 ρ. (4) Therefore x T F x = (e 2 (F X) + e 2 ρ) T F X = (e 2 (F X)) T F X + ρe T 2 F X. (5) Since e 2 (F X) is a vector that is perpendicular to F X the term (e 2 (F X)) T F X = 0. Furthermore, since e 2 is in the null space of F T the term ρe T 2 F X = ρ(f T e 2 ) T X = 0. Therefore the projections of these camera matrices will fulfill the epipolar constraints. Exercise 1. What is the camera center of P 2 = [[e 2 ] F e 2 ]? (Hint: Recall that F e 1 = 0.) Since there is a projective ambiguity there are many choices for P 2. Given that P 1 = [I 0] the general formula for P 2 is P 2 = [ [e 2 ] F + e 2 v T ] λe 2, (6) where v is some vector in R 3 and λ is a non-zero scalar. Exercise 2. Verify that the projections in P 1 = [I v R 3 and λ 0. 0] and P 2 given by (6) fulfill the epipolar constraints for any 1 Figure 1: Two images of a chair with 14 known point correspondences. Blue are the image measurements and red o are the reprojections. The 3D points (to the right) look strange because of the projective ambiguity (note the difference in scale on the axes). 2 Relative Orientation: The Calibrated Case When solving the relative orientation problem without camera calibration there is, as we saw in Lecture 3, an ambiguity. Basically any projective transformation can be applied to the 3D-points to give a new solution. Therefore the resulting constructions can often look strange even though the reprojections are correct (see Figure 1). To remove this ambiguity one has to add additional knowledge about the solution to the problem. For example, if we have some knowledge about the 3D scene, such as the distance between a few of the points, then we can apply a transform to the solution that make these distances correct. Alternatively we can add knowledge about the cameras. If the inner parameters K 1 and K 2 are known we consider the calibrated two-view structure from motion problem. Given two sets of corresponding points x i and x i, i = 1,...n and inner parameters K 1 and K 2 our goal is to find [R 1 t 1 ], [R 2 t 2 ] and X i such that and R 1,R 2 are rotation matrices. x i K 1 [R 1 t 1 ]X i (7) x i K 2 [R 2 t 2 ]X i, (8) We can make two simplifications to the problem. First we normalize the cameras by multiplying equations (7) and (8) with K1 1 and K2 1 respectively. Furthermore, we apply the euclidean transformation [ ] R T H = 1 R1 T t 1 (9) to the cameras (and H 1 to the 3D points). This gives us the new cameras P 1 H = [ ] [ ] R1 R 1 t T R1 T t 1 1 = [ I 0 ] (10) P 2 H = [ ] [ ] [ R1 R 2 t T R1 T t 1 R2 R1 T R 2 R T ] 1 t 1 t 2 2 = }{{}}{{}. (11) =R =t Therefore we search for a solution to the equations y i [I 0]X i (12) ȳ i [R t]x i, (13) 2 where y i = K 1 1 x i and ȳ i = K 1 1 x i are the normalized image coordinates. 2.1 The Essential Matrix The fundamental matrix for a pair of cameras of the form [I 0] and [R t] is given by E = [t] R, (14) and is called the Essential matrix. A rotation has 3 degrees of freedom and a translation 3. Since the scale of the essential matrix does not matter it therefore has 5 degrees of freedom. The reduction in freedom results in extra constraints on the singular values of E. In addition to having det(e) = 0 the two non-zero singular values have to be equal. Furthermore, since the scale is arbitrary we can assume that these singular values are both 1. Therefore E has the SVD E = Udiag([1 1 0])V T. (15) The decomposition is not unique. We will assume that we have a singular value decomposition where det(uv T ) = 1. It is easy to ensure this; If we have an SVD as in (15) with det(uv T ) = 1 then we can simply switch the sign of the last column of V. Alternatively we can switch to E which then has the SVD E = Udiag([1 1 0])( V ) T. (16) with det(u( V ) T ) = ( 1) 3 det(uv T ) = 1. Note however that if we recompute the SVD for E we might get another decomposition since it is not unique. To find the essential matrix we can use a slightly modified 8-point algorithm. From 8 points correspondences we form the M matrix (see Lecture 6) and solve the homogeneous least squares system min v 2 =1 Mv 2 (17) using SVD. The resulting vector v can be used to form a matrix Ẽ that does not necessarily have the right singular values 1, 1, 0. We therefore compute the svd decomposition Ẽ = USV T and construct an essential matrix using E = Udiag([1 1 0])V T. 1 Since the essential matrix has only 5 degrees of freedom it is possible to find it using only 5 correspondences. However as in the case of the fundamental matrix the extra constraints are non-linear which makes estimation more difficult. We summarize the steps of the modified 8-point algorithm here: Extract at least 8 point correspondences. Normalize the coordinates using K 1 1 and K 1 2 where K 1 and K 2 are the inner parameters of the cameras. Form M and solve using SVD. min v 2 =1 Mv 2, Form the matrix E (and ensure that E has the singular values 1, 1, 0). Compute P 2 from E (next section). Compute the scene points using triangulation (see Lecture 4). 1 Alternatively E = Udiag([1 1 0])( V ) T if det(uv T ) = 1. 3 3 Computing Cameras from E Once we have determined the essential matrix E we need extract cameras from it. Basically we want to decompose it into E = SR where S is a skew symmetric matrix and R is a rotation. We will use the two matrices W = and Z = (18) The matrix W is a rotation and Z is skew symmetric. Furthermore, for these matrices we have that ZW = diag([1 1 0]) (19) ZW T = diag([1 1 0]). (20) If E has the SVD from (15) we can now find two solutions; E = S 1 R 1, where and E = S 2 R 1, where S 1 = UZU T, R 1 = UW T V T (21) S 2 = UZU T, R 2 = UW V T. (22) To see that these are valid solutions we first verify that R 1 and R 2 are rotations. Since R 1 is orthogonal. Furthermore, R T 1 R 1 = (UW T V T ) T UW T V T = V W U T UW T V T = I (23) det(r 1 ) = det(uw T V T ) = det(u) det(w T ) det(v T ) = det(w ) det(uv T ) = 1, (24) and therefore R 1 is a rotation. (Note that if det(uv T ) = 1 then the R 1 that we obtain is not a rotation but a rotation composed with a reflexion and therefore not a valid solution.) That S 1 is skew symmetric is easy to see since S T 1 = (UZU T ) T = UZ T U T = UZU T = S 1. (25) Finally we see that S 1 R 1 = UZU T UW V T = UZW T V T = U( diag([1 1 0]))V T = E, (26) and therefore S 1 R 1 is a valid decomposition. E = S 2 R 2 can be verified similarly. Furthermore, it can be shown that these are the only two possibilities for each given E (see Hartley, Zisserman 2004). When we have determined a decomposition E = SR we need to compute a translation vector t from S such that [t] = S. For such a t we have St = [t] t = t t = 0. (27) Therefore the vector t is in the null space of S. The null space of the two matrices S 1 and S 2 are the same and we can find it by looking in the third column of U. Note that if t is in the null space of S then so is λt. In fact any non-zero λ gives a valid solution since [λt] R = λ[t] R = λe which is also a valid essential matrix for the problem. Different λ corresponds to rescaling the solution and since there is a scale ambiguity we cannot determine a true value of λ. However the sign of λ is important since it determines whether points are in front of the cameras or not in the final reconstruction. To make sure that we can find a solution where the points are in front of both the cameras we therefore test λ = ±1. If u 3 is the third column of U we get the four solutions P 2 = [UW V T u 3 ] or [UW T V T u 3 ] (from λ = 1) (28) or [UW V T u 3 ] or [UW T V T u 3 ] (from λ = 1) (29) When we have computed these four solutions we compute the 3D points using triangulation for all the choices of P 2 and select the one with where points are in front of both P 1 and P 2. Figure 2 shows the four calibrated reconstructions obtained using the images in Figure 1. Only one of them have all the points in front of both the cameras. 4 Figure 2: The 4 solutions when solving calibrated structure from motion for the chair image images in Figure 1. Only the second one have positive depths. 5
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!