# Image Stitching Image stitching combines two or more overlapping images to make one large image. ![[stitching.jpg]] Basic Procedure 1. Take a sequence of images from the same position - Rotate the camera about its optical center 2. Compute transformation between second image and first - Extract ineterest points using [[Harris Corner Detection]] or [[Scale-Invariant Feature Transform (SIFT)]] - Find Matches using euclidean distance or maximum correlation. Then use [[RANSAC]] to remove incorrect matches. - Compute transformation (using [[RANSAC]] again) ![[keypoints.jpg]] 3. Shift the second image to overlap with the first ![[overlapping.jpg]] 4. Blend the two together to create a mosaic ![[blendingmosiac.jpg]] 5. If there are more images, repeat ## Finding the transformation How many corresponding points do we need to solve? ![[transformations-sticting.jpg]] ### Translation - 2 degrees of freedom - For each point $\left(\mathbf{x}_{i}, \mathbf{y}_{i}\right)$ $ \begin{aligned} \mathbf{x}_{i}+\mathbf{x}_{\mathbf{t}} &=\mathbf{x}_{i}^{\prime} \\ \mathbf{y}_{i}+\mathbf{y}_{\mathbf{t}} &=\mathbf{y}_{i}^{\prime} \end{aligned} $ - we define the residuals as $ \begin{aligned} r_{\mathbf{x}_{i}}\left(\mathbf{x}_{t}\right) &=\left(\mathbf{x}_{i}+\mathbf{x}_{t}\right)-\mathbf{x}_{i}^{\prime} \\ r_{\mathbf{y}_{i}}\left(\mathbf{y}_{t}\right) &=\left(\mathbf{y}_{i}+\mathbf{y}_{t}\right)-\mathbf{y}_{i}^{\prime} \end{aligned} $ - Solve using least squares formulation: $ C\left(\mathbf{x}_{t}, \mathbf{y}_{t}\right)=\sum_{i=1}^{n}\left(r_{\mathbf{x}_{i}}\left(\mathbf{x}_{t}\right)^{2}+r_{\mathbf{y}_{i}}\left(\mathbf{y}_{t}\right)^{2}\right) $ ### Similarity - 4 degrees of freedom Affine - 6 degrees of freedom - Residuals $ \begin{aligned} r_{x_{i}}(a, b, c, d, e, f) &=\left(a x_{i}+b y_{i}+c\right)-x_{i}^{\prime} \\ r_{y_{i}}(a, b, c, d, e, f) &=\left(d x_{i}+e y_{i}+f\right)-y_{i}^{\prime} \end{aligned} $ - Cost function $ \begin{array}{l} C(a, b, c, d, e, f)= \quad \sum_{i=1}^{n}\left(r_{x_{i}}(a, b, c, d, e, f)^{2}+r_{y_{i}}(a, b, c, d, e, f)^{2}\right) \end{array} $ ### Homography - 8 degrees of freedom - No longer linear $ \begin{array}{l} {\left[\begin{array}{l} x_{i}^{\prime} \\ y_{i}^{\prime} \\ 1 \end{array}\right] \cong\left[\begin{array}{lll} h_{00} & h_{01} & h_{02} \\ h_{10} & h_{11} & h_{12} \\ h_{20} & h_{21} & h_{22} \end{array}\right]\left[\begin{array}{l} x_{i} \\ y_{i} \\ 1 \end{array}\right]} \\ x_{i}^{\prime}=\frac{h_{00} x_{i}+h_{01} y_{i}+h_{02}}{h_{20} x_{i}+h_{21} y_{i}+h_{22}} \\ y_{i}^{\prime}=\frac{h_{10} x_{i}+h_{11} y_{i}+h_{12}}{h_{20} x_{i}+h_{21} y_{i}+h_{22}} \end{array} $ Linearize it and solve using least squares as usual $ \begin{array}{l} x_{i}^{\prime}\left(h_{20} x_{i}+h_{21} y_{i}+h_{22}\right)=h_{00} x_{i}+h_{01} y_{i}+h_{02} \\ y_{i}^{\prime}\left(h_{20} x_{i}+h_{21} y_{i}+h_{22}\right)=h_{10} x_{i}+h_{11} y_{i}+h_{12} \\ {\left[\begin{array}{ccccccccc} x_{i} & y_{i} & 1 & 0 & 0 & 0 & -x_{i}^{\prime} x_{i} & -x_{i}^{\prime} y_{i} & -x_{i}^{\prime} \\ 0 & 0 & 0 & x_{i} & y_{i} & 1 & -y_{i}^{\prime} x_{i} & -y_{i}^{\prime} y_{i} & -y_{i}^{\prime} \end{array}\right]\left[\begin{array}{c} h_{00} \\ h_{01} \\ h_{02} \\ h_{10} \\ h_{11} \\ h_{12} \\ h_{20} \\ h_{21} \\ h_{22} \end{array}\right]=\left[\begin{array}{l} 0 \\ 0 \end{array}\right]} \end{array} $ ## Choosing a projection surface ![[matching-planes.jpg]] --- ## References