# Camera Model and Image formation Computer graphics: Forward problem Computer vision: Inverse problem ## Designing a camera Bare Sensor Camera - puts the sensor or a piece of a film in front of an object. - Every point in 3D object provides light to every point in the sensor. So we don't get a reasonable image. Camera Obscura - Apperture reduces the blurring, but it won't still be super sharp. - Virtual image concept is created to not deal with negative signes etc. Add a Lens - Lens focuses light onto the film - Rays passing through the center are not deviated Depth of field ![[Screenshot 2020-09-04 at 1.44.15 PM.jpg]] Is the apperture getting bigger or smaller? - Smaller apperture means larger depth of field. f number and apperture is inversely proportional. Apperture - Why not make the apperture as small as possible? - Less light gets through - Diffraction, so blur ## Modeling projection Coordinate system - Pinhole camera (as an approximation) is the imager. - The optical center (center of projection) is at the origin. - Image plane at one focal length away form the center. Projection operator: $P: \mathbf{R}^3 \rightarrow \mathbf{R}^2$ $P$ takes a point (x,y,z) to the point $(x\frac{f}{z}, y\frac{f}{z})$ This mapping is not linear since division by z is non linear. Solution is to add one more coordinate, called homogenous coordinates. ![[Screenshot 2020-09-04 at 2.07.20 PM.jpg]] Homogenous coordinates are invariant to scaling. A point in cartesian is a ray in homogenous coordinates. $ \left[\begin{array}{llll} 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 / f & 0 \end{array}\right]\left[\begin{array}{l} x \\ y \\ z \\ 1 \end{array}\right]=\left[\begin{array}{l} x \\ y \\ z / f \end{array}\right] $ Thanks to homogenous coordinates, projection is now written as a vector matrix multiplication. Projection does not preserve length and area. ![[Screenshot 2020-09-04 at 2.16.38 PM.jpg]] Intrinsic assumptions - Unit aspect ratio - Optical center is at (0,0) - No skew Extrinsic assumptions - No rotation - Camera at (0,0,0) In this case, ![[Screenshot 2020-09-04 at 2.32.03 PM.jpg]] This comes from similar triangles: ![[Screenshot 2020-09-04 at 2.32.58 PM.jpg]] Removing the assumptions: ![[Screenshot 2020-09-04 at 2.34.02 PM.jpg]] ![[Screenshot 2020-09-04 at 2.34.08 PM.jpg]] ![[Screenshot 2020-09-04 at 2.34.22 PM.jpg]] Allowing camera rotation and translation: $ w\left[\begin{array}{l} u \\ v \\ 1 \end{array}\right]=\left[\begin{array}{lll} \alpha & s & u_{0} \\ 0 & \beta & v_{0} \\ 0 & 0 & 1 \end{array}\right]\left[\begin{array}{llll} r_{11} & r_{12} & r_{13} & t_{x} \\ r_{21} & r_{22} & r_{23} & t_{y} \\ r_{31} & r_{32} & r_{33} & t_{z} \end{array}\right]\left[\begin{array}{l} x \\ y \\ z \\ 1 \end{array}\right] $ For intrinsics, we have 5 degrees of freedom and for extrinsics we have 6 degrees of freedom. ## Digital image representation An image is a function of spatial coordinates.$f$ may represent intensity or color. Spatial resolution - number of pixels Quantization level - Number of level of intensity/color ex 8 bit grayscale image is 256 levels of gray ## Fundamentals of inverse problems for 3D cameras Inverse problems - start with the observations, and go back to unobserved set of modal parameters. It provides information that we are unable to observe directly. Inverse problem is mathematically ill-posed. A solution might not exist or there may be infinately many solutions. Solutions may not be stable. ![[Screenshot 2020-09-04 at 3.47.12 PM.jpg]] Properties: Regularized using deterministic or stoachasic methods - regularization or inference with gaussian priors Low dimensional data representation is critical in inverse problems - sparse representations - better reconstruction quality - optimization problem is nonlinear --- ## References