Metric learning for image alignment

Image alignment has been a long standing problem in computer vision. Parameterized Appearance Models (PAMs) such as the Lucas-Kanade method, Eigentracking, and Active Appearance Models are commonly used to align images with respect to a template or to a previously learned model. While PAMs have numerous advantages relative to alternate approaches, they have at least two drawbacks. First, they are especially prone to local minima in the registration process. Second, often few, if any, of the local minima of the cost function correspond to acceptable solutions. To overcome these problems, this paper proposes a method to learn a metric for PAMs that explicitly optimizes that local minima occur at and only at the places corresponding to the correct fitting parameters. To the best of our knowledge, this is the first paper to address the problem of learning a metric to explicitly model local properties of the PAMs' error surface. Synthetic and real examples show improvement in alignment performance in comparison with traditional approaches. In addition, we show how the proposed criteria for a good metric can be used to select good features to track.