SOREPP

 

Yehonatan Goldman, Ehud Rivlin, and Ilan Shimshoni

 

Abstract

Epipolar geometry estimation is fundamental to many computer vision algorithms. It has therefore attracted a lot of interest in recent years, yielding high quality estimation algorithms for wide baseline image pairs. Currently many types of cameras such as smartphones produce geo-tagged images containing pose and internal calibration data.

These include a GPS receiver, which estimates the position, a compass, accelerometers, and gyros, which estimate the orientation, and the focal length. Exploiting this information as part of an epipolar geometry estimation algorithm may be useful but not trivial, since the pose measurement may be quite noisy. We introduce SOREPP (Soft Optimization method for Robust Estimation based on Pose Priors), a novel estimation algorithm designed to exploit pose priors naturally. It sparsely samples the pose space around the measured pose and for a few promising candidates applies a robust optimization procedure. It uses all the putative correspondences simultaneously, even though many of them are outliers, yielding a very efficient algorithm whose runtime is independent of the inlier fraction. SOREPP was extensively tested on synthetic data and on hundreds of real image pairs taken by smartphones. Its ability to handle challenging scenarios with extremely low inlier fractions of less than 10% was demonstrated. It outperforms current state-of-the-art algorithms that do not use pose priors as well as others that do.

 

The paper

Y. Goldman, E. Rivlin, and I. Shimshoni, “Robust Epipolar Geometry Estimation Using Noisy Pose Priors”. In Image and vision computing, Vol 67, November 2017, pages 16-28.

Implementation

A demo, including Matlab mex and the SOREPP's dll are available here. The code is compiled for Windows 64 bit. The demo includes also a description of the interface, a dll which reads the pose data from jpeg files of images taken by SitisMobile's GeoCam application, and instruction how to apply the demo on new images.

The code of SOREPP and a readme file describing how to compile it are available here.

The full datasets used for the paper and some Matlab functions which runs SOREPP on it are available here.

If you have any questions please refer to Yehonatan Goldman yehonatan.goldman@gmail.com or Ilan Shimshoni ishimshoni@is.haifa.ac.il .

 

Examples

Some results of SOREPP on challenging image pairs are given in the following table. The left images are the reference images with some control points marked, and the right images are the target images with control points marked, as well as the corresponding epipolar lines based on the estimation result of SOREPP.

example_7_9_target.jpg

example_13_11_ref.jpg

example_13_11_target.jpg

example_16_22_ref.jpg

example_16_22_target.jpg

example_18_23_ref.jpg

example_18_23_target.jpg

example_60_76_ref.jpg

example_60_76_target.jpg

example_66_59_ref.jpg

example_66_59_target.jpg

example_72_62_ref.jpg

example_72_62_target.jpg

example_58_69_ref.jpg

example_58_69_target.jpg

example_82_88_ref.jpg

example_82_88_target.jpg

example_80_91_ref.jpg

example_80_91_target.jpg

example_88_83_ref.jpg

example_88_83_target.jpg

example_82_84_ref.jpg

example_82_84_target.jpg