Abstract

We present a new approach to image indexing and retrieval, which integrates appearance with global image geometry in the indexing
process, while enjoying robustness against viewpoint change, photometric variations, occlusion, and background clutter. We exploit
shape parameters of local features to estimate image alignment via a single correspondence. Then, for each feature, we construct
a sparse spatial map of all remaining features, encoding their normalized position and appearance, typically vector quantized to
visual word. An image is represented by a collection of such feature maps and RANSAC-like matching is reduced to a number of
set intersections. The required index space is still quadratic in the number of features. To make it linear, we propose a novel feature
selection model tailored to our feature map representation, replacing our earlier hashing approach. The resulting index space is
comparable to baseline bag-of-words, scaling up to one million images while outperforming the state of the art on three publicly
available datasets. To our knowledge, this is the first geometry indexing method to dispense with spatial verification at this scale,
bringing query times down to milliseconds.