Columbia University

Technology Ventures

Accelerated high-performance visual search for mobile devices

Technology #cu12200

Questions about this technology? Ask a Technology Manager

Download Printable PDF

Collections
Image Gallery
Researchers
Shih-fu Chang
Managed By
Satish Rao
Patent Protection
US Patent 9,009,149

This technology is a system for performing visual searches from a mobile device in wireless communication with a server. Local visual features are extracted from a query image acquired by the mobile device and encoded using a compact hash method. The encoded features and image boundary information are transmitted to a server that matches the features against a database. Geometry verification is performed on the match results to determine whether the query image may be a geometric transformation of an image in the database. The query image’s boundary data is used to rank the search results before they are transmitted back to the mobile device for viewing by the device user.

Reranking of search results reduces mobile visual search bandwidth requirements and increases search performance.

Mobile visual search systems typically must extract relevant features of a captured image and transmit them to a server that attempts to match them over a database. Since transmission of visual features in graphical format can consume considerable bandwidth, there is a need to compress extracted image features. As the use of compressed graphics formats for this purpose does not afford a sufficient reduction in bandwidth usage, features must be encoding using some other method. By employing a bag of hash bits method to encode local images features, the technology can represent features using only a small number of bits. The hash bit representation also accelerates search speeds on the server by enabling flexible creation of the data indexing structure used to match encoded features. Search accuracy is boosted by using image boundaries to rerank the returned search results.

The technology’s ability to outperform existing state-of-the-art mobile search methods was demonstrated over multiple sets of product images each containing between 300,000 and 400,000 images.

Lead Inventor:

Shih-Fu Chang, Ph.D.

Applications:

  • Real-time security screening (e.g., at airports, restricted areas) against a database of person images.
  • Search-assisted shopping.
  • Digital tourist assistance, e.g., obtain useful information about a venue and its surroundings using a photo.

Advantages:

  • Rapid visual searches over large databases of images.
  • Increased search accuracy.
  • Reduced wireless bandwidth requirements.
  • Can be implemented to run efficiently on mobile devices.

Patent information:

Patent Pending (US 20130254191)

Tech Ventures Reference: IR CU12200

Related Publications: