Columbia University

Technology Ventures

Focal Sweep for Space-time Depth Recovery and User Interactive Refocusing

Technology #cu14239

This technology uses a focal sweep method to capture space-time volume for depth recovery and interactive focusing. During the time that a sequence of images is being captured, the camera focus is swept over a range of depths in the scene. Since objects in the captured range can be moving, the image sequence captures object motion as well. This allows each image sequence to capture 3D space-time volume. This technology also contains an optimal strategy for camera sensor movement that allows a camera to capture the entire 3D space-time volume in the most efficient way. Furthermore, the technology contains algorithms to recover all-in-focus images and in-focus index maps, which can form the basis for interactive refocusing. This technology thus helps solve the problem of blurred images and may allow users several layers of focus (or lack thereof) in their photographs.

Focal sweep technology allows high fidelity image capturing at several different layers of depth

Traditionally when users took pictures with cameras, they had to adjust the lens focus prior to taking the photograph. If the focus proved to be incorrect, they would adjust the focus and then take another picture, repeating the process until the correct focus was obtained. Today many cameras have auto-focus technology that uses automated software to adjust the lens and take a focused picture. Still, such technology doesn’t have give users the nuance and control required for a particular, professional look. This focal sweep technology is the latest in camera focusing technology and it solves the issues plaguing previous approaches. When taking the picture, the acquisition is fully automated but when reviewing the range of images captures, it allows users to interactively choose between several different depths of focus. The viewer allows users to click an object in a scene and bring that object into focus, no matter whether or not the object is static in motion and without regards to where on the stack the object was captured.

A prototype of the technology has been tested and been shown to be capable of adjusting a camera lens and translating the sensor along the optical axis during video capture which is required for a focal sweep.

Lead Inventor:

Shree K. Nayar, Ph.D..

Applications:

  • Refocusing photo camera.
  • Taking professional pictures
  • Image stabilization
  • Merging images with different areas of focus
  • Low frame rate refocusing video camera
  • High-resolution surveillance camera
  • Software update to transform current cameras into light field cameras
  • Algorithm technology can be adapted into a 3D camera

Advantages:

  • Works with existing hardware
  • Decreases the number of flash photographs that need to be taken
  • More efficient use of battery power
  • Can be implemented as a software update
  • Can be implemented in software or hardware
  • Limits the time needed to get a high quality shot
  • Limits the number of additional photographs that must be taken
  • Obviates the need for a steep learning curve required to learn how to properly focus a camera for a range of shots

Patent Information:

Patent Pending (WO/2013/162747)

Tech Ventures Reference: IR CU14239

Related Publications: