Camera motion for image deblurring

Summary: 




Project Description: 
High resolution surveillance cameras have a wide range of applications, such as airport surveillance, event monitoring and traffic safety. Circumstances in traffic can create quite a challenging environment for capturing high quality footage. The combination of high speed, possible dark areas at night and high resolution requirements for license plate recognition demand extremely light-sensitive camera systems. In practice, motion blur is one of the most common types of image degradations. At highway speeds, the exposure time can be no longer than a few milliseconds in order to “freeze” motion. In darker circumstances this will yield a very poor Signal-to-Noise Ratio (SNR). Up until now, cameras have been equipped with more sensitive sensors, large aperture lenses and an external flash in order to get a higher SNR. However, these approaches are costly and there are physical limits to sensor and lens performance.

Figure 1: UHD camera from Prodrive Technologies

Our approach is to shift the camera field of view while the camera shutter is open. This allows us to “shape” motion blur in a way which is very easy to deblur afterwards. There is no need to freeze motion anymore and thus the exposure time can be greatly increased, which in turn increases the SNR. An off-the-shelf DSLR lens with Optical Image Stabilizer is modified in order to shift the field of view. A small module shifts a lens in the optical path up to 1 mm (almost 200 pixels) in 2 axes. The lens position controller is carefully tuned to provide a settling time as low as 10 ms and a positioning accuracy within 15 μm. The lens motion trajectory is designed to optimize the numerical stability of the blur removal. Hence, the motion trajectory depends on the velocities of interest in a scene. The more accurate information about object velocities is available prior to the exposure, the better the lens motion can be matched to those velocities and the better the final image quality.

Figure 2 Optical Image Stabilizer lens shift module extracted from Canon EF 28-135 f/3.5-5.6 IS USM lens.

Experiments in a controlled testing environment have shown substantial image quality improvements, both in terms of sharpness and SNR. However, several issues need to be overcome in practical situations. For instance, the finite acceleration and limited position accuracy of the lens shift module causes deviations from the theoretically optimal motion trajectory. Furthermore, the limited dynamic range of cameras might cause visible artefacts around bright light sources. The current research focus is on improving the robustness of the system in practical applications. This project is a cooperation between Eindhoven University of Technology and Prodrive Technologies B.V. (Son, the Netherlands).

Figure 3 Left: Normal camera (blurred vehicle). Middle: Photo with camera motion. Right: Middle photo deblurred.

Application Area: 
Surveillance
Video/Imaging Discipline: 
Content Analysis
Participants: 
Partners: 
Partners: 

Eindhoven University of Technology SPS-VCA Research group at Eindhoven and Prodrive Technologies B.V. at Son