Improvised Explosive Devices (IEDs) are one of the main causes of casualties amongst troops during transportation of people and materials in conflict zones. One effective method of periodical surveillance of high-risk areas is ground-based patrol. During such patrols, potential threats are localized by searching for suspicious patterns in the environment. Any suspicious change in the environment (new objects, moved structures, etc.) may form a potential threat. The current project uses stereo cameras to capture the 3D geometry of the scene, so that any changes like newly placed objects can be found by advanced image processing using both the texture and the depth of the video signals.
Improvised Explosive Devices (IEDs) are one of the main causes of casualties amongst troops during transportation of people and materials in conflict zones. In order to reduce casualties, periodical surveillance of the high-risk areas is required in areas where such transportation is planned. One effective method of surveillance is ground-based patrol. During such patrols, potential threats are localized by searching for suspicious patterns in the environment (e.g. suspiciously placed objects, markers, etc.) and comparing current and past environment situations. This comparison usually has to be made by military personnel, while the convoy is moving. This involves the need to remember the environment state, as observed during a previous patrol. Any suspicious change in the environment (new objects, moved structures, etc.) may form a potential threat, so that the soldier is effectively performing a manual intelligence task focusing on change detection to find possible IEDs.
This is a very demanding task for a human, because his ability to concentrate on a task for a longer time interval in an unknown environment is limited. Furthermore, memorizing multiple details about the appearance of a specific environment is hard if the time and distance difference is significant. This project focuses on developing a real-time change detection system using automated image analysis. This system can aid the personnel involved in detecting IEDs and thereby help prevent accidents during surveillance.
The current project is a follow-up of an earlier change detection project for countering IEDs, which employed a monocular (single) camera to detect changes in the scene. The current project uses stereo cameras to capture the 3D geometry of the scene. This has several obvious advantages, such as the possibility to segment the scene in 3D and use volumetric verification of detected changes. A less obvious advantage lies in improved robustness to shadows and other lighting conditions. While a shadow will result in strong differences in RGB color space, it will not affect the depth of the scene. This allows us to reject false alarms due to shadows.
One of the main challenges in the current project is the large viewpoint difference between live and historic scene. For safety reasons, convoys try to drive in unpredictable trajectories, meaning they will drive on different parts of the road if possible. This results in large viewpoint differences between the live and historic scene, which have to be compensated in order to apply pixel-wise comparison techniques. This can be achieved through 2.5D scene alignment, which involves simulating the historic scene as if viewed by the live camera. The historic texture is projected onto a 3D shoebox model of the historic scene (Figure 2), which is transformed such that it aligns with the 3D live scene. This transformed 3D model is then projected back to 2D, allowing for pixel-wise comparison between the live and reference scene.
For more information on the project or methods applied, the reader is referred to the publication page of the participants.