Overview | People | Collaborators | Sponsors | Publications | Tools

The Global War on Terror presents unique challenges in its intelligence requirements, due to the need to focus on decentralized network threats from non-state entities. In this setting, key information may be contained in the actions of individuals or small groups, activities which may be easily hidden within routine civilian activity in an urban environment. To detect these threats and create actionable intelligence to support expeditionary war fighting, one of the few available sensor modalities is unattended surveillance video, perhaps the only currently-available technology with the promise of both detecting and identifying individuals at a distance and without their knowledge. While such video is one of the most ubiquitous real-time sensing modalities available; paradoxically, it is not generally used for real-time interdiction. Traditional video surveillance systems require hundreds or thousands of cameras that are usually backhauled to a central monitoring location for storage on media or real-time display on a video monitor. The only hope for turning terabytes of video data into actionable intelligence on human activities comes in combining state-of-the-art computer vision techniques with new developments from quantitative studies of human behavior and organization.

Terrorist organizations have social network structures that are distinct from those in typical hierarchical organizations. They are cellular and distributed, making it difficult to understand how such networks evolve and adapt. Many attempts at static modeling of these networks generate information that is misleading, out-of-date, or incomplete. Recently, there has been interest in dynamic network analysis, which extends modeling to include multiple co-evolving networks under conditions of information uncertainty. Networks are built out of people, resources, locations, and organizations, so that a group such as a terrorist network can be represented in terms of a time-sequence of events linking these entities. Co-attendance at an event for two people, or co-incidence at spatially adjacent locations suggests a tie between these two people. The network is constructed out of these linkages, taking into account the probabilistic nature of all inferred ties and the expectation that the network may be constantly changing, such as with the addition or removal of members.

In this project, dynamic network analysis techniques will be applied to outputs derived from the video data, inferring connections between co-incidence of people in spatially-adjacent locations or in association with key locations or events. Techniques will be developed to adaptively alter the inferred network over time, with the capability to incorporate new individuals, locations, or events as they become identified as being relevant to terrorist activities. All metrics developed by CMU are incorporated into *ORA. *ORA reports are called by the overall TAVI system. The result is an integrated tool that captures video, encodes it as networks, and then assesses those networks.