Hendon Publishing - Article Archive Details
Intelligent Surveillance Systems
Written by Pat Kiernan
Video surveillance cameras have become a mainstay of modern existence. They peer down from high perches in city centers. They’re found in buildings and transit centers where safety and security are paramount. We spy them at countless other venues in our everyday lives, while they spy us. In the U.S. alone, more than 30 million surveillance cameras capture billions of hours of video footage each week. Video surveillance has grown into a $160 billion global industry and emerged as one of the pre-eminent high-tech security and crime-fighting tools of the 21st century.
While all eyes are on video surveillance, our vision isn’t always 20/20 when it comes to selecting the right video surveillance solution. Many factors come into play. If you’re considering video surveillance for your city center or law enforcement operation, here are eight tips you can use to improve your video surveillance solution’s success.
Tip #1: Proven Expertise
In the early days of CCTV, the camera passively recorded video, as the human eye on the other side of the camera kept watch. But as it turned out, humans were ill-suited to the task of threat detection, especially when it came to staring at hordes of video screens for hours on end. Research shows that after just 20 minutes of watching several monitors, operators can overlook up to 95% of what they’re “seeing” on the screens. Other research points out that an operator responsible for monitoring anywhere from 9 to 12 cameras will start to tune out after a mere 15 minutes.
While humans are not adept at the rote and repetitive task of video monitoring—technology is. Hence the appeal of modern-day intelligent surveillance systems that employ content analytics to automate threat detection. These analytics are essentially pre-defined algorithms and rules built into the video surveillance system. This can monitor video from thousands of cameras, 24/7, without complaint, distraction, boredom or breaks.
Through content analytics, an intelligent surveillance system can detect many different types of threats, conditions, and behaviors that could be overlooked by the human eye. When a threat is detected, the system signals a human operator to take action.
Unheard of in the early days of CCTV, intelligent surveillance systems are becoming more the norm. They can identify many different kinds of potential security infractions, such as: intrusions (unauthorized entry into secured areas); “tailgating” (scenarios where two or more people pass through a secured door using the same security pass); counter-flow (people or objects moving in the wrong direction); abandoned bags (in subway stations or airports); vehicles parked in restricted areas; perimeter breaches; and excessive crowds.
Intelligent content analytic-based video solutions offer more capability than early generation CCTV systems, but they’re also more complex. “Beware of vendors who claim their solutions are plug-and-play,” said Michael Rubinov, director of marketing for NICE Systems. “When it comes to smart video solutions, there’s no such thing. Typically, these are ether very inexperienced vendors or their systems are designed for specific indoor environments and have very limited capabilities.”
The process of implementing a content-analytics based solution begins with analyzing and understanding customer needs. A dialogue is initiated with the customer to determine what problem he is trying to solve with analytics. Then those expectations are matched with what the technology can deliver.
The next step is a detailed site survey (essentially a thorough inventory and analysis of processes, people, locations, infrastructure, environmental factors, etc.)—all of the different elements that can have an impact on the system’s design and implementation.
For example, if the customer has guards walking the perimeter but would like to change that so they’re stationary and only respond when the perimeter is breached, the starting point would be to review the cameras in place and the distances they cover. Next, environmental factors would be examined—such as anything blocking the view of the cameras; or problems that could be caused by heavy rain, snow, fog, abrupt light changes, sun, and so on. Any obstacles that could potentially cause problems for the algorithms would need to be identified.
In a controlled indoor environment, it is typically easier to implement analytics, but an outdoor environment introduces more variables that can lead to higher misdetections and false alarms. The analytics may work fine in the controlled environment, but once you take the system outdoors, you can encounter problems. For example, a cloud in the sky that makes a shadow on the ground could be mistaken for a stopped vehicle, and that could trigger a false alarm.
False alarms are problematic. In the best-case scenario, they consume time, energy and resources. In the worst case, a false alarm misinterpreted as a looming threat causes buildings to be evacuated and bomb squads to be dispatched.
No technology is foolproof, and you can get false alarms with analytics. But too many false alarms generate a lot of non-relevant data that can try a customer’s patience. The remedy is to make sure that the smart video system is tried and tested in its natural environment. When a system is installed, it should always be calibrated properly to minimize the occurrence of false alarms.
The analytics should also be flexible enough for automated time scheduling. For instance, an area that might be perfectly normal for people to congregate in during the daytime could be off bounds at night. A fly-by-night or inexperienced vendor can be weeded out by examining the vendor’s track record. Even more specifically, you should ask for reference sites—with like needs, environments (indoor/outdoor) and applications.
Tip #2: Tight Integration
Select a solution that offers tight integration between content analytics and video recording for immediate human visual verification. Even with the right degree of calibration, a smart video solution may not be able to distinguish between threatening and non-threatening events. Why? Because while content analytics excels at differentiating between binary conditions (e.g. did someone cross a perimeter or walk through a gate the wrong way?), it doesn’t view the event in the bigger context of human understanding. It can’t infer intent.
Imagine for a moment you’re an operator working in a control room that monitors thousands of cameras encircling a city that is also home to a well-known national landmark. Suddenly, an alert sounds, reverberating through the center. You immediately look down to one of your monitors to see an abandoned bag—the size and shape of a knapsack—sitting limply at the base of the monument. Your eyes swivel to the video wall where you view throngs of tourists taking in the sights and scenery on the esplanade, just adjacent to the monument.
You need to act quickly. Is the bag a concealed threat? Or simply a knapsack left behind by a forgetful sight-seer? Instead of erring on the side of caution and dispatching the police, you step over to the video recorder and frantically search for that one segment of video that will help you solve the mystery. Minutes tick away as your frustration escalates while you rewind and fast forward again and again.
Now, let’s revisit this scenario—but this time assuming the city had implemented a solution where content analytics and video recording were tightly integrated. The alarm sounds. Your attention instinctively shifts to your monitor screen. But this time you see something different. The suspicious object—the abandoned bag—is circled in red on the video screen. You click on the object with your mouse, and within a split second, video starts to replay from the point where the bag first appeared in the scene.
You see a group of a dozen or so school children accompanied by a teacher arrive at the monument and momentarily gaze up at the statue. Less than a minute later, the teacher corals the children and motions them on. They swiftly follow her toward the esplanade. The last lone child absentmindedly walks off, leaving his backpack.
The level of immediate visual verification in the latter scenario can only be achieved when content analytics are intimately integrated with video recording. Here’s how it works. The camera outputs a stream of bytes of data. That data (video stream) is analyzed by the detection algorithm to screen for objects, and accordingly tagged with meta-data. Another rules engine then determines whether or not a detected object is suspicious. If it’s determined to be suspicious, the video stream is marked with a unique tag—so when there’s an alert, the system can immediately jump to the first time in the video that the suspicious object appeared in the scene.
In a potentially dangerous situation, time is critical. A control room operator needs to assess the nature and seriousness of the threat right away. The operator needs to know why the bag is there and who put it there. There’s no time to fast forward and rewind to look for the video.
Mark Jaimes, director for The CBORD Group, concurred. “Basic non-integrated video surveillance solutions can show you after the fact what happened, and it may take some effort to find it. We’re saying we can show you what’s happening while it’s happening. There’s a big difference there. The moment somebody kicks in the door and you have an alarm going off, you’re notified about that with an alarm, and you can see the pertinent video at that same time.”
Headquartered in Ithaca, NY, CBORD specializes in integrated security solutions that combine card systems, access control and intelligent video surveillance devices, for college campuses and other security-minded environments. The company worked with NICE Systems to integrate its CS Access™ IP-based access control technology with NICE Systems’ NiceVision™ digital video surveillance technology.
The NICE/CBORD solution combines real-time video analysis and recording with access control. The idea is to place video cameras at strategic points across campus, tied into access control. Different access breaches then generate alerts, which operators can view, along with video and other information, through a unified interface. Operators instantly know when an alert occurs, where it happened, even what card was involved. The system simultaneously pushes video content to them so they can screen out false alarms from legitimate security threats.
Most access control systems are designed for a specific location; but in a campus, there can be dozens or even hundreds of buildings that require some form of access control, so it presents a unique challenge.
In such environments, integrating real-time video analysis and recording with access control takes some of the security burden off of campus police departments. On a campus, for example, door props occur all of the time. The department might not want to dispatch a police officer every time a door prop happens, but an operator might want to take a look at what’s going on.
With an integrated solution, the operator will get that alert automatically and can then either click on the event in the access log or click on the alarm on the alarm map, and that signals the system to automatically display the live video or retrieve the recorded video, typically from 15 seconds before the event to 15 seconds after.
An integrated solution can help operators ferret out false alarms, but it’s even more vital in situations where time is of the essence. Without it, an operator might have to pull information from different systems, a slow solution to a crisis that requires a quick response. Additionally, by the time an operator is able to find and review the video, the person of interest is usually gone.
Today, many universities have the ability to completely lock down a campus. But that’s only part of the solution because in a lockdown situation, people might, in fact, be locked out of buildings where they need to be. Having the video and the access control systems tied together enables operators to visualize what’s happening and make faster and more accurate determinations for decision making.
Tip #3: Automate the Response
When a threat is detected and visually verified, urgent and informed action is needed. The right video surveillance system can help to automate and improve your response. Pick a solution that can scale automatically, reduce video bandwidth, and stream video onto handheld devices (for example, a PDA), so that the same video seen by the control room operator can be streamed in real time to first responders or incident commanders in the field. This capability can arm officers with vital visual information and provide decision support on how to respond before they even arrive on the scene.
Second, look for a solution that can incorporate predefined response scenarios to specific types of events so an operator immediately knows what to do. When a threat is detected, instructions specific to that incident type can automatically pop up on the operator’s screen, directing what steps to take, whom to contact, and providing other essential information.
Tip #4: Open Solutions
The fourth tip is to select a video surveillance solution that is based on an open system. Choose a proprietary solution and you could get pigeonholed into a system with limited capabilities and a short lifespan. An open system is built from standard off-the-shelf computer components, rather than proprietary hardware. The software is also open, which means that it has special device drivers that enable it to integrate to sensors, access control devices, cameras, user keyboards and security software.
For example, some city centers employ sensing devices, such as gunshot detectors, coupled with video surveillance to alert control center operators when a gun is fired. Another example is optical character recognition, which is used to recognize and read license plates, or biometrics for facial recognition. Having an open system allows you to leverage these and other best-of-breed technologies and applications as part of your intelligent video solution.
An open system is important for integrating peripherals, such as IP video cameras, as well. There are no standard protocols for IP cameras. Still, if a new deca-Pixel camera comes out in the next two weeks, you may want your video surveillance solution to be able to support it. There are also many different types of custom keyboards that are used in security applications and different varieties of video monitors and video walls.
The true test of an open system is the time and effort involved in the integration. It should be a two-week testing procedure, not a half-year software project—which will save time and money in the end.
Tip #5: Encoder Redundancy
In smart video surveillance solutions, the analytics (that is, the analysis of the video for real-time detection and alerting) either takes place in a central server or on the network edge in an encoder. A small embedded device attached via coax to one or more analog cameras, the encoder also serves a secondary but no less important function—it stores analog video locally and then converts it to a digital stream that can be transmitted over the network. As the industry migrates to IP video networks, more and more cameras are connected to encoders.
Whether analytics are done on the edge of the network or in a central server, the encoder is a mission-critical component of the video surveillance system. Either way, if an encoder fails, the analytics won’t function and the video feed won’t be streamed. In some ways, the encoder is analogous to the eyes of the video surveillance system because without it, the control room operator wouldn’t be notified of an impending threat.
One way to ensure that these critical functions are not compromised is to invest in a solution with built-in encoder redundancy. If you make the decision to use video content analytics, make sure that your system uptime is 99.999%. You can do this by implementing a zero points of failure architecture using “N+1” redundancy on encoders, NVRs and network configuration. With N+1 encoder redundancy, if one encoder unexpectedly goes down, the live video stream will automatically be sent to the next one.
Tip #6: Seamless Migration
Today, the lion’s share of video surveillance cameras in use—more than 90%—are of the conventional analog variety. But with the convergence of IT and security, IP cameras are poised to make significant headway. By one estimate, in just two year’s time, a third of all video cameras shipped will be IP-based. IP camera technology is promising for many reasons—ease of integration, lower cost of deployment, embedded analytics, improved image quality, and the ability to leverage the Internet to transmit multiple data streams over virtually limitless geographic distances.
Still, for its many impressive benefits, IP video has some hurdles too, not the least of which is a security industry resistant to change. While many organizations are looking toward the IP light at the end of the tunnel, they’re cautiously keeping one eye on their existing investments as well. In short, they’re migrating to IP video surveillance at their own comfortable pace.
With this in mind, it’s important to choose your video surveillance solution accordingly. Look for a system that accommodates both analog and IP cameras so you can maintain your current infrastructure while having a path to migrate without doing a major overhaul.
Here are some things you should look for. First, the system should support hybrid systems and be capable of recording any type of camera—analog or IP. Typically in an analog-only CCTV camera system, the analog video cameras are connected directly to an encoder or to a DVR (digital video recorder) device. In a hybrid system, a network video recorder (NVR) can accept digital streams from IP cameras and from analog cameras (via the encoders or DVRs).
Also look for a solution that allows you to deploy the same content analytics across your hybrid IP/analog infrastructure. Some vendors may only offer the analytics on IP cameras or analog, but not on both. Regardless of the infrastructure that’s in place—be it video over IP, analog, or some combination of both, make sure the algorithms are the same, so you have the same level of detection system-wide.
Also ensure that the solution you select offers unified management capabilities for your hybrid system so IP and analog cameras, DVRs, NVRs, encoders, and so on, can be managed and controlled through one application and one GUI. In the case of a mixed environment, as a system user, you don’t care if you’re looking at IP or analog cameras, so the system should make it seamless for you.
Tip #7: Navigate the Requirements
Implementing a video surveillance solution can be a bit like navigating a labyrinth. Conquering the maze of complexities requires a knowledgeable guide. Below is a sampling of some of the variables and questions you will encounter along the way. An experienced solution provider will have in-depth knowledge in all of these areas to analyze and evaluate your specific needs, help you weigh the pros and cons, and put you on the path to an optimal solution.
One of the first questions that typically comes up is, “What’s better—a distributed or centralized solution?” The answer might not be as simple as you think. For example, a university public safety department might need video cameras placed in far-flung buildings across campus. Add bandwidth limitations into the equation and you have a good argument for recording locally at each site (from analog camera to encoder to DVR).
Storing video on the edge is less taxing on your LAN because you’re not transmitting all of the video over the network to a centralized storage device. You don’t need the same high network availability that would be required with a centralized solution because video is only being pulled back for monitoring and replay.
But there is one potential disadvantage to the latter approach. Because of its distributed nature, you may end up paying for more storage than you need. A centralized architecture, on the other hand, offers certain economies of scale by providing a shared storage device (an NVR), which serves as a repository for all the video on the network.
If your system is analytics-based or tied into a sensing device for detection, you also have the option of only recording for specific events (instead of total recording), which can also reduce bandwidth and storage consumption. Aside from analytics and storage, there are numerous other variables that come into play when designing and implementing a video surveillance solution. Consider these examples.
Integration: Does the solution need to work with other solutions in your environment (e.g. sensing devices, access control) in order to provide the desired capabilities? Scheduling: Should analytics / alerting be activated only at certain times of the day? Can centralized recording be scheduled after hours to optimize network bandwidth? Resiliency: How mission critical is the application? What are the critical potential points of failure in the system? What level of fault tolerance is acceptable? How much resiliency is needed?
Compression: The type of video compression employed can have a direct impact on video quality, storage costs and network bandwidth. For example MPEG-4 employs a method of compression that only transmits the changes in a scene. Motion JPEG (M-JPEG) compresses and transmits each frame of video as a JPEG image. The latter method generally produces higher quality video but consumes more bandwidth. Depending on your specific environment, either could be suitable.
If you plan to use content analytics, you might elect to vary the compression rate (e.g. lower compression for video with detected events, higher compression for all other video). If you’re compressing the video before it’s sent over the network for content analysis, make sure the algorithms are tested and proven to work on the compressed video.
Frame rates: This is the rate (frames per second) at which the video is recorded. Most motion pictures and TV shows we view are anywhere between 24 and 30 frames per second. Exactly how much we can see is debatable, but some say the limit of the human eye is 30 frames per second. From a video surveillance standpoint, the application (why and what are you recording), the environment (your network bandwidth), the cost (your budget) and risk factors (the impact of not capturing something) dictate what frame rate to use.
If you want to implement different frame rates for recording, live monitoring and replay, then look for a solution that offers a flexible level of service framework. You might want to record at a high frame rate / high resolution and then employ a lower frame rate for viewing, especially if you have bandwidth restrictions—for instance if someone’s dialing in from home over the Internet or monitoring via a satellite link.
Monitoring requirements: What monitoring capabilities do you need, and how can you achieve these at the lowest cost? Here, the type of switch you employ makes a big difference. When it comes to traditional analog switches—(visualize an old telephone-type switchboard where an operator manually connects an incoming call with a patch cable)—there are a lot of cons. A single analog switch can cost thousands of dollars and has physical limitations in terms of the number of monitors it can support because the switching is in essence hard wired.
In a “Virtual Matrix” solution, on the other hand, the DVR/NVR becomes the input point, and the switching is performed by software, which means that any camera can be switched to any monitor. There is no physical limit to number or type of monitors (analog or digital) that can be supported, and video can be played back and monitored live simultaneously.
Special environments, i.e., wireless: Cabling is the most expensive component of most video surveillance installations, and that is making Wi-Fi networks very popular, especially for city center video surveillance systems that need to span wide distances. While increasing in popularity, wireless can also add complexity to a video surveillance project because there are many different types of wireless technologies (e.g. mesh, canopy, H.2.11) that behave and function differently.
With various wireless technologies, the latency of the packets is different—where the data packets don’t arrive in the right order—and that can cause jitter. The video surveillance solution needs to be designed, tested, and proven to withstand these anomalies to function properly. At the end of the day, the video surveillance solution provider you choose needs to deliver a solution that’s tested and proven for your environment.
Tip #8: From Detection to Prosecution
If you employ content analytics to detect threats, those same analytics can be applied to the investigative process too. For example, content-based video searches can be performed to identify video that’s likely to be of interest in an investigation (e.g. video containing suspicious objects, movements, behaviors or events).
While surveillance video is becoming increasingly common in investigations, the outcome of a case can often hinge on other forms of multimedia, such as 9-1-1 and first responder voice recordings, GPS data, and even cell phone photos. The problem is that most of this information is typically stored in different formats and different systems that weren’t designed for interoperability, and that makes it difficult to connect the dots in an investigation.
By employing a multimedia incident information management solution that links voice, video and other information from different sub-systems into a common interface, you can increase the efficiency and effectiveness of your investigations. With one query, you can assemble multimedia information from different sources to get a complete audible and visual timeline of an incident from the first detection of an event via the video or 9-1-1 call to the time that first responders arrived on the scene.
The multimedia incident reconstruction can then be stored in an incident folder along with other information and evidence, such as crime scene photos, fingerprints, arrest records, incident reports, etc. All of the captured information can be authenticated through the use of digital signatures and shared with other agencies, investigating units, and district attorneys.
In the final analysis, the best advice is to approach video surveillance as a building block to a larger holistic solution that can boost your efficiency and effectiveness at every security cycle stage, from detection to prosecution. If you try to piecemeal a solution together, it isn’t going to accomplish what you want to do at the end of the day.
Patrick Kiernan is the director of marketing for public safety for NICE Systems. He has more than 20 years of experience in the telecommunications sector and holds an MBA from American University in Washington, DC. Kiernan can be contacted at firstname.lastname@example.org.
Published in Law and Order, May 2008
Rating : Not Yet Rated
Related CompaniesNice Systems Inc.
Related ProductsSurveillanceSurveillance EquipmentVideo SurveillanceVideo Technology
Click to enlarge images.