HR: 1340h
AN: H13A-0979    [Abstracts]
TI: Real-Time Bayesian Anomaly Detection in Streaming Environmental Data
AU: * Hill, D J
EM: djhill1@uiuc.edu
AF: National Center for Supercomputing Applications, 4018 NCSA, MC-257, 1205 W. Clark St, Urbana, IL 61801, United States
AU: Minsker, B S
EM: minsker@uiuc.edu
AF: University of Illinois at Urbana-Champaign, 3230d Newmark Lab, MC-250, 205 N. Mathews Ave., Urbana, IL 61801, United States
AU: Amir, E
EM: eyal@cs.uiuc.edu
AF: University of Illinois at Urbana-Champaign, 3314 Siebel Center, 201 N. Goodwin Rd., Urbana, IL 61801, United States
AB: Recent advances in sensor technology are facilitating the deployment of sensors into the environment that can produce measurements at high spatial and/or temporal resolutions. Not only can these data be used to better characterize systems for improved modeling, but they can also be used to improve understanding of the mechanisms of environmental processes. With large volumes of data arriving in near-real time, however, there is a need for automated anomaly detection to identify data that deviate from historical patterns. These anomalous data can be caused by sensor or data transmission errors or by infrequent system behaviors that may be of interest to the scientific or public safety communities. This study develops and evaluates two automated anomaly detection methods that employ Dynamic Bayesian Networks (DBNs). Dynamic Bayesian networks are Bayesian networks with network topology that evolves over time, adding new state variables to represent the system state at the current time. Filtering (e.g. Kalman filtering or Rao-Blackwellized particle filtering) can then be used to infer the expected value of unknown system states, as well as the likelihood that a particular sensor measurement is anomalous. Measurements with a high likelihood of being anomalous are classified as such. The methods developed in this study perform fast, incremental evaluation of data as it becomes available; scale to large quantities of data; and require no a priori information regarding process variables or types of anomalies that may be encountered. Furthermore, these methods can be extended to large networks of heterogeneous sensors and can consider several data streams at once, using all of the streams concurrently to perform coupled anomaly detection. This study investigates these methods' abilities to identify anomalies in eight meteorological data streams from Corpus Christi, Texas. The results indicate that DBN-based detectors, using either robust Kalman filtering or Rao-Blackwellized particle filtering outperform a DBN-based detector using Kalman filtering. These methods were successful at identifying data anomalies caused by two real events: a sensor failure and a large storm.
DE: 1817 Extreme events
DE: 1848 Monitoring networks
DE: 1855 Remote sensing (1640)
DE: 1894 Instruments and techniques: modeling
DE: 1895 Instruments and techniques: monitoring
SC: Hydrology [H]
MN: 2007 Fall Meeting