System Maintenance Notice:
Single article purchases and IEEE account management are currently unavailable. We apologize for the inconvenience.
By Topic

Anomaly Detection Algorithms on IBM InfoSphere Streams: Anomaly Detection for Data in Motion

Sign In

Cookies must be enabled to login.After enabling cookies , please use refresh or reload or ctrl+f5 on the browser for the login options.

The purchase and pricing options are temporarily unavailable. Please try again later.
3 Author(s)
Yulevich, Y. ; Software Lab., Software Solutions Dept., IBM Israel Software Lab., Rehovot, Israel ; Pyasik, A. ; Gorelik, L.

This paper presents and shares excerpts from our implementation of near real-time anomaly detection algorithms on the IBM InfoSphere Streams platform. The purpose of this article is to: 1) Describe how to design and implement known anomaly detection algorithms on IBM InfoSphere Streams. 2) Present some performance optimization capabilities of IBM InfoSphere Streams platform and propose a method to use them in anomaly detection applications. 3) Present some IBM InfoSphere Streams best practices and describe how their adoption in the context of anomaly detection application. The document describes the architecture and design of anomaly detection algorithms developed on IBM InfoSphere Streams. Although the solution was designed to be used for cyber security, the implemented algorithms are agnostic regarding the data type that they monitor and therefore can detect anomalies in data from various industries such as healthcare, finance and retail. The document describes the implementation of two anomaly detection algorithms: KOAD and PCA. The KOAD algorithm performs online anomaly detection with incremental learning and the PCA algorithm in performs offline anomaly detection. The solution was designed to provide near real-time insight into low latency on large data volume observation.

Published in:

Parallel and Distributed Processing with Applications (ISPA), 2012 IEEE 10th International Symposium on

Date of Conference:

10-13 July 2012