[PDF] Tensor Based Spatio Temporal Outlier Detection In Large Datasets eBook

Author : Yanan Sun
Publisher :
Page : 322 pages
File Size : 49,21 MB
Release : 2014
Category :
ISBN :

GET BOOK

Spatio-Temporal data is inherently large since each spatial node has spatial attributes and may also be associated with large amounts of measurement data captured over time. In such large and multi-dimensional data identifying anomalies can be a challenge due to the massive data size and relationships among spatial objects. Discovering anomalies in spatio-temporal data is relevant in several domains, such as detecting rare disease outbreaks, detecting oil spills, discovering regions with highway traffic congestion. Most existing techniques for discovering anomalies in spatio-temporal data may find the spatial outliers first and then identify the spatio-temporal anomalies from the data of that specific spatial location. Alternatively, some approaches may discover anomalous time periods and then discover the unusual spatial location in them. This may lead to identifying incorrect spatio-temporal outliers or missing important spatio-temporal phenomena due to the elimination of information after each step. Thus, there is a need to address capturing both space and time simultaneously. A tensor is a multi-dimensional array. It is considered as a powerful tool to manipulate multi-dimensional and multi-variate data. It has a concise mathematical framework for formulating and solving complex data problems efficiently. Tensors can handle complex relationship in spatio-temporal data and their mathematical framework can help us detect spatio-temporal outliers in an effective and efficient manner. Tensors multiplication can integrate spatial and temporal aspects in the data at the same time. In this dissertation, we present our novel approach addressing the key limitation of existing spatio-temporal outlier detection methods by using an efficient tensor based model that supports complex relationships in spatio-temporal data to detect outliers by looking at space and time simultaneously as well as handling the scalability issue when it comes to manipulating large datasets. In this dissertation, we are going to present our novel spatio-temporal tensor model. Based on the spatio-temporal tensor model, we present our clustering-based neighborhood discovery algorithm, neighborhood-based spatial, and spatio-temporal outlier detection algorithms to discover different types of spatio-temporal outliers namely point based and window based outliers. We discuss detailed experimental results for each of the algorithms proposed and also present comparative results.

Author : Manish Gupta
Publisher : Springer Nature
Page : 110 pages
File Size : 28,72 MB
Release : 2022-06-01
Category : Computers
ISBN : 3031019059

GET BOOK

Outlier (or anomaly) detection is a very broad field which has been studied in the context of a large number of research areas like statistics, data mining, sensor networks, environmental science, distributed systems, spatio-temporal mining, etc. Initial research in outlier detection focused on time series-based outliers (in statistics). Since then, outlier detection has been studied on a large variety of data types including high-dimensional data, uncertain data, stream data, network data, time series data, spatial data, and spatio-temporal data. While there have been many tutorials and surveys for general outlier detection, we focus on outlier detection for temporal data in this book. A large number of applications generate temporal datasets. For example, in our everyday life, various kinds of records like credit, personnel, financial, judicial, medical, etc., are all temporal. This stresses the need for an organized and detailed study of outliers with respect to such temporal data. In the past decade, there has been a lot of research on various forms of temporal data including consecutive data snapshots, series of data snapshots and data streams. Besides the initial work on time series, researchers have focused on rich forms of data including multiple data streams, spatio-temporal data, network data, community distribution data, etc. Compared to general outlier detection, techniques for temporal outlier detection are very different. In this book, we will present an organized picture of both recent and past research in temporal outlier detection. We start with the basics and then ramp up the reader to the main ideas in state-of-the-art outlier detection techniques. We motivate the importance of temporal outlier detection and brief the challenges beyond usual outlier detection. Then, we list down a taxonomy of proposed techniques for temporal outlier detection. Such techniques broadly include statistical techniques (like AR models, Markov models, histograms, neural networks), distance- and density-based approaches, grouping-based approaches (clustering, community detection), network-based approaches, and spatio-temporal outlier detection approaches. We summarize by presenting a wide collection of applications where temporal outlier detection techniques have been applied to discover interesting outliers. Table of Contents: Preface / Acknowledgments / Figure Credits / Introduction and Challenges / Outlier Detection for Time Series and Data Sequences / Outlier Detection for Data Streams / Outlier Detection for Distributed Data Streams / Outlier Detection for Spatio-Temporal Data / Outlier Detection for Temporal Network Data / Applications of Outlier Detection for Temporal Data / Conclusions and Research Directions / Bibliography / Authors' Biographies

Author : Manish Gupta
Publisher :
Page : 0 pages
File Size : 40,79 MB
Release : 2014-03
Category : Outliers (Statistics)
ISBN : 9781627053754

GET BOOK

Outlier (or anomaly) detection is a very broad field which has been studied in the context of a large number of research areas like statistics, data mining, sensor networks, environmental science, distributed systems, spatio-temporal mining, etc. Initial research in outlier detection focused on time series-based outliers (in statistics). Since then, outlier detection has been studied on a large variety of data types including high-dimensional data, uncertain data, stream data, network data, time series data, spatial data, and spatio-temporal data. While there have been many tutorials and surveys for general outlier detection, we focus on outlier detection for temporal data in this book. A large number of applications generate temporal datasets. For example, in our everyday life, various kinds of records like credit, personnel, financial, judicial, medical, etc., are all temporal. This stresses the need for an organized and detailed study of outliers with respect to such temporal data. In the past decade, there has been a lot of research on various forms of temporal data including consecutive data snapshots, series of data snapshots and data streams. Besides the initial work on time series, researchers have focused on rich forms of data including multiple data streams, spatio-temporal data, network data, community distribution data, etc. Compared to general outlier detection, techniques for temporal outlier detection are very different. In this book, we will present an organized picture of both recent and past research in temporal outlier detection. We start with the basics and then ramp up the reader to the main ideas in state-of-the-art outlier detection techniques. We motivate the importance of temporal outlier detection and brief the challenges beyond usual outlier detection. Then, we list down a taxonomy of proposed techniques for temporal outlier detection. Such techniques broadly include statistical techniques (like AR models, Markov models, histograms, neural networks), distance- and density-based approaches, grouping-based approaches (clustering, community detection), network-based approaches, and spatio-temporal outlier detection approaches. We summarize by presenting a wide collection of applications where temporal outlier detection techniques have been applied to discover interesting outliers.

Author : Mahashweta Das
Publisher :
Page : 61 pages
File Size : 45,14 MB
Release : 2009
Category :
ISBN :

GET BOOK

Abstract: Recent advances in computational sciences have led to the generation and utilization of enormous amounts of spatio-temporal data in numerous scientific disciplines, such as wireless sensor networks, bioinformatics, astrophysics and computational fluid dynamics. The need to efficiently handle and effectively analyze such humongous amount of data has led to the application of data mining techniques in this domain of research. Spatio-temporal data mining is the process of maneuvering such datasets to extract interesting knowledge and meaningful insights from the data. In this thesis, we propose a spatio-temporal and a temporal-spatial anomaly detection algorithm in the context of wireless sensor network application. The latter can also be interpreted as a prediction model for spatio-temporal datasets. The anomaly detection algorithms identify local abnormalities in sensor data collected over space and time, independent of the global analytical view presented by the entire dataset. The first of our proposed algorithms identifies candidate spatial outliers by computing an outlierness indicator, which we call ANOI (Antimonotonic Outlierness Indicator) and then determines the spatio-temporal outliers by checking the stability of the candidates over time. Our novel idea of antimonotonicity in an outlier detection framework helps us to prune the search space and reduce computational complexity, which is a serious bottleneck for neighborhood-based outlier detection methodologies. The second algorithm first models time series data and then refines the temporal estimates by integrating spatial association information. Since the model predicts short-term energy availability based on past historical records, it can also assist wireless sensor nodes to automatically adapt to changing environmental conditions and use its energy harvesting and activity scheduling abilities intelligently. Experimental results on climate data empirically demonstrate the effectiveness of both the approaches.

Author : Vincent Lemaire
Publisher : Springer Nature
Page : 236 pages
File Size : 22,2 MB
Release : 2020-01-22
Category : Computers
ISBN : 3030390985

GET BOOK

This book constitutes the refereed proceedings of the 4th ECML PKDD Workshop on Advanced Analytics and Learning on Temporal Data, AALTD 2019, held in Würzburg, Germany, in September 2019. The 7 full papers presented together with 9 poster papers were carefully reviewed and selected from 31 submissions. The papers cover topics such as temporal data clustering; classification of univariate and multivariate time series; early classification of temporal data; deep learning and learning representations for temporal data; modeling temporal dependencies; advanced forecasting and prediction models; space-temporal statistical analysis; functional data analysis methods; temporal data streams; interpretable time-series analysis methods; dimensionality reduction, sparsity, algorithmic complexity and big data challenge; and bio-informatics, medical, energy consumption, on temporal data.

Author : Michael Gertz
Publisher : Springer
Page : 454 pages
File Size : 20,11 MB
Release : 2017-08-07
Category : Computers
ISBN : 3319643673

GET BOOK

This book constitutes the refereed proceedings of the 15th International Symposium on Spatial and Temporal Databases, SSTD 2017, held in Arlington, VA, USA, in August 2017.The 19 full papers presented together with 8 demo papers and 5 vision papers were carefully reviewed and selected from 90 submissions. The papers are organized around the current research on concepts, tools, and techniques related to spatial and temporal databases.

Author : N. N. R. Ranga Suri
Publisher : Springer
Page : 214 pages
File Size : 37,89 MB
Release : 2019-01-10
Category : Technology & Engineering
ISBN : 3030051277

GET BOOK

This book, drawing on recent literature, highlights several methodologies for the detection of outliers and explains how to apply them to solve several interesting real-life problems. The detection of objects that deviate from the norm in a data set is an essential task in data mining due to its significance in many contemporary applications. More specifically, the detection of fraud in e-commerce transactions and discovering anomalies in network data have become prominent tasks, given recent developments in the field of information and communication technologies and security. Accordingly, the book sheds light on specific state-of-the-art algorithmic approaches such as the community-based analysis of networks and characterization of temporal outliers present in dynamic networks. It offers a valuable resource for young researchers working in data mining, helping them understand the technical depth of the outlier detection problem and devise innovative solutions to address related challenges.

Author : Mohamed Ben Ahmed
Publisher : Springer Nature
Page : 1117 pages
File Size : 49,8 MB
Release : 2022-03-03
Category : Technology & Engineering
ISBN : 3030941914

GET BOOK

This book sets the innovative research contributions, works, and solutions for almost all the intelligent and smart applications in the smart cities. The smart city concept is a relevant topic for industrials, governments, and citizens. Due to this, the smart city, considered as a multi-domain context, attracts tremendously academics researchers and practitioners who provide efforts in theoretical proofs, approaches, architectures, and in applied researches. The importance of smart cities comes essentially from the significant growth of populations in the near future which conducts to a real need of smart applications that can support this evolution in the future cities. The main scope of this book covers new and original ideas for the next generations of cities using the new technologies. The book involves the application of the data science and AI, IoT technologies and architectures, smart earth and water management, smart education and E-learning systems, smart modeling systems, smart mobility, and renewable energy. It also reports recent research works on big data technologies, image processing and recognition systems, and smart security and privacy.

Author : Gerald Liso
Publisher :
Page : 71 pages
File Size : 27,77 MB
Release : 2018
Category :
ISBN :

GET BOOK

Network monitoring systems are important for network operators to easily analyze behavioral trends in flow data. As networks become larger and more complex, the data becomes more complex with increased size and more variables. This increase in dimensionality lends itself to tensor-based analysis of network data as tensors are arbitrarily sized multi-dimensional objects. Tensor-based network monitoring methods have been explored in recent years through work at Carnegie Mellon University through their algorithm DenseAlert. DenseAlert identifies events anomalous events in tensors through quick detection of dense sub-tensors in positive-valued tensors. However, from experimentation, DenseAlert fails on larger datasets. Drawing from RED Alert, we developed an algorithm called RED Alert that uses recursive filtering and expansion to handle anomaly detection in large tensors of positive and negative valued data. This is done through the use of network parameters that are structured in a hierarchical fashion. That is, network traffic is first modeled at low granular data (e.g. host country), and events detected as anomalous in lower spaces are tracked down to higher granular data (e.g. host IP). The tensors are built on-the-fly in streaming data, filtering data to only consider the parameters deemed anomalous in previous granularity levels. RED Alert is showcased on two network monitoring examples, packet loss detection and botnet detection, comparing results to DenseAlert. In both cases, RED Alert was able to detect suspicious events and identify the root cause of the behavior from a sole IP. RED Alert was developed as part of a greater project, InSight2, that provides several different network monitoring dashboards to aid network operators. This required additional development of a tensor library that worked in the context of InSight2 as well as the development of a dashboard that could run the algorithm and display the results in meaningful ways.

Author : Haoran Zhang
Publisher : Elsevier
Page : 224 pages
File Size : 18,83 MB
Release : 2023-01-29
Category : Business & Economics
ISBN : 0443184291

GET BOOK

Handbook of Mobility Data Mining, Volume One: Data Preprocessing and Visualization introduces the fundamental technologies of mobile big data mining (MDM), advanced AI methods, and upper-level applications, helping readers comprehensively understand MDM with a bottom-up approach. The book explains how to preprocess mobile big data, visualize urban mobility, simulate and predict human travel behavior, and assess urban mobility characteristics and their matching performance as conditions and constraints in transport, emergency management, and sustainability development systems. The book contains crucial information for researchers, engineers, operators, administrators, and policymakers seeking greater understanding of current technologies' infra-knowledge structure and limitations. Further, the book introduces how to design MDM platforms that adapt to the evolving mobility environment, new types of transportation, and users based on an integrated solution that utilizes sensing and communication capabilities to tackle significant challenges faced by the MDM field. This volume focuses on how to efficiently pre-process mobile big data to extract and utilize critical feature information of high-dimensional city people flow. The book first provides a conceptual theory and framework, then discusses data sources, trajectory map-matching, noise filtering, trajectory data segmentation, data quality assessment, and more, concluding with a chapter on privacy protection in mobile big data mining. Introduces the characteristics of different mobility data sources, like GPS, CDR, and sensor-based mobility data Summarizes existing visualization technologies of the current transportation system into a multi-view frame, covering the perspective of the three leading actors Provides recommendations for practical open-source tools and libraries for system visualization Stems from the editor’s strong network of global transport authorities and transport companies, providing a solid knowledge structure and data foundation as well as geographical and stakeholder coverage