Effective Density-Based Concept Drift Detection for Evolving Data Streams

Zelin Cui, Hui Tian, Hong Shen

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Concept drift is a common phenomenon appearing in evolving data streams of a wide range of applications including credit card fraud protection, weather forecast, network monitoring, etc. For online data streams it is difficult to determine a proper size of the sliding window for detection of concept drift, making the existing dataset-distance based algorithms not effective in application. In this paper, we propose a novel framework of Density-based Concept Drift Detection (DCDD) for detecting concept drifts in data streams using density-based clustering on a variable-size sliding window through dynamically adjusting the size of the sliding window. Our DCDD uses XGBoost (eXtreme Gradient Boosting) to predict the amount of data in the same concept and adjusts the size of the sliding window dynamically based on the collected information about concept drifting. To detect concept drift between two datasets, DCDD calculates the distance between the datasets using a new detection formula that considers the attribute of time as the weight for old data and calculates the distance between the data in the current sliding window and all data in the current concept rather than between two adjacent windows as used in the exiting work DCDA [2]. This yields an observable improvement on the detection accuracy and a significant improvement on the detection efficiency. Experimental results have shown that our framework detects the concept drift more accurately and efficiently than the existing work.

Original languageEnglish
Title of host publicationParallel and Distributed Computing, Applications and Technologies - Proceedings of PDCAT 2023
EditorsJi Su Park, Hiroyuki Takizawa, Hong Shen, James J. Park
PublisherSpringer Science and Business Media Deutschland GmbH
Pages190-201
Number of pages12
ISBN (Print)9789819982103
DOIs
Publication statusPublished - 2024
Event24th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2023 - Jeju, Korea, Republic of
Duration: 16 Aug 202318 Aug 2023

Publication series

NameLecture Notes in Electrical Engineering
Volume1112 LNEE
ISSN (Print)1876-1100
ISSN (Electronic)1876-1119

Conference

Conference24th International Conference on Parallel and Distributed Computing, Applications and Technologies, PDCAT 2023
Country/TerritoryKorea, Republic of
CityJeju
Period16/08/2318/08/23

Keywords

  • Concept-Drift Detection
  • Data Mining
  • Data-Stream Clustering
  • Machine Learning

Fingerprint

Dive into the research topics of 'Effective Density-Based Concept Drift Detection for Evolving Data Streams'. Together they form a unique fingerprint.

Cite this