Development of a Water Quality Event Detection and Diagnosis Framework in Drinking Water Distribution Systems with Structured and Unstructured Data Integration

Taewook Kim, Donghwi Jung, Do Guen Yoo, Seunghyeok Hong, Sanghoon Jun, Joong Hoon Kim

Research output: Contribution to journalArticlepeer-review

Abstract

Recently, various detection approaches that identify anomalous events (e.g., discoloration, contamination) by analyzing data collected from smart meters (so-called structured data) have been developed for many water distribution systems (WDSs). However, although some of them have showed promising results, meters often fail to collect/transmit the data (i.e., missing data) thus meaning that these methods may frequently not work for anomaly identification. Thus, the clear next step is to combine structured data with another type of data, unstructured data, that has no structural format (e.g., textual content, images, and colors) and can often be expressed through various social media platforms. However, no previous work has been carried out in this regard. This study proposes a framework that combines structured and unstructured data to identify WDS water quality events by collecting turbidity data (structured data) and text data uploaded to social networking services (SNSs) (unstructured data). In the proposed framework, water quality events are identified by applying data-driven detection tools for the structured data and cosine similarity for the unstructured data. The results indicate that structured data-driven tools successfully detect accidents with large magnitudes but fail to detect small failures. When the proposed framework is used, those undetected accidents are successfully identified. Thus, combining structured and unstructured data is necessary to maximize WDS water quality event detection.

Original languageEnglish
Article number9300
JournalEnergies
Volume15
Issue number24
DOIs
Publication statusPublished - 2022 Dec

Bibliographical note

Publisher Copyright:
© 2022 by the authors.

Keywords

  • anomaly detection
  • framework
  • structured and unstructured data integration
  • water distribution system
  • water quality
  • water quality event

ASJC Scopus subject areas

  • Control and Optimization
  • Energy (miscellaneous)
  • Engineering (miscellaneous)
  • Energy Engineering and Power Technology
  • Electrical and Electronic Engineering
  • Fuel Technology
  • Renewable Energy, Sustainability and the Environment

Fingerprint

Dive into the research topics of 'Development of a Water Quality Event Detection and Diagnosis Framework in Drinking Water Distribution Systems with Structured and Unstructured Data Integration'. Together they form a unique fingerprint.

Cite this