From soft sensing to anomaly detection in combined sewer systems

Abstract

Urban drainage systems are a strategic component of cities’ infrastructure, as they safeguard citizens’ health and properties. Most cities in Europe and North America are served by combined sewer systems built to meet the service levels of decades ago. Climate change, intensified urbanization and stricter environmental regulations are straining the existing infrastructure, and large investments are needed for futureproofing. The digitalization of urban drainage offers cost-effective solutions to upgrade the existing system without building new structures or adding new equipment. With a combination of models and data, utility companies can maximize the use of the available capacity and reduce the risk of flooding and pollution. The hydraulic state of combined sewer systems is typically monitored with a network of sensors. These are expensive to install and maintain, therefore they are placed only in strategic locations, leaving large parts of the system unobserved. Moreover, sensor observations are vulnerable to instrument faults, communication errors and cyberattacks. Models of the system can supplement the information from the sensors with a system-wide picture of the hydraulic state. Both physics-based and data-driven models exist capable of reproducing the behavior of sewer systems. If the prediction accuracy is sufficiently high, models can also act as soft(ware) sensors, working alongside hardware sensors or replacing them for periods of time. Model predictions can also be used to validate available observations and detect anomalous behavior. The potential of using models for real-time soft sensing and anomaly detection in combined sewer systems has not been fully exploited yet. Hydrodynamic models describe in detail the spatial and temporal distribution of the hydraulic variables but are computationally expensive and tend to drift off reality. Assimilating observations in real-time allows to update hydrodynamic models to the current state of the system, thus increasing their prediction accuracy. This is usually done by running an ensemble of model instances in parallel, which further increases the computational cost. A data assimilation scheme was developed and tested which limits the assimilation to a sub-system of the larger sewer system. This approach yielded accurate system-wide predictions of water depth, thus effectively enabling the soft sensing capabilities of the hydrodynamic model. The prediction was also used to validate an independent sensor located 3.5 km upstream, revealing an issue of false echo in one of the analysed events. Purely data-driven models offer an alternative to physics-based models. They learn the behavior of the system from a series of historical observations and predict accordingly its response to given inputs. Artificial neural networks have long been used to predict water depths in combined sewer systems, and recent advancements in terms of algorithms and hardware have unlocked new possibilities. Long Short-Term Memory (LSTM) neural networks have gained popularity in natural language processing but are also particularly suited for time series forecasting. However, little is known about their efficacy with water depth observations from combined sewers. An LSTM neural network was trained with several months of water depth observations at 1-min resolution from different locations. The key network settings were calibrated to find an optimal setup that could work across different locations. The prediction accuracy was compared between scenarios with different gaps in the antecedent observations to simulate missing data. It was proven that the model could compensate the missing information on the antecedent state of the system with the other sources of information, namely the observed rainfall and the time of the day. This demonstrated the robustness of LSTM networks and their potential as soft sensing tools. In a separate test, the LSTM prediction was used as basis for anomaly detection. The observations were flagged as anomalous when they deviated significantly from the expected behavior of the system. However, the detection efficacy was dependent on the quality and quantity of the training data and changed across locations. Updated hydrodynamic models can generate accurate system-wide predictions but require detailed physical information. For these reasons, they are suited for soft sensing and anomaly detection in strategic structures within combined sewer systems. On the other hand, LSTM neural networks are trained on observations alone but have point-wise validity. Therefore, they can easily be deployed as a screening tool for large networks of sensors. By investigating and testing updated hydrodynamic models and LSTM neural networks, this thesis demonstrates their concrete potential for soft sensing and anomaly detection applications and promotes their integration in the monitoring and control workflows of modern utility companies.