Anomaly detection is a crucial task in the operation of complex systems such as industrial facilities, manufacturing plants, and large-scale science experiments. Failures in a sub-system can result in low yield, faulty products, or damage to components, making it essential to detect anomalies as quickly as possible. Despite the abundance of data available for complex systems, labeled anomalies are rare and expensive to obtain. To address this issue, we present a novel approach called CoAD that trains anomaly detection models on unlabeled data by leveraging the correlation between sub-systems and products. CoAD works by analyzing two data streams, s and q, which represent subsystem diagnostics and final product quality, respectively. We define an unsupervised metric, akin to the supervised classification F_beta statistic, to assess the performance of independent anomaly detection algorithms on s and q based on their coincidence rate. Our method is demonstrated in four cases, including a synthetic outlier data set, a synthetic imaging data set generated from MNIST, a metal milling data set, and a data set obtained from a particle accelerator. By using CoAD, we can detect anomalies in complex systems more effectively, even when labeled anomalies are scarce.

Email jnespos@seas.upenn.edu for the Zoom link.