Multi-class image anomaly detection for practical applications: Requirements and robust solutions

Recent advances in image anomaly detection have expanded unsupervised models from single-class scenarios to multi-class frameworks, mainly to reduce training cost and model storage. However, when a single model simultaneously handles multiple classes, its per-class detection accuracy often falls short compared to class-specific approaches. Accordingly, most prior work has focused on closing this accuracy gap. Despite this effort, one important factor remains overlooked: how the availability of class information influences anomaly detection, particularly in defining detection thresholds. Whether thresholds are class-specific or class-agnostic, this choice plays a critical role in multi-class image anomaly detection. In this study, we identify and formalize the essential requirements that multi-class unsupervised anomaly detection models must satisfy under both known-class and unknown-class evaluation settings. We then re-evaluate existing approaches under these requirements and reveal their limitations. To address these challenges, we propose Hierarchical Coreset (HierCore), a novel framework that can operate without class labels by leveraging a hierarchical memory bank to infer class-wise decision criteria. We conduct extensive experiments to assess the robustness and applicability of both prior methods and our own under varying label conditions. The results show that HierCore consistently satisfies all defined requirements and delivers stable, high detection performance across settings, demonstrating its strong potential for real-world multi-class anomaly detection.