Zero-Shot Industrial Anomaly Segmentation with Image-Aware Prompt Generation

Zero-Shot Anomaly Segmentation by IAP-AS, a model designed to detect anomalies in industrial images, precisely identify their locations, and restore abnormal images to normal using an LLM prompting and zero-shot segmentation techniques.

Requirements & Setup

This codebase utilizes Anaconda for managing environmental dependencies. Please follow these steps to set up the environment:

1. Download Anaconda: Click here to download Anaconda.
2. Clone the Repository: Clone the repository using the following command.

   git clone https://github.com/../IAP-ZSAS.git

3. Install Requirements:
   • Navigate to the cloned repository:

     cd IAP-AS

   • Create a Conda environment from the provided environment.yaml file:

     conda env create -f environment.yaml

   • Activate the Conda environment:

     conda activate iap-as

This will set up the environment required to run the codebase.

Datasets

Below are the details and download links for datasets used in our experiments:

[Dataset details]

All datasets should be stored in the datasets folder, with each dataset placed in a subfolder named after the dataset (e.g., datasets/MPDD/, datasets/btad/, etc.).

1. MVTec-AD (Download): The MVTec AD dataset comprises approximately 5,000 images across 15 classes, including texture-related categories such as fabric and wood.
2. MPDD (Download): MPDD is a dataset designed for visual defect detection in metal part manufacturing. It contains over 1,000 images.
3. BTAD (Download): BTAD (beanTech Anomaly Detection) is a dataset of real-world industrial anomalies, consisting of 2,830 images of three industrial products that exhibit body and surface defects.
4. KSDD1 (Download): The KSDD1 dataset includes 347 normal images and 52 abnormal images, specifically for detecting micro-defects on metal surfaces.
5. MTD (Download): The MTD dataset contains images of magnetic tiles, featuring various types of defects.
6. DTD-Synthetic (Download): DTD-Synthetic is based on the DTD (Describable Texture Dataset) and includes synthesized texture images with anomalies. IIt consists of 47 diverse texture classes.
7. DAGM (Download): The DAGM2007 dataset comprises artificially generated images with characteristics similar to real-world problems. It is divided into 10 datasets: six for algorithm development and four for performance evaluation.

These datasets provide valuable resources for our experiments and each known for their high-resolution, texture-rich images that are well-suited for industrial anomaly segmentation.

Zero-Shot Anomaly Segmentation (ZSAS) TEST

Replace <dataset> with one of the following options: mvtec, MPDD, btad, KSDD, mtd, DTD-Synthetic, DAGM.

Replace <model> with one of the following options: base, iap_zsas.

python test_zsas.py --dataset <dataset name> --model <model name> 

This command excel our proposed model for zero-shot anomaly segmentation(ZSAS) on the specified dataset using the selected model, with best configurations loaded, running 10 epochs each.

Ablation Study TEST

Ablatin study on MVTec-AD texture dataset.

python test_ablation.py --image True --prompt True --filter True 

Optional arguments

  --gpu                             gpu number
  --dataset                         dataset name
  --model                           model name
  --box_threshold                   GroundingSAM box threshold
  --text_threshold                  GroundingSAM text threshold
  --size_threshold                  Bounding-box size threshold
  --iou_threshold                   IoU threshold
  --random_img_num                  random image extraction number
  --eval_resolution                 Description of evaluation resolution
  --exp_idx                         Description of experiment index
  --version                         Description of evaluation version

Special Thanks to

We extend our gratitude to the authors of the following libraries for generously sharing their source code and dataset:

RAM, Llama3, Grounding DINO, SAM, SAA+ Your contributions are greatly appreciated.

Citation